Cleaning management program

Abstract

The present invention facilitates creation of a cleaning-area map in cleaning.　A cleaning management program for causing a user terminal, which is a computer attached to a terminal holder of a cleaner, to function as an input unit that receives a plurality of designated positions from a user; and a cleaning-area map creating unit that creates a cleaning-area map by line segments connecting the designated positions, in which the cleaning-area map creating unit controls execution of an end instruction for completing the cleaning-area map and determination of self-intersection between a final line segment specified in accordance with the end instruction among the line segments and the other line segments in cooperation with each other.

Description

Cleaning Management Program

[0001] The present invention relates to so-called home appliances, particularly vacuum cleaners, and information processing technology for managing the same.

[0002] Currently, household appliances such as vacuum cleaners are widely used for housework. So-called IoT (Internet of Things) appliances have been proposed for these appliances. IoT appliances connect to networks such as the Internet and provide various functions and services, such as operational control. For example, Patent Document 1 describes the following: "A vacuum cleaner system 1 includes a vacuum cleaner 2 and an information terminal device 3. The vacuum cleaner 2 includes a suction nozzle 13, a fan 14 that sucks dust through the suction nozzle 13, a dust sensor 15 that detects dust sucked through the suction nozzle 13 by the fan 14, and a (first) transmitter that transmits detection data from the dust sensor to a predetermined destination. The information terminal device 3 includes a receiver that receives the detection data from the (first) transmitter, a notification unit that issues a notification based on the detection data received by the receiver, and a (second) transmitter that transmits the detection data received by the receiver to a predetermined server" (abstract).

[0003] Patent Publication No. 2021-176423

[0004] Here, housework, including cleaning, is time-consuming, so various support functions are required. For this reason, there is a need to support housework by visualizing the housework that has been done and the housework that needs to be done. In particular, there is a need to more easily create a cleaning area map that serves as a guide for cleaning.

[0005] Therefore, the present invention aims to realize various functions and services related to practical home appliances, particularly vacuum cleaners, that can assist users with housework such as cleaning, and in particular to more easily create a cleaning area map for cleaning.

[0006] In order to solve the above problems, the present invention performs processing for visualizing housework. In a representative embodiment, the present invention is a cleaning area map showing a cleaning area, in which a plurality of line segments connecting vertices at specified positions are identified, and the cleaning area map is created using an area (e.g., a polygon) defined by the identified plurality of line segments. The present invention controls execution of an end instruction for completing the cleaning area map and a determination of self-intersections (also called self-crossings) between the final line segment identified in response to the end instruction and other line segments in cooperation with each other.

[0007] More specifically, this is a cleaning management program that causes a user terminal, which is a computer attached to the terminal holder of a vacuum cleaner, to function as an input unit that accepts multiple specified positions from the user and a cleaning area map creation unit that creates a cleaning area map using lines connecting the specified positions, and the cleaning area map creation unit is a cleaning management program that controls the execution of an end instruction to complete the cleaning area map and the determination of self-intersections between the final line segment identified among the lines in response to the end instruction and other line segments in cooperation with each other.

[0008] Furthermore, the present invention also includes the above-mentioned cleaning management program, a storage medium for storing the program, information processing devices such as a user terminal or a cleaning management device, and a cleaning management method executed by the cleaning management system and each device or subsystem that constitutes the system.

[0009] According to the present invention, a cleaning area map for cleaning can be created more easily.

[0010] 1 is a perspective view showing a state in which a stick vacuum cleaner 2 according to one embodiment of the present invention is stored in a charging base in a stick state. FIG. 2 is a diagram showing a state in which a user terminal 1 is installed on a stick vacuum cleaner 2 according to one embodiment of the present invention. FIG. 3 is a functional block diagram of a user terminal 1 according to one embodiment of the present invention. FIG. 4 is a hardware configuration diagram of a user terminal 1 according to one embodiment of the present invention. FIG. 5 is a diagram showing user management information 112 used in one embodiment of the present invention. FIG. 6 is a diagram showing cleaning management information 113 used in one embodiment of the present invention. FIG. 7 is a diagram showing cleaning area map information 114 used in one embodiment of the present invention. FIG. 8 is a diagram showing guidance information 115 used in one embodiment of the present invention. FIG. 9 is a flowchart showing a processing flow for cleaning according to one embodiment of the present invention. FIG. 10 is a diagram showing a display screen 1101 of an output unit 110 displaying terms of use according to one embodiment of the present invention. FIG. 11 is a diagram showing a display screen 1102a of an output unit 110 for registering a stick vacuum cleaner 2 according to one embodiment of the present invention. FIG. 12 is a diagram showing a display screen 1102b of an output unit 110 for registering a stick vacuum cleaner 2 according to one embodiment of the present invention. FIG. 13 is a diagram showing a display screen 1102c of an output unit 110 for registering a stick vacuum cleaner 2 according to one embodiment of the present invention. 1 is a diagram showing a display screen 1103a of the output unit 110 for registering a related party in Example 1. FIG. 2 is a diagram showing a display screen 1103b of the output unit 110 for registering a related party in Example 1. FIG. 3 is a diagram showing a display screen 1103c of the output unit 110 for registering a related party in Example 1. FIG. 4 is a diagram showing a display screen 1104 of the output unit 110 for displaying usage instructions in Example 1. FIG. 5 is a diagram showing a display screen 1105a (first example) for explaining the creation of a map in Example 1. FIG. 6 is a diagram showing a display screen 1105b (second example) for explaining the creation of a map in Example 1. FIG. 7 is a diagram showing a display screen 1105c (second example) for explaining the creation of a map in Example 1. FIG. 8 is a diagram showing a display screen 1106a (first example) for creating a map in Example 1. FIG. 9 is a diagram showing a display screen 1106b (first example) for creating a map in Example 1. FIG. 10 is a diagram showing a display screen 1106c (first example) for creating a map in Example 1. FIG. 11 is a diagram showing a display screen 1106d (first example) for creating a map in Example 1.1 is a diagram showing a display screen 1106e (first example) for map creation in Example 1. FIG. 2 is a diagram showing a display screen 1106f (second example) for map creation in Example 1. FIG. 3 is a diagram showing a display screen 1106g (second example) for map creation in Example 1. FIG. 4 is a diagram showing a home screen 1107a (first example) in Example 1. FIG. 5 is a diagram showing a home screen 1107b (first example) in Example 1. FIG. 6 is a diagram showing a home screen 1107c (second example) in Example 1. FIG. 7 is a diagram showing a map selection screen 1108a (first example) in Example 1. FIG. 8 is a diagram showing a terminal attachment instruction screen 1108b (first example) in Example 1. FIG. 9 is a diagram showing a player map selection screen 1108c (second example) in Example 1. FIG. 10 is a diagram showing a player map selection result screen 1108d (second example) in Example 1. FIG. 11 is a flowchart showing details of a cleaning-related information display process (step S22) in Example 1. FIG. 12 is a diagram showing a cleaning-related information display screen 1109a (using a game function) in Example 1. 1 is a diagram showing a display screen 1109b (using the game function) for cleaning-related information in Example 1. FIG. 2 is a diagram showing a display screen 1109c (using the game function) for cleaning-related information in Example 1. FIG. 3 is a diagram showing a display screen 1109d (using the game function) for cleaning-related information in Example 1. FIG. 4 is a diagram showing a display screen 1110a (not using the game function) for cleaning termination processing in Example 1. FIG. 5 is a diagram showing a display screen 1110b (not using the game function) for cleaning termination processing in Example 1. FIG. 6 is a diagram showing a display screen 1110c (not using the game function) for cleaning termination processing in Example 1. FIG. 7 is a diagram showing a display screen 1110d (not using the game function) for cleaning termination processing in Example 1. FIG. 8 is a flowchart showing details of the display processing (step S22) for cleaning-related information in Example 2. FIG. 9 is a diagram showing a display screen 1111a (using the game function) for cleaning-related information in Example 2. FIG. 10 is a diagram showing a display screen 1111b (using the game function) for cleaning-related information in Example 2. FIG. 11 is a diagram showing a display screen 1112a (not using the game function) for cleaning-related information in Example 2. Fig. 11 is a diagram showing a display screen 1112b of cleaning-related information (without using a game function) in Example 2. Fig. 12 is a system configuration diagram of a cleaning management system in Example 3.31 is an external view showing the state in which the terminal holder 20 is attached to the stick vacuum cleaner 2 in Example 4. FIG. 32 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 33 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 34 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 35 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 36 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 37 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 5 are connected. FIG. 38 is an external view showing the state in which the terminal holder 20 and the handy brush 25 in Example 6 are connected. 40A is a side view showing a state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. FIG. 40B is a top view showing a state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. FIG. 40C is a bottom view showing a state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. FIG. 40D is a rear view showing a state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. FIG. 40E is a cross-sectional view taken along the line CC of FIG. 40A in Example 6. FIG. 40F is a diagram for explaining a map creation process in Example 7. FIG. 40G is a flowchart for explaining details of a display process in Example 8. FIG. 40H is a flowchart for explaining details of a display process in Specific Example 1 of Example 8. FIG. 40H is a diagram showing a display screen 1113a in Specific Example 1 of Example 8. FIG. 40H is a diagram showing a display screen 1113b in Specific Example 1 (Modified Example) of Example 8. FIG. 40I is a diagram showing transitions of display screens 1113c to 1113e in Specific Example 1 (Modified Example) of Example 8. FIG. 40I is a diagram showing a display screen 1113a including a peripheral display area in Specific Example 1 of Example 8. FIG. 20 is a diagram illustrating an example of controlling whether or not to display the operating map icon 1114a-1 in the eighth embodiment.11 is a diagram showing a drive instruction display screen in Example 8. FIG. 11 is a diagram showing a display screen 1115a with a cleaning end display in Example 8. FIG. 11 is a diagram showing a display screen 1116a with a cleaning end display including points (number of icons) in Example 8. FIG. 11 is a flowchart showing a processing flow of a map creation process in Example 9. FIG. 11 is a flowchart showing a processing flow of a cleaning management process (normal mode) in Example 9. FIG. 11 is a flowchart showing a processing flow of a cleaning management process (simple mode) in Example 9. FIG. 11 is a diagram showing the concept of trajectory drawing in Example 9. FIG. 11 is a diagram showing a display screen 1106h on which a movement trajectory is drawn in Example 9. FIG. 11 is a diagram showing a cleaning area map in Example 10. FIG. 11 is a flowchart showing a cleaning area map creation process in Specific Example 1 of Example 10. FIG. 11 is a flowchart showing a cleaning area map creation process in Specific Example 2 of Example 10. FIG. 11 is a flowchart showing a cleaning area map creation process in Specific Example 3 of Example 10. FIG. 11 is a diagram showing distortion of a cleaning area map 1106i-1 in Example 11. FIG. 11 is a diagram showing a downward rightward slope (distortion) of the upper side of a cleaning area map 1106i-1 in Example 11. 11 is a diagram showing a state where the downward rightward slope (distortion) of the upper side of the cleaning area map 1106i-1 in Example 11 has been corrected. FIG. 12 is a diagram showing a grid map 1106i-1a in Example 11. FIG. 13 is a diagram showing a dot map 1106i-1b in Example 11. FIG. 14 is another diagram showing a state where the downward rightward slope (distortion) of the upper side of the cleaning area map 1106i-1 in Example 11 has been corrected. FIG. 15 is a diagram showing a display screen 1106j of cleaning-related information in Example 11. FIG. 16 is a diagram showing a display screen 1106k of cleaning-related information in Example 11. FIG. 17 is a diagram showing a display screen 1106l (part 1) for adjusting a movement trajectory in Example 12. FIG. 18 is a diagram showing a display screen 1106l (part 2) for adjusting a movement trajectory in Example 12. FIG. 19 is a diagram showing a display screen 1106l (part 3) for adjusting a movement trajectory in Example 12. FIG. 19 is a diagram showing a display screen 1106l (part 4) for adjusting a movement trajectory in Example 12. FIG. 19 is a diagram showing a display screen 1106m during cleaning management processing in Example 13. FIG. 23 is a diagram for explaining the arrangement of the virtual camera in Example 13.13 is a diagram for explaining the arrangement of the virtual camera in the height direction in Example 13. FIG. 14 is a diagram for explaining a display screen 1106m in which an actual trajectory 1106m-3 protrudes from a real map 1106m-2 in Example 13. FIG. 15 is a diagram for explaining determination of protrusion of the actual trajectory in Example 13. FIG. 16 is a diagram for explaining adjustment processing for a protruding actual trajectory in Example 13. FIG. 17 is a diagram for explaining adjustment processing for a protruding actual trajectory in Example 13. FIG. 18 is a diagram for explaining adjustment processing for a display screen 1106n for switching between a game mode and a normal mode. FIG. 19 is a diagram (part 1) showing screen transitions of the display screen 1106o during map creation processing. FIG. 20 is a diagram (part 2) showing screen transitions of the display screen 1106o during map creation processing. FIG. 21 is a diagram (part 3) showing screen transitions of the display screen 1106o during map creation processing. FIG. 19 is a diagram (part 1) showing screen transitions of the display screen 1106p for cleaning after map creation processing. FIG. 20 is a diagram (part 2) showing screen transitions of the display screen 1106p for cleaning after map creation processing. FIG. 11 is a diagram (part 3) showing screen transitions of a display screen 1106p for cleaning after the map creation process.

[0011] An embodiment of the present invention will be described below. In this embodiment, a stick vacuum cleaner, which is an example of a manual vacuum cleaner operated by a user, will be used as the vacuum cleaner. Furthermore, the stick vacuum cleaner of this embodiment may or may not have a communication function with other devices such as a smartphone or a router. A configuration for realizing cleaning management processing related to such a stick vacuum cleaner will be described below.

[0012] 1 is a perspective view showing a stick vacuum cleaner 2 according to this embodiment stored in a charging stand 26 in a stick state. The stick vacuum cleaner 2 comprises a cleaner body 21, a handle 22, an extension tube 23, and a cleaner head 24. The cleaner body 21 and the cleaner head 24 are connected via the extension tube 23, which is a pipe through which sucked dust passes.

[0013] The vacuum cleaner body 21 is also provided with a handle 22 that the user grips. A handy brush 25 can be attached to the vacuum cleaner body 21 below the handle 22, and a vacuum cleaner terminal holder (hereinafter referred to as the terminal holder 20) is connected to the handy brush 25. The terminal holder 20 holds a user terminal 1 such as a smartphone, and may be configured to be detachable from the vacuum cleaner body 21 or may be configured as an integrated unit. Attaching the terminal holder 20 and stick vacuum cleaner 2 using the handy brush 25 is just one example, and may be realized as a separate configuration, as in Example 4 described below. In this embodiment, at least a portion of the cleaning management process is executed by the user terminal 1 held in the terminal holder 20. This function (processing flow) will be described in each example.

[0014] The stick vacuum cleaner 2 can be changed into various states for cleaning, such as a handheld state or a stick state. The charging base 26 in which the stick vacuum cleaner 2 is stored stores the stick vacuum cleaner 2 in a stick state, and is configured with a base member 29, three stand members 28, and a holder member 27. The charging base 26 can charge the stick vacuum cleaner 2.

[0015] The stick vacuum cleaner 2 may be provided with a label that records identification information for identifying itself and that can be photographed and read by the user terminal 1. For example, a label such as an AR marker or nameplate may be provided on the cleaner head 24 that faces the cleaning surface, such as the floor. This label records identification information for identifying the stick vacuum cleaner 2. The identification information may be information that individually identifies the stick vacuum cleaner 2, or information that identifies its type, such as its model. The following describes how the user terminal 1 is installed in the terminal holder 20 connected to the handheld brush 25.

[0016] FIG. 2 is a diagram showing the user terminal 1 installed on the stick vacuum cleaner 2 of this embodiment. In this embodiment, as shown in FIG. 2 , the user terminal 1 is held in a terminal holder 20 connected to a handheld brush 25. The handheld brush 25 is connected to the vacuum cleaner main body 21. In this case, the back of the user terminal 1 faces the cleaner head 24, and the screen side faces the user's head. As a result, the user can check cleaning information displayed on the user terminal 1. For example, AR information showing the cleaning status of the stick vacuum cleaner 2 and a cleaning area map showing a unit cleaning area such as a room or floor are displayed. More preferably, these are displayed on a single screen. As an example of this display on a single screen, the cleaning area map is superimposed on the AR information. In this way, a unit cleaning area is an area that constitutes a cleaning area to be cleaned, such as a house, and the unit cleaning area may be a cleaning area.

[0017] Furthermore, when a sign is provided, the user terminal 1 is held in the terminal holder 20 so that the user terminal 1's camera can capture the sign. It is desirable that the orientation of the terminal holder 20 is variable, and that the direction in which the user terminal 1 faces can also be changed accordingly. Furthermore, the camera of the user terminal 1 captures an image of the floor surface to be cleaned by the stick vacuum cleaner 2.

[0018] In this embodiment, the cleaning management process is executed based on conditions such as the user's operation on the user terminal 1 and an identification process using the sign identification information captured by the user terminal 1. That is, the display of the above-mentioned AR information and cleaning area map is executed. In the following Examples 1 to 3, which are specific examples of this embodiment, the display process of the AR information and cleaning area map will be described in detail. Furthermore, in Examples 4 to 6, the structure of the terminal holder 20 will be described in detail. Furthermore, in Example 7, a modified example of the map creation process will be described. Furthermore, in Example 8, an example of the display format of AR information and cleaning-related information including AR icons and an example of dynamically changing the display format will be described. Note that it is possible to combine at least two of each example.

[0019] Example 1 is an example in which AR information and a cleaning area map are displayed as part of a cleaning management process using a single user terminal 1. Example 1 also illustrates an example in which cleaning can be performed like a game. FIG. 3 is a functional block diagram of the user terminal 1 in Example 1. The user terminal 1 includes a communication unit 101, an imaging unit 102, a cleaner identification unit 103, an operation status detection unit 104, a cleaning area map creation unit 105, an AR information creation unit 106, a cleaning management unit 107, a history information creation unit 108, an input unit 109, an output unit 110, and a memory unit 111.

[0020] The communication unit 101 has a function of communicating via a router or a network and a function of short-distance wireless communication. The photographing unit 102 is realized by a camera or the like and captures an image of the surroundings.

[0021] Furthermore, the vacuum cleaner identification unit 103 identifies the stick vacuum cleaner 2 using the image captured by the imaging unit 102. In Examples 1 and 2, the image of the sign unit captured by the imaging unit 102 is used to identify the product information recorded on the sign unit, and this is used to identify the vacuum cleaner. Note that an AR (Augmented Reality) marker, a two-dimensional code, or a nameplate can be used as the sign unit. Here, the nameplate records the product information and a two-dimensional code in which this information is recorded in a visible format. Note that the vacuum cleaner identification unit 103 may perform identification by user operation or by communicating with the stick vacuum cleaner 2. In the former case, product information for one or more stick vacuum cleaners 2 is stored in advance in the memory unit 111 of the user terminal 1, and the product information is identified by the user selecting from this.

[0022] Alternatively, the label unit may be omitted and the stick vacuum cleaner 2 may be identified using the shape of the vacuum cleaner head 24 photographed by the photographing unit 102. If the shape of the vacuum cleaner head 24 is used, the vacuum cleaner identification unit 103 will identify the type of stick vacuum cleaner 2. Furthermore, an RFID (Radio Frequency Identification) tag on which identification information is recorded may be used as the label unit. In this case, an RFID reader is provided in the user terminal 1 to read the identification information.

[0023] Furthermore, the operation status detection unit 104 detects the operation status of the stick vacuum cleaner 2, such as identifying cleaned positions, including the movement trajectory of the stick vacuum cleaner 2, using images of the floor surface captured by the imaging unit 102. For example, the operation status detection unit 104 calculates a coverage rate, which indicates the ratio of cleaned positions to a unit cleaning area, as the operation status. Furthermore, the AR information creation unit 106 creates AR information using images of the floor surface, etc., captured by the imaging unit 102. More preferably, the AR information creation unit 106 creates AR information using the cleaned area identified by the operation status detection unit 104. Here, the AR information is information created using augmented reality technology and is information indicating the cleaning status of the cleaned area, etc. It is desirable that cleaning-related information including the AR information be provided to relevant parties, including users. Here, the cleaning-related information may be the AR information itself, or additional information may be added to the AR information. Furthermore, the AR information may be a captured image (including a still image and a video), an animation showing the cleaned area, or text showing the cleaned area.

[0024] The AR information creation unit 106 also creates AR information including an AR icon whose display is controlled in accordance with cleaning performed with the stick vacuum cleaner 2. Furthermore, the AR information creation unit 106 creates AR information that distinguishes between positions that have been cleaned with the stick vacuum cleaner 2 and positions that have not been cleaned. The cleaning management unit 107 then displays the boundary between the cleaned positions and the uncleaned positions using a gradation.

[0025] The cleaning management unit 107 also executes cleaning management processing for cleaning performed by the stick vacuum cleaner 2 and preparation processing for the same. First, as part of the cleaning management processing, the cleaning management unit 107 controls, i.e., causes the output unit 110 to display cleaning-related information including AR information and a cleaning area map. At this time, the cleaning management unit 107 causes the AR information and cleaning area map to be displayed on one screen of the output unit 110. At this time, the cleaning management unit 107 displays AR information including AR icons. Then, the cleaning management unit 107 displays map icons, which are displayed in conjunction with the display of the AR icons, in the cleaning area map. Furthermore, the cleaning management unit 107 changes the shapes of the AR icons and various map icons.

[0026] Furthermore, the preparation process includes the cleaning management unit 107 displaying an attachment instruction to the terminal holder 20 of the user terminal 1, in other words, an attachment instruction to the stick vacuum cleaner 2 and an attachment confirmation button on the output unit 110. Furthermore, as part of the preparation process, the cleaning management unit 107 displays on the output unit 110 an instruction to select a cleaning area map of a unit area to be cleaned from a plurality of cleaning area maps stored in the memory unit 111 described below.

[0027] Furthermore, the history information creation unit 108 creates history information indicating the cleaning history of the stick vacuum cleaner 2 using the operating status of the stick vacuum cleaner 2 detected by the operating status detection unit 104. In this embodiment, cleaning results in the cleaning management information 113 are created as the history information. The cleaning management unit 107 then displays the history information on the output unit 110. At this time, it is preferable that the cleaning management unit 107 displays the history information on the same screen as the cleaning-related information. Furthermore, the cleaning management unit 107 can also display the history information for a predetermined period of time. Furthermore, the history information creation unit 108 creates multiple pieces of history information for each cleaning area map, and the cleaning management unit 107 displays two or more pieces of the multiple pieces of history information in an overlapping manner on the output unit 110. Then, the cleaning management unit 107 switches between the two or more pieces of history information and displays them depending on the tab selected. Note that the history information includes cleaning results including the movement trajectory of the vacuum cleaner.

[0028] The input unit 109 also accepts various operations from the user. The output unit 110 also outputs AR information, a cleaning area map, and the like. The input unit 109 and the output unit 110 may be integrated into one unit, such as a touch panel. The storage unit 111 also stores user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116. These pieces of information will be described after the implementation example of the user terminal 1.

[0029] Next, an implementation example of the user terminal 1 will be described. The user terminal 1 can be realized by various computers such as a smartphone, tablet, mobile phone, or PC. Fig. 4 is a hardware configuration diagram of the user terminal 1 in Example 1. In Fig. 4, the user terminal 1 has a camera 11, a touch panel 12, a processing device 13, a communication device 14, and a storage device 15, which are connected to each other via a communication path.

[0030] First, the camera 11 is an example of the imaging unit 102 and has an imaging function. The touch panel 12 serves both as the input unit 109 and the output unit 110 shown in FIG. 3 , and receives user operations and displays various information such as AR information and a cleaning area map. The touch panel 12 may be configured as an input device and an output device.

[0031] The processing device 13 can be realized by a processor such as a CPU (Central Processing Unit), and executes calculations according to a cleaning management program 16 stored in a storage device 15, which will be described later.

[0032] The cleaning management program 16 is composed of a cleaner identification module 161, an operating status detection module 162, a cleaning area map creation module 163, an AR information creation module 164, a cleaning management module 165, and a history information creation module 166, each for one of its functions. Each of these modules may be implemented as an individual program or a partial combination. In particular, the AR information creation module 164 may be implemented as a program with an augmented reality function separate from the home appliance management program. Furthermore, the cleaning management program 16 may be implemented as one function of an integrated management application that manages multiple home appliances.

[0033] 3, which perform the same functions as each module, is as follows: vacuum cleaner identification module 161: vacuum cleaner identification unit 103; operating status detection module 162: operating status detection unit 104; cleaning area map creation module 163: cleaning area map creation unit 105; AR information creation module 164: AR information creation unit 106; cleaning management module 165: cleaning management unit 107; history information creation module 166: history information creation unit 108. Therefore, the processing device 13 executes the processes of the vacuum cleaner identification unit 103, operating status detection unit 104, cleaning area map creation unit 105, AR information creation unit 106, cleaning management unit 107, and history information creation unit 108 in accordance with the cleaning management program 16. The cleaning management program 16 can be stored in the storage device 15 (described later) or other storage media.

[0034] The communication device 14 corresponds to the communication unit 101 in Fig. 3 and has the function of communicating via a router or a network and performing short-range wireless communication. The storage device 15 corresponds to the storage unit 111 in Fig. 3 and stores the cleaning management program 16, user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116. Therefore, the storage device 15 may be realized by a main storage device such as a memory and a secondary storage device (storage medium) that is a so-called storage. The secondary storage device may be realized by an external hard disk drive (HDD), solid state drive (SSD), memory card, etc.

[0035] Here, the user management information 112, cleaning management information 113, cleaning area map information 114, guidance information 115, and game information 116 will be described. FIG. 5 is a diagram showing the user management information 112 used in Example 1. The user management information 112 is information about the user of the user terminal 1 and related parties. Therefore, as shown in FIG. 5, the user management information 112 has the following fields: user / related parties, points, address, contact information, and owned vacuum cleaner. Here, the user / related parties are information that identifies the user, such as the name and membership number of the user or related party. Furthermore, the points indicate the points or score that the user or related party has earned through the game function. Note that the points may be stored as cleaning management information 113, which will be described later. In this case, the cleaning performance field is managed for each user or related party, and the points are stored corresponding to the user or related party.

[0036] The address indicates the address of the user. The contact information indicates the contact information of the user, and may be a telephone number, email address, or the like. The owned vacuum cleaner is information that identifies the home appliance used by the user or related parties, i.e., the vacuum cleaner. Here, the user management information 112 can be used to manage and authenticate users in the cleaning management program 16. The user management information 112 may be omitted.

[0037] 6 is a diagram showing cleaning management information 113 used in Example 1. The cleaning management information 113 is information for managing the cleaning results of the stick vacuum cleaner 2. Therefore, as shown in Fig. 6, the cleaning management information 113 has the following items for each vacuum cleaner: user / person involved, date and time, AR information, and cleaning results (cleaned positions).

[0038] Here, the vacuum cleaner identifies the vacuum cleaner used by the user and is the target for displaying the presentation information, and product information such as identification information can be used. In Example 1, a stick vacuum cleaner 2 is recorded. Note that if a user uses multiple vacuum cleaners, there will be multiple records in the cleaning management information 113.

[0039] Furthermore, the vacuum cleaner ID is identification information that identifies the vacuum cleaner in question. This vacuum cleaner ID is, for example, information equivalent to the identification information recorded in the label unit. Furthermore, the user / related parties refer to the user and related parties who perform the cleaning and who may be the recipients of the presented information. Typical examples of these users and related parties include the family or cohabitants of the user of the user terminal 1 (A in the figure) who is the main cleaner. Note that if a commercial vacuum cleaner is used as the target vacuum cleaner, the user's colleagues, managers, customers, etc. can be registered. However, these related parties are merely examples, and any person can be registered by the user's operation, etc. Furthermore, when it comes to users / related parties, it is desirable to distinguish users from other related parties. Note that related parties may be limited to users.

[0040] The date and time indicates the date and time when cleaning was performed or the date and time when the AR information was created. The AR information indicates a log of AR information created by the AR information creation unit 106. Note that this item may be omitted due to storage capacity limitations. The cleaning history (cleaned position) indicates the position cleaned by the corresponding stick vacuum cleaner 2 based on the AR information. Therefore, the cleaning history (cleaned position) has the items of coordinates and floor. Here, the coordinates are coordinates that indicate the area cleaned by the stick vacuum cleaner 2, i.e., the trajectory of movement. Furthermore, the floor indicates the floor that has been cleaned (or the floor that has not been cleaned) according to the coordinates. Here, the floor is an example of a unit cleaning area, and a room may also be used.

[0041] 7 is a diagram showing cleaning area map information 114 used in Example 1. The cleaning area map information 114 is map information that defines the layout of a place of use where the stick vacuum cleaner 2 is used, such as a house or an office, i.e., each unit cleaning area that makes up the place of use. For this reason, the cleaning area map information 114 has items for floor and coordinates corresponding to the place of use. The floor is an example of a unit cleaning area, and the coordinates are items that indicate the coordinates of the corresponding floor, i.e., location information. Each record in the cleaning area map information 114 indicates a cleaning area map for each unit cleaning area, such as a floor or room.

[0042] FIG. 8 is a diagram illustrating guidance information 115 used in the first embodiment. The guidance information 115 is information indicating various types of guidance, such as terms of use and guidance on how to use the device, including user operations, for executing the cleaning management process and preparation process in the first embodiment. For this purpose, the guidance information 115 indicates the operation details for each function for executing the cleaning management process and preparation process. Therefore, when the functions of the first embodiment are implemented by an app, the functions can be implemented as information related to the app. In the first embodiment, the functions used are terms of use, map creation, cleaning (display of cleaning-related information), cleaning track (movement track), history information, maintenance, and support. Each of these functions is associated with a guidance content. Note that the cleaning track (movement track) and history information may be treated as a single function.

[0043] The guidance for each function will now be described. The terms of use stipulate terms of use that require consent from the user upon first use. Map creation indicates the procedure for creating cleaning area map information 114 for each unit cleaning area. Cleaning (display of cleaning-related information) indicates the procedure for displaying cleaning-related information when cleaning. The cleaning movement trajectory and history information indicate the procedure for displaying the cleaning movement trajectory and cleaning history, which are examples of cleaned positions, as cleaning-related information. Maintenance indicates the procedure for performing various maintenance tasks, such as maintenance, replacing consumables, and repairs. Support indicates the procedure for receiving support for repairs and purchasing consumables from the manufacturer or retailer of the stick vacuum cleaner 2. Each of these functions may be a manual for the corresponding vacuum cleaner, i.e., information summarizing how to use the vacuum cleaner, or a separate manual may be recorded.

[0044] The game information 116 is information used to realize game functions used in cleaning. The game information 116 includes AR icons used in the game, such as characters, point calculation rules, and AR icon placement rules. In this embodiment, the cleaning management unit 107 realizes the game functions, but a separate game unit (program) may also realize the game functions.

[0045] Next, a process flow for cleaning with the stick vacuum cleaner 2 in Example 1 will be described. Here, a process flow including a preparation process and a cleaning management process for the above-mentioned cleaning will be described. Figures 9A and 9B are flowcharts showing a process flow for cleaning in Example 1. Below, the creation of a cleaning area map as a preparation process will be described, and the configuration of the user terminal 1 will be described using the configuration shown in Figure 3.

[0046] First, in step S1 of Fig. 9A, the input unit 109 accepts input from the user and activates the configuration for executing this processing flow. For example, the cleaning management program 16 in Fig. 4 is activated. At this time, the user places the user terminal 1 in the terminal holder 20. Then, in step S2, the cleaning management unit 107 determines whether the configuration for executing this processing flow, for example, the cleaning management program 16, is being used for the first time. If it is being used for the first time, the process proceeds to step S3. If it is being used for the second or subsequent time, the process proceeds to step S14.

[0047] In step S3, the cleaning management unit 107 reads the contents of the terms of use from the guidance information 115 and displays them on the output unit 110. FIG. 10 is a diagram showing a display screen 1101 of the output unit 110 that displays the terms of use in Example 1. The display screen 1101 displays the guidance contents of the terms of use. Below the display screen 1101, a checkbox that the user can enter if they confirm the contents and an "Agree" button that the user can press (designate) if they agree are displayed. When these are entered and pressed, the process transitions to step S4. Below these, text indicating "Do not agree" is displayed. When this button is pressed, the cleaning management unit 107 terminates the app (cleaning management program 16), that is, this processing flow. When this processing flow is terminated, the cleaning management unit 107 may display a pop-up on the display screen 1101 to confirm whether to terminate, and require the user to input their consent to the termination.

[0048] Furthermore, in step S4, the cleaning management unit 107 registers the user and the stick vacuum cleaner 2. To do this, the cleaning management unit 107 receives information identifying the user and the stick vacuum cleaner 2 from the user via the input unit 109. The cleaning management unit 107 then registers this information as the user and the owned vacuum cleaner in the user management information 112. Furthermore, the cleaning management unit 107 registers this identifying information as the related party and the vacuum cleaner in the cleaning management information 113. Here, it is desirable that the user and other related parties be recorded separately.

[0049] Note that the registration of the stick vacuum cleaner 2, i.e., the owned vacuum cleaner, may be performed using the imaging unit 102. The details of this will be explained below with reference to Figures 11A to 11C. First, the cleaning management unit 107 activates the imaging unit 102 and causes an instruction to read the marker to be displayed on the display screen 1102a of the output unit 110. That is, the display screen 1102a of the output unit 110 shown in Figure 11A for registering the stick vacuum cleaner 2 is displayed. The display screen 1102a displays instruction text 1102a-1 indicating an instruction to read the two-dimensional code on the product nameplate, which is an example of a marker.

[0050] Then, in accordance with instruction text 1102a-1, the user moves user terminal 1 and photographs the two-dimensional code recorded on the nameplate with imaging unit 102. In Fig. 11A, the photographed image is displayed in imaging area 1102a-2. Also in Fig. 11A, supplemental text 1102a-3 is linked to information indicating the position of the product nameplate, which is an example of a sign. Then, vacuum cleaner identification unit 103 recognizes product information that identifies the vacuum cleaner from the photographed two-dimensional code.

[0051] Furthermore, when the two-dimensional code in the photographed area 1102a-2 is read and the product information of the vacuum cleaner is recognized, the cleaning management unit 107 causes the output unit 110 to display a display screen 1102b. This display screen 1102b is shown in FIG. 11B . Display screen 1102b displays registration confirmation text 1102b-1, which requests confirmation of product information 1102b-2 recognized from the photographed two-dimensional code. Also, above this, product information 1102b-2 recognized from the two-dimensional code is displayed. This product information 1102b-2 can be information identifying the stick vacuum cleaner 2, such as an individual identification number, as well as the model and product name. Furthermore, below this, an OK button 1102b-3 is displayed, which the user presses if they wish to register the product information 1102b-2. On condition that this OK button 1102b-3 is pressed, the cleaning management unit 107 registers the displayed product information 1102b-2 in the owned vacuum cleaners of the user management information 112 and the vacuum cleaners of the cleaning management information 113.

[0052] Furthermore, if the product information cannot be recognized because the two-dimensional code in the photographing area 1102a-2 cannot be read, for example, the cleaning management unit 107 causes the output unit 110 to display a display screen 1102c. This display screen 1102c is shown in FIG. 11C. Display screen 1102c displays unconfirmed text 1102c-1, indicating that the target product, i.e., the vacuum cleaner, cannot be confirmed. Also, below that, an unconfirmed icon 1102c-2, symbolically indicating that the vacuum cleaner cannot be confirmed, is displayed. Also, below that, a reread button 1102c-3 and an end button 1102-4 are displayed. When the user presses the reread button 1102c-3, the cleaning management unit 107 causes the display screen 1102a to be displayed again, and the photographing unit 102 captures the two-dimensional code. Also, when the end button 1102-4 is pressed, the cleaning management unit 107 terminates the app (cleaning management program 16), i.e., this processing flow. When this processing flow is to be ended, the cleaning management unit 107 may display a pop-up on the display screen 1101 to confirm whether to end the processing flow, and require the user to input consent to the end of the processing flow.

[0053] Furthermore, when registering a user, related parties can be registered in addition to the user. Here, the registration of the user and other related parties will be described. In this example, related parties such as family members are registered as players so that cleaning can be done like a game. First, the cleaning management unit 107 causes the output unit 110 to display a display screen 1103 for registering a player, which is an example of a related party. An example of this display screen is shown in FIG. 12A. In FIG. 12A, the display screen 1103a displays explanatory text 1103a-1 encouraging registration. In this figure, since this is the first use, a sentence encouraging the registration of the first player to play is displayed. Note that this first player corresponds to the user described above.

[0054] Below this, a player icon area 1103a-2 is displayed, which displays the player icon of the player. In FIG. 12A, the player icon is blank because it has not yet been registered. Below the player icon area 1103a-2, a player name area 1103a-3 is displayed, which displays the player name entered by the user via the input unit 109. In FIG. 12A, "Sonotaro" is displayed as the player name. Below the player name area 1103a-3, text is displayed prompting the user to enter the player name, i.e., the user's name. The player name is not limited to a given name, and may include a family attribute such as "father" or an account name for another game.

[0055] Below these, a decision button 1103a-4 and a hold button 1103a-5 are displayed. When the decision button 1103a-4 is pressed, the cleaning management unit 107 transitions the display screen 1103a to a display screen 1103b for registering a player icon. At this time, the cleaning management unit 107 may register the input player name as the user in the user management information 112 and as the related person in the cleaning management information 113, or may register the player name together with the player icon.

[0056] When the hold button 1103a-5 is pressed, the player name is registered without registering the player icon. For this purpose, the cleaning management unit 107 registers the input player name to the user in the user management information 112 and to the related person in the cleaning management information 113.

[0057] Next, the display screen 1103b will be described with reference to FIG. 12B. In FIG. 12B, a player icon type selection button 1103b-6 is superimposed on the same display as the display screen 1103a of FIG. 12A, and the enter button 1103a-4 has been removed. The player icon type selection button 1103b-6 is a button for selecting whether to use a default icon or image data available on the user terminal 1, such as a photograph, as the type of player icon. When the default icon is selected with the player icon type selection button 1103b-6, the cleaning management unit 107 reads out the default icon stored in the memory unit 111. When the photo library is selected with the player icon type selection button 1103b-6, the cleaning management unit 107 reads out image data, such as a photograph, from the so-called photo library stored in the memory unit 111. The cleaning management unit 107 then transitions the display screen 1103b to a display screen 1103c for selecting a player icon.

[0058] Next, the display screen 1103b will be described using FIG. 12C . Because the same display screen appears whether the default icon or the photo library is selected for the player icon type selection button 1103b-6, both will be described together using FIG. 12C . In FIG. 12C , the player icon type selection button 1103c-7 is superimposed on the same display as the display screen 1103b of FIG. 12B . The player icon type selection button 1103c-7 displays a group of default icons or image data stored in the storage unit 111. When one of these default icons or image data is selected, the cleaning management unit 107 registers the selected default icon or image data with the user in the user management information 112 and the related person in the cleaning management information 113. Note that, in this registration, information identifying the default icon or image data may be registered instead of the default icon or image data itself. This concludes the description of step S4.

[0059] Furthermore, in step S5, the cleaning management unit 107 causes the output unit 110 to display instructions for use. FIG. 13 is a diagram showing a display screen 1103 of the output unit 110 that displays instructions for use in the first embodiment. Guidance 1104-1 is displayed on the display screen 1104. This guidance corresponds to a function of the guide information 115. When a function is designated by the user, the cleaning management unit 107 displays the corresponding guidance content from the guide information 115 within the area of ​​the guidance 1104-1. When the user presses the start button 1104-3 on the display screen 1104, the cleaning management unit 107 activates the designated function. In this processing flow, it is assumed that map creation is selected from the guidance 1104-1, and map creation from step S6 onwards is activated. Note that map creation may be automatically activated on the condition that the device is used for the first time.

[0060] Furthermore, the display screen 1103 displays progress information 1104-2 indicating the progress stage of the display screen 1103. Furthermore, the example of Fig. 13 shows that the guidance 1104-1 is in a "picture story show" format, that is, the content is displayed by transitioning between multiple still images or screens. In this case, the progress information 1104-2 indicates the progress of the "picture story show."

[0061] Then, a map creation process, which is an example of a preparation process, is executed in steps S6 to S12. First, in step S6, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1105 that explains how to create the map. In the first embodiment, two examples of this display screen 1105 are used. These will be described below.

[0062] FIG. 14A is a diagram illustrating a display screen 1105a (first example) for explaining the creation of a cleaning area map in Example 1. Note that, hereinafter, the cleaning area map will also be simply referred to as a map. Display screen 1105a displays explanatory text 1105a-1 explaining that this screen is a screen for creating a map. Below that, instruction text 1105a-2 is displayed, prompting the user to attach the smartphone (user terminal 1) to the terminal holder 20 in order to create the map, i.e., the cleaning area map. In other words, instruction text 1105a-2 instructs the user to attach the user terminal 1 to the terminal holder 20. Below that, an attachment image 1105a-3 is displayed, which shows the user terminal 1 being attached to the terminal holder 20. By visually viewing attachment image 1105a-3, the user can intuitively understand how to attach the user terminal 1. Note that attachment image 1105a-3 may be a moving or still image, and may also be realized using live action, animation, or illustration.

[0063] In addition, supplemental text 1105a-4 is displayed below the installation image 1105a-3, which contains a link to a solution for when the installation location or method is unknown. As shown in FIG. 14A, the supplemental text 1105a-4 preferably includes a link to an instruction manual. The instruction manual may be stored in the storage unit 111 or on a server of the manufacturer or the like on the Internet. In the former case, the instruction manual may be stored as independent information, or may be stored as part of the user management information 112 or the guidance information 115.

[0064] Also, below the supplemental text 1105a-4, a skip button 1105a-5 is displayed. When this button is pressed, map creation is skipped and the process proceeds to step S14. Below this, an installation complete button 1105a-6 is displayed. When this button is pressed, the process proceeds to step S7, where the camera, i.e., the photographing unit 102, is activated to create a map.

[0065] Next, a second example of the display screen 1105 will be described. Figures 14B and 14C are diagrams showing display screens 1105b and 1105c (second example) that explain the creation of a map in Example 1. In this example, the display screen 1105b transitions to the display screen 1105c. First, the display screen 1105b in Figure 14B is a screen related to map creation, and displays instruction text 1105b-1 that prompts the user to create a map.

[0066] Additionally, a creation image showing an image of map creation is displayed below the instruction text 1105b-1. And below that, explanatory text 1105b-3 showing how to create the map is displayed. As a result, the user can understand that by cleaning or moving the vacuum cleaner along the wall of the floor as shown in the instruction text 1105b-1, a map will be created as shown by the solid line in the creation image 1105b-2. In this way, by cleaning or moving the vacuum cleaner along the wall, a cleaning area map showing the perimeter of a unit cleaning area such as a floor is created.

[0067] The created image 1105b-2 may be a moving image or a still image, and may be realized as a live-action image, animation, illustration, etc. Furthermore, either the created image 1105b-2 or the explanatory text 1105b-3 may be used alone.

[0068] Also, below the explanatory text 1105b-3, a skip button 1105b-4 is displayed. When this button is pressed, map creation is skipped and the screen transitions to step S14. Below this, a back button 1105b-5 and a next button 1105b-6 are displayed. When the back button 1105b-5 is pressed, the screen returns to one of the screens before step S5. When the next button 1105b-6 is pressed, the screen transitions to the display screen 1105c of FIG. 14C.

[0069] Next, display screen 1105c displays instruction text 1105c-1, which indicates that this screen is a screen for creating a map and prompts the user to attach the smartphone to create the map. Below this, an attachment image 1105c-2 and a skip button 1105c-3, which are the same as the attachment image 1105a-3 and skip button 1105a-5 on display screen 1105a in FIG. 14A, are displayed. Furthermore, below skip button 1105c-3, a back button 1105c-4 and a next button 1105c-5 are displayed. Pressing the back button 1105c-4 returns the user to display screen 1105b in FIG. 14B. Pressing the next button 1105c-5 transitions to step S7, where the camera, i.e., the photographing unit 102, is activated to create the map. This concludes the explanation of step S6.

[0070] Next, in step S7, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning area map creation unit 105. Then, in step S8, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1106 for map creation including images photographed by the photographing unit 102. That is, the cleaning area map creation unit 105 provides guidance for creating a cleaning area map. Then, in step S9, the cleaning area map creation unit 105 creates a cleaning area map in response to the user's operation of the stick vacuum cleaner 2 in accordance with the guidance. Steps S8 and S9 will be described below with reference to the display screen 1106 for map creation. In steps S8 and S9, as in step S6, two examples of the display screen 1106 are used. These will be described below.

[0071] First, a first example will be described using FIGS. 15A to 15E. Here, as a prerequisite for creating a cleaning area map, a user is holding a stick vacuum cleaner 2 and is ready to operate it. The cleaning area map creation unit 105 then creates a display screen 1106a including an image captured by the image capture unit 102 and displays this on the output unit 110. FIG. 15A shows the display screen 1106a. The display screen 1106a displays operation text 1106a-1 explaining that creation of a cleaning area map can be started and including a prompt to confirm whether the vacuum cleaner suction head is included in the display screen 1106a. Here, the vacuum cleaner suction head refers to the vacuum cleaner head 24 of the stick vacuum cleaner 2. In the example of FIG. 15A, a vacuum cleaner 1106a-5 including the vacuum cleaner head 24 is displayed, allowing the user to continue operating the device. If the vacuum cleaner 1106a-5 is not displayed, the user will need to adjust the device holder 20 or the like so that the vacuum cleaner 1106a-5 is displayed.

[0072] In addition, an AR information area 1106a-2 is provided below the operation text 1106a-1, and an AR image is displayed therein. This AR information is created by the AR information creation unit 106 using the image captured by the image capture unit 102. This AR information includes the floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 to be cleaned. As will be described below with reference to FIG. 15B , it is desirable to display the wall surface 1106a-4 in order to create a cleaning area map by moving the stick vacuum cleaner 2 along the wall. Then, in response to a user's operation on the input unit 109, such as completion of preparation for creation, the cleaning area map creation unit 105 transitions the display from FIG. 15A to FIG. 15B .

[0073] FIG. 15B shows a display screen 1106b for map creation in Example 1, transitioned from FIG. 15A. Operation text 1106b-1 is displayed on the display screen 1106b, explaining that creation of a cleaning area map can be started, and including an instruction to move around a room, which is an example of a unit cleaning area, along a wall (wall surface 1106b-4). The user confirms this to start map creation. In other words, the user moves the stick vacuum cleaner 2 along the wall. At this time, it is desirable that the cleaning area map creation unit 105 accepts an input from the user via the input unit 109 instructing the start of creation.

[0074] In addition, an AR information area 1106b-2 is provided below the operation text 1106b-1, similar to the display screen 1106a of FIG. 15A. The AR information area 1106b-2 displays AR information based on a captured image including the floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 to be cleaned, similar to the display screen 1106a. When conditions such as input to start creation are met, the cleaning area map creation unit 105 transitions the display from FIG. 15B to FIG. 15C and creates a cleaning area map. The AR information in the AR information area 1106a-2 and the AR information area 1106b-2 may be the image itself captured by the image capture unit 102.

[0075] FIG. 15C shows a display screen 1106c for map creation in Example 1, transitioned from FIG. 15B. Operation text 1106c-1 is displayed on the display screen 1106c, explaining that a cleaning area map is being created and including operations for completing or canceling map creation. At this time, the cleaning area map creation unit 105 creates the cleaning area map using the cleaned positions identified by the operation status detection unit 104. Here, the cleaned positions indicate the movement trajectory of the stick vacuum cleaner 2, regardless of whether cleaning has been performed or not. Details of this cleaning area map and its creation will be described later.

[0076] Also, an AR information area 1106c-2 is provided below the operation text 1106c-1, similar to the display screens 1106a and 1106b. However, the AR information area 1106c-2 displays AR information including a movement trajectory 1106c-6 of the stick vacuum cleaner 2 in addition to a captured image including the floor surface 1106c-3, wall surface 1106c-4, and vacuum cleaner 1106c-5 to be cleaned. Specifically, the movement trajectory 1106c-6 is displayed around the vacuum cleaner 1106a-5 in FIG. 15C.

[0077] This AR information is created by the AR information creation unit 106. In the example of FIG. 15C, it can be seen that the stick vacuum cleaner 2 has moved from bottom to top in the figure. Furthermore, the movement trajectory 1109c-6 may display the boundary between cleaned positions and uncleaned positions using a gradation. Then, when the user taps the display screen 1106c while the screen of FIG. 15C is displayed, as indicated by operation text 1106c-1, the cleaning area map creation unit 105 transitions the display from FIG. 15C to FIG. 15D.

[0078] FIG. 15D shows a display screen 1106d for map creation in Example 1, transitioned from FIG. 15C. This display screen 1106d is a screen for canceling map creation and editing AR information. For this reason, in addition to the display screen 1106c, the display screen 1106d has additional operation buttons (1106d-6 and 1106d-7). First, the back button 1106d-6 is a button for canceling map creation. When pressed by the user, the cleaning area map creation unit 105 cancels map creation. Furthermore, the color change button 1106d-7 is a button for changing the color of the AR information, particularly the movement trajectory 1106d-8, to the user's preference. When a color change instruction is input using the color change button 1106d-7, the AR information creation unit 106 changes the color of the movement trajectory 1106d-8. The operation buttons may include buttons other than the back button 1106d-6 and the color change button 1106d-7, such as buttons for changing brightness or shape.

[0079] Furthermore, the color of the AR icon or map icon may be changed using the color change button 1106d-7. To this end, the cleaning management unit 107 changes the color of the AR icon or map icon in response to an operation on the color change button 1106d-7. Furthermore, instead of or in addition to the color change button 1106d-7, a color change button 1106d-7 for changing the form of the AR icon or map icon may be used. In this case, the cleaning management unit 107 changes the form of the AR icon or map icon in response to an operation on the form change button. The form to be changed includes the color (including brightness, saturation, and tone), shape (including size), transparency, blurring, and other effects of each icon.

[0080] Furthermore, when completion of map creation is received from the user via the input unit 109, the cleaning area map creation unit 105 transitions the display from Fig. 15D to Fig. 15E. This transition may be performed depending on the position of the stick vacuum cleaner 2 detected by the operation status detection unit 104. For example, the condition may be that the stick vacuum cleaner 2 has returned to the start position of map creation.

[0081] 15E shows a display screen 1106e for map creation in Example 1, transitioned from FIG. 15D. The display screen 1106e is a screen for ending map creation. Operation text 1106e-1 for confirming the end of map creation is displayed on the display screen 1106e.

[0082] An AR information area 1106e-2 is provided below the operation text 1106e-1. The AR information area 1106e-2 includes a unit cleaning area input area 1106e-3, a created map icon 1106e-4, a continue button 1106e-5, and a done button 1106e-6.

[0083] The unit cleaning area input field 1106e-3 is an area for inputting the name of the unit cleaning area that the created cleaning area map covers. A room or floor can be used as the unit cleaning area, and the name can be easily understood by the user.

[0084] Furthermore, the created map icon 1106e-4 is an icon that schematically shows the created cleaning area map, where the rectangle represented by a thick line in the figure indicates the movement trajectory, and the area inside the rectangle containing this indicates the cleaning area map. Furthermore, the continue button 1106e-5 and the done button 1106e-6 are buttons for accepting from the user whether map creation will continue or is complete, respectively. When the continue button 1106e-5 is pressed, the cleaning area map creation unit 105 transitions to the screen of Figure 15C or Figure 15D and continues map creation. When the done button 1106e-6 is pressed, the cleaning area map creation unit 105 ends map creation. In other words, step S9 ends.

[0085] Next, a second example of step S9 will be described with reference to Figures 15F and 15G. In the second example, as a premise for creating a cleaning area map, a user holds the stick vacuum cleaner 2 and performs cleaning. At this time, the cleaning area map creation unit 105 causes the output unit 110 to display display screen 1106f. This display screen 1106f corresponds to display screen 1106c of Figure 15C in the first example. Display screen 1106f is shown in Figure 15F.

[0086] The display screen 1106f displays explanatory text 1106f-1 explaining that a cleaning area map is being created. At this time, as in the first example, the cleaning area map creation unit 105 creates a cleaning area map using the cleaned positions identified by the operation status detection unit 104. As in the first example, an AR information area 1106f-2 is provided below the explanatory text 1106f-1. The AR information area 1106f-2 displays a captured image including the floor surface 1106f-3, wall surface 1106-4, and vacuum cleaner 1106f-5 to be cleaned, as well as a movement trajectory 1106f-7 of the stick vacuum cleaner 2. Furthermore, the AR information area 1106f-2 displays a map creation icon 1106f-6 indicating the creation status of the cleaning area map. The AR information creating unit 106 creates AR information including the captured image, the movement trajectory 1106f-7, and the map icon under creation 1106f-6.

[0087] The map creation icon 1106f-6 may be treated as information separate from the AR information. The map creation icon 1106f-6 also includes a line segment indicating the movement trajectory of the stick vacuum cleaner 2 and a circle indicating the current location. Furthermore, a pause button 1106f-8 and a cleaning complete button 1106f-9 are displayed below the display screen 1106f. When these buttons are pressed by the user, the cleaning area map creation unit 105 suspends or terminates the creation of the cleaning area map. When the creation of the cleaning area map for the unit cleaning area is terminated, that is, when the cleaning complete button 1106f-9 is pressed, the cleaning area map creation unit 105 transitions from FIG. 15F to FIG. 15G.

[0088] 15G shows a display screen 1106g for map creation in Example 1, transitioned from FIG. 15F. The display screen 1106g is a screen for ending map creation. Furthermore, the display screen 1106g displays explanatory text 1106g-1 indicating the end of map creation.

[0089] In addition, a unit cleaning area input field 1106g-2 is displayed below the explanatory text 1106f-1. This unit cleaning area input field 1106g-2 is an area for inputting the name of the unit cleaning area that the created cleaning area map targets. A room or floor can be used as the unit cleaning area, and the user can input a name that is easy for the user to understand. In addition, a created map display field 1106g-3 is displayed below that. The created map display field 1106g-3 displays a created map icon 1106g-4 that schematically indicates the created cleaning area map. In the example of FIG. 15G, a cleaning area map has been created for a room (living room) that has eight corners.

[0090] Furthermore, a retake button 1106g-5 and a complete button 1106g-6 are displayed below the created map display area 1106g-3. These are buttons for receiving from the user whether they want to redo map creation or whether they have completed it, respectively. When the retake button 1106g-5 is pressed, the cleaning area map creation unit 105 transitions to the screen of FIG. 15F and creates the map. When the complete button 1106g-6 is pressed, the cleaning area map creation unit 105 ends map creation. In other words, step S9 ends. This concludes the explanation of steps S8 and S9.

[0091] In step S10, the cleaning area map creation unit 105 determines whether map creation for the unit cleaning area to be created, which is exemplified by each room in the house, is complete. This determination is made using, for example, a completion instruction input by the user. As a result, if there are any rooms left to be created (incomplete), the process proceeds to step S11. If the process is complete, the process proceeds to step S12.

[0092] In step S11, the cleaning area map creation unit 105 receives a selection of the next room from the user via the input unit 109. That is, the cleaning area map creation unit 105 receives a request to create a map of the next room. In step S12, the photographing unit 102 ends photographing in response to an instruction from the cleaning area map creation unit 105.

[0093] In step S13, the cleaning area map creation unit 105 registers the created cleaning area map in the cleaning area map information 114. This may be the end of the processing flow, but in the first embodiment, the description will continue assuming that the cleaning management process is subsequently executed using the created cleaning area map.

[0094] In step S14, the cleaning management unit 107 causes the output unit 110 to display a home screen. First and second examples of this home screen will be described. First, FIGS. 16A and 16B show a first example of a home screen 1107 in Example 1. In this example, a screen displaying a cleaning history is used as the home screen 1107. FIG. 16A is a diagram showing a home screen 1107a in the first example. The home screen 1107a displays explanatory text 1107a-1 explaining that the home screen 1107a displays the cleaning history. Below this, history image information 1107a-2 is displayed, which schematically shows the cleaning history for each unit cleaning area. In the figure, the white area represents a cleaning area map, within which the movement history is displayed. This history image information 1107a-2 uses floors (floors) as unit cleaning areas, and switching between these areas can be achieved using tabs. In the example of FIG. 16A, the history information for "1F" to "3F" is displayed overlapping each other. Among these, the cleaning history for "1F" is displayed, and when switching to "2F" using the tab, the display transitions to a home screen 1107b shown in Fig. 16B. Fig. 16B will be described later.

[0095] In addition, a game function selection button 1107a-3 is displayed below the history image information 1107a-2. The game function selection button 1107a-3 is a button for accepting whether or not to use the game function in step S15, which will be described later. Furthermore, below that, history text 1107a-4 is displayed. This represents the cleaning history in text information, and indicates the date and time when the cleaning was performed, the time, and the coverage rate for each unit cleaning area. In addition, a selection button is provided for each history. The cleaning management unit 107 creates a movement history within the cleaning area map of the history image information 1107a-2 according to this selected cleaning history. In the example of FIG. 16A, all cleaning history is selected. By displaying information according to such selection, the user can intuitively grasp the cleaning history up to the desired period.

[0096] Also, below the history text 1107a-4, a cleaning start button 1107a-5 is displayed. Pressing this cleaning start button 1107a-5 will cause the cleaning management process to be executed.

[0097] Next, an example will be described in which the home screen is switched using the tabs of the history image information 1107a-2. When the "2F" tab in the history image information 1107a-2 is selected, the cleaning management unit 107 transitions the screen to FIG. 16B. FIG. 16B is a diagram showing a home screen 1107b transitioned from FIG. 16A. Similar to the home screen 1107a, the home screen 1107b displays explanatory text 1107b-1, history image information 1107b-2, a game function selection button 1107b-3, history text 1107b-4, and a cleaning start button 1107a-5. However, the contents of the history image information 1107b-2 and history text 1107b-4 have been changed from 1F to 2F. In this way, the history image information 1107b-2 and history text 1107b-4 are changed in conjunction with the selection of the tab.

[0098] 16B, the cleaning history for December 25th is not selected. Therefore, the cleaning history for December 30th and December 31st is shown in the cleaning history text 1107b-4. This concludes the explanation of the first example.

[0099] Next, a second example of the home screen will be described with reference to FIG. 16C . In this second example, a cleaning game function is enabled. In this game, acorns are collected by cleaning. Acorns can be collected by virtually sucking in the acorns themselves and other monsters representing dust contained in the AR information.

[0100] For this reason, in FIG. 16C, a game character 1107c-1 is displayed on the home screen 1107c. Below that, collection points 1107c-2 indicating the number of acorns collected through the game function are displayed. The number of acorns is an example of game points. Displaying the points in this way helps motivate cleaning. Note that while it is desirable for the number of acorns to be the number of all related parties, it may also be the number for a specific related party, such as a user.

[0101] Additionally, below the collection point 1107c-2, a start cleaning button 1107c-3 and a display history button 1107c-4 are displayed. When the start cleaning button 1107c-3 is pressed, the cleaning management unit 107 starts the cleaning management process. For example, the cleaning management unit 107 displays the screen shown in FIG. 17C. This screen will be described later. When the display history button 1107c-4 is pressed, the cleaning management unit 107 displays the cleaning history, such as the image history information and text history shown in FIGS. 16A and 16B, on the output unit 110. Below these, a cleaning settings button 1107c-5 is displayed. When the cleaning settings button 1107c-5 is pressed, the cleaning management unit 107 displays a screen on the output unit 110 for inputting various settings related to cleaning. Then, in accordance with the input on this screen, the cleaning management unit 107 records various settings related to cleaning. This concludes the description of step S14.

[0102] Next, the cleaning management process is executed from step S15 shown in FIG. 9B. This does not have to be executed consecutively with the map creation process. For example, when the stick vacuum cleaner 2 is started, the cleaning management process is executed using the stored cleaning area map. First, in step S15, the cleaning management unit 107 accepts from the user whether or not to use the game function. For example, an operation on the game function selection button 1107a-3 in FIG. 16A or the game function selection button 1107b-3 in FIG. 16B is accepted. Note that in the first embodiment, step S15 may be omitted, and the game function may be enabled by default.

[0103] Furthermore, in step S16, the cleaning management unit 107 accepts a user selection, i.e., a selection of a game player, via the input unit 109. Furthermore, in step S17, the cleaning management unit 107 accepts a selection of a map of a unit cleaning area to be cleaned from the user via the input unit 109. Furthermore, in step S18, the cleaning management unit 107 causes the output unit 110 to display a terminal attachment instruction screen that prompts the user to attach the user terminal 1 to the terminal holder 20. Then, in step S19, the cleaning management unit 107 accepts an input from the user via the input unit 109 to confirm that the user terminal 1 has been attached.

[0104] Note that if the cleaning management process is executed immediately after the map creation process, steps S18 and S19 can be omitted. Steps S15 to S19 above can be considered as preparatory processing for the cleaning management process. Two examples of this preparatory processing will be explained below using display screens.

[0105] 17A is a diagram showing a map selection screen 1108a for accepting map selection in step S17 in the first example of the preparation processing. The map selection screen 1108a is displayed on the output unit 110 by the cleaning management unit 107. The map selection screen 1108a displays explanatory text 1108a-1 explaining that this screen is a screen for selecting a cleaning area map.

[0106] Below that, a created map icon 1108a-2 is displayed for each created cleaning area map. The user selects a cleaning area map by instructing (for example, pressing) one of these icons. That is, the cleaning management unit 107 selects the corresponding cleaning area map from the cleaning area map information 114. Also, below the created map icon 1108a-2, a confirmation button 1108a-3 for confirming the selection of the cleaning area map is displayed. When the confirmation button 1108a-3 is pressed, the cleaning management unit 107 transitions to step S18, and the screen transitions from FIG. 17A to FIG. 17B.

[0107] Next, Figure 17B is a diagram showing a terminal attachment instruction screen 1108b in step S18 in the first example. Similar to Figure 14A, the terminal attachment instruction screen 1108b is a display screen for attaching the user terminal 1 to the terminal holder 20. The terminal attachment instruction screen 1108b displays instruction text 1108b-1 prompting the user to attach the terminal. Also, below this, an attachment image 1108b-2 is displayed, which is an image showing how the user terminal 1 is attached to the terminal holder 20. The attachment image 1108b-2 may be a moving image or a still image, and may be realized as an animation or illustration in addition to a live action.

[0108] Furthermore, a back button 1108b-3 and a next button 1108b-4 are displayed below the installation image 1108b-2. When the back button 1108b-3 is pressed, the cleaning management unit 107 transitions the screen from FIG. 17B to FIG. 17A. When the next button 1108b-4 is pressed, the cleaning management unit 107 transitions the processing flow to a process (step S19) to confirm that the user terminal 1 has been installed. At this time, it is desirable that the cleaning management unit 107 display an installation confirmation screen for the user terminal 1 on the output unit 110. This concludes the explanation of the first example of the preparation processing, and a second example will now be explained.

[0109] 17C is a diagram showing a player / map selection screen 1108c for receiving both player selection (user selection) and map selection in steps S16 and S17 in the second example of the preparation processing. Instructive text 1108c-1 is displayed on the player / map selection screen 1108c to prompt the user to select a player and a cleaning area map.

[0110] Below that, a group of player icons 1108c-2 for selecting a player is displayed. A corresponding player is selected by the user's designation of one of the group of player icons 1108c-2. In other words, the cleaning management unit 107 identifies the player (person involved) corresponding to the selected player or icon.

[0111] In addition, a map selection area 1108c-3 is displayed below the group of player icons 1108c-2. The map selection area 1108c-3 includes the name of the room that is the unit cleaning area and a created map icon that indicates this. When a triangle mark in the map selection area 1108c-3 is specified, the cleaning management unit 107 switches between the room name and the created map icon. As a result, the user can select the desired room. Furthermore, the map selection area 1108c-3b displays the most recent cleaning date and time of the room.

[0112] Furthermore, a back button 1108c-4 and a next button 1108c-5 are displayed below the map selection area 1108c-3. When the back button 1108c-4 is pressed, the cleaning management unit 107 changes the screen from Fig. 17C to Fig. 16C. When the next button 1108c-5 is pressed, the cleaning management unit 107 changes the screen from Fig. 17C to Fig. 17D.

[0113] 17D is a diagram showing a player / map selection results screen 1108d when a player selection (user selection) and a map selection are executed in steps S16 and S17 in the second example of the preparation processing. The player / map selection results screen 1108d displays the player icon 1108d-1 and player name 1108d-2 of the selected player. Below that, a unit cleaning area name 1108d-3 indicating the selected unit cleaning area is displayed. As a result, the user can confirm the selected player (Taro) and unit (living room).

[0114] Additionally, a back button 1108d-4 and a next button 1108d-5 are displayed below the unit cleaning area name 1108d-3. When the back button 1108d-4 is pressed, the cleaning management unit 107 transitions the display to FIG. 17C. When the next button 1108d-5 is pressed, the cleaning management unit 107 transitions the processing flow to a process for confirming that the user terminal 1 has been installed (step S19). At this time, it is desirable that the cleaning management unit 107 display an installation confirmation screen for the user terminal 1 on the output unit 110. This concludes the explanation of the second example of the preparation processing. In other words, the explanation of the preparation processing up to step S19 is completed.

[0115] Furthermore, in step S20, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning management unit 107. Furthermore, in step S21, the operation status detection unit 104 begins detecting the operation status of the stick vacuum cleaner 2 and identifying the cleaned position using this. Then, in step S22, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information including AR information. Here, the details of step S22 will be described using specific examples of cleaning-related information.

[0116] 18 is a flowchart showing details of the cleaning-related information display process (step S22) in Example 1. In step S2201, the cleaning management unit 107 determines whether or not to use the game function. For example, the cleaning management unit 107 can make this determination based on an input to the game function selection button 1107a-3 in FIG. 16A or the game function selection button 1107b-3 in FIG. 16B. As a result, if the game function is to be used, the process proceeds to step S2202. If the game function is not to be used, the process proceeds to step S2208.

[0117] Then, in step S2202, the cleaning management unit 107 identifies AR icons to be displayed and calculates the number of such icons. Here, the AR icons are included in the AR information and are used when executing the game function. The AR icons include dust character icons, and points can be obtained by virtually sucking these up with the stick vacuum cleaner 2. An example of this is the dust monster 1109a-2 in FIG. 19A.

[0118] The cleaning management unit 107 also calculates the number of AR icons according to the unit cleaning area indicated by the map selected in step S17. For example, the calculation can be performed according to at least one of the attributes (room type), area, cleaning frequency, and most recent cleaning time of the unit cleaning area. Furthermore, the number of AR icons may be determined according to the user selected in step S16.

[0119] In step SS2203, the cleaning management unit 107 determines the placement of the identified icons in the target unit cleaning area. That is, the placement of the AR icons in the cleaning area map is determined. The placement can be uniform placement in the unit cleaning area, placement of more icons in areas where dust tends to accumulate (near corners, areas far from entrances, etc.), or placement of more icons in areas that have not been cleaned much based on the cleaning history.

[0120] In step S2204, the cleaning management unit 107 creates AR information by superimposing a movement trajectory and an AR icon on the captured image. Here, the captured image is the one captured in step S20. The movement trajectory is the one identified in step S21. The AR icon is the one identified in step S2202. At this time, the history information creation unit 108 uses the movement trajectory to create cleaning records, which are an example of history information.

[0121] In step S2205, the cleaning management unit 107 superimposes the movement trajectory on the corresponding cleaning area map. That is, the movement trajectory can be displayed on the cleaning area map. The placement position of the AR icon may also be superimposed on the cleaning area map. As a result, cleaning-related information is created.

[0122] Then, in step S2206, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information including an operating map icon indicating a cleaning area map on which the AR information created in step S2204 and the movement trajectory identified in step S2205 are superimposed. The cleaning-related information may also include points acquired during the cleaning, a cleaning history, and various explanations and instructions. As a result, the user visually checks the cleaning-related information and cleans with the stick vacuum cleaner 2 as if sucking in the AR icon. Furthermore, the user can thoroughly clean the unit cleaning area while checking the operating map icon of the cleaning-related information. At this time, the cleaning management unit 107 controls the display on the output unit 110 so that the AR icon appears to be sucked into the stick vacuum cleaner 2. In this way, the cleaning management unit 107 changes the display of the AR icon, such as by erasing it, in response to the user's operation on the stick vacuum cleaner 2.

[0123] In step S2207, the cleaning management unit 107 calculates points for the game function according to the cleaning status and adds them up. For example, when an AR icon is sucked in, the cleaning management unit 107 calculates points according to the AR points using the game information 116 and adds these points to the points already acquired by the user. The cleaning management unit 107 may also calculate points according to the movement trajectory, i.e., the amount of cleaning. Here, an example of steps S2202 to S2207, which are the processing when the game function is used, will be described using display screens. Specifically, the description will be given using display screens 1109a and 1109b of cleaning-related information in Example 1 shown in FIGS. 19A and 19B.

[0124] First, display screen 1109a is the screen when cleaning has started. Display screen 1109a then displays an operating map icon 1109a-1. Because this screen is the screen when cleaning has started, the operating map icon 1109a-1 does not display a movement trajectory. Also, a dust monster 1109a-2, which is a character representing dust, is displayed as an AR icon. The user can enjoy cleaning by operating the stick vacuum cleaner 2 as if virtually sucking up the dust monster 1109a-2.

[0125] Furthermore, the vacuum cleaner 1109a-3 in the display screen 1109a indicates the vacuum cleaner head of the stick vacuum cleaner 2 included in the captured image. Furthermore, the point display area 1109a-4 indicates the cumulative value of points calculated by the corresponding cleaning, such as when the dust monsters 1109a-2 are virtually sucked in. Here, in this example, when a certain number of dust monsters are sucked in, an acorn can be obtained, and the number of acorns obtained is displayed as points.

[0126] Then, when the user operates the stick vacuum cleaner 2 and the dust monster 1109a-2 is sucked in, the cleaning management unit 107 transitions the display from FIG. 19A to FIG. 19B. The display screen 1109b shown in FIG. 19B is a screen that has been changed from the display screen 1109a by the cleaning management unit 107 in response to the operation and operation of the stick vacuum cleaner 2. Specifically, the dust monster 1109a-2 has been erased from the display screen 1109a, and a movement trajectory has been added to the operating map icon 1109b-1. Furthermore, as the stick vacuum cleaner 2 moves, a movement trajectory 1109b-4 is displayed around the vacuum cleaner 1109b-2. This concludes the explanation of the specific example of steps S2202 to S2207 in which the game function is used. Returning to FIG. 18, the processing in the case where the game function is not used will be described.

[0127] In step S2208, the cleaning management unit 107 creates AR information by superimposing the movement trajectory on the captured image. Here, the captured image is the one captured in step S20. The movement trajectory is the one identified in step S21. In step S2209, the cleaning management unit 107 reads a cleaning area map of the unit cleaning area being cleaned from the storage unit 111. In step S2209, the cleaning management unit 107 superimposes the movement trajectory on the cleaning area map of the unit cleaning area being cleaned. In other words, the movement trajectory can be displayed on the cleaning area map. The placement position of the AR icon may also be superimposed on the cleaning area map. At this time, similar to step S2206, the history information creation unit 108 uses the movement trajectory to create cleaning records, which are an example of history information.

[0128] Then, in step S2210, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information, including the AR information created in step S2208 and the active map icon showing the cleaning area map with the movement trajectory superimposed in step S2209. The cleaning-related information may also include the cleaning history and various explanations and instructions. As a result, the user can thoroughly clean the unit cleaning area while checking the active map icon in the cleaning-related information. This concludes the explanation of steps S2202 to S2207, which are the processing when the game function is used.

[0129] Next, an example of steps S2208 to S2210, which are the processing when the game function is not in use, will be described using display screens. Specifically, the description will be given using display screens 1109c and 1109d for cleaning-related information in Example 1 shown in Figures 19C and 19D. First, display screen 1109c is a screen displayed during cleaning. Display screen 1109c also displays cleaning history 1109c-1. In this example, the cleaning time and coverage rate, which indicates the proportion of the area that has been cleaned relative to the unit cleaning area, are also displayed. Since these are both 0, it can be understood that this is the display when cleaning has begun.

[0130] In addition, below the cleaning history 1109c-1, instruction text 1109c-2 is displayed to allow appropriate photography on the user terminal 1. In addition, as photographed images of cleaning-related information, floor surface 1106c-3, wall surface 1106c-4, and vacuum cleaner 1106c-5 to be cleaned are displayed. These are the same as floor surface 1106a-3, wall surface 1106a-4, and vacuum cleaner 1106a-5 when creating the map.

[0131] The cleaning-related information also displays a movement trajectory 1109c-8, which is AR information. Here, the movement trajectory 1109c-8 may display the boundary between cleaned positions and uncleaned positions using a gradation. Here, a sign 1109c-7 is reflected in part of the vacuum cleaner 1106c-5. This sign 1109c-7 corresponds to the marker of the instruction text 1109c-2.

[0132] Furthermore, an operating map icon 1109c-5 is displayed on the display screen 1109c. This operating map icon 1109c-5 is similar to the operating map icon 1109a-1 and the operating map icon 1109b-1. However, the operating map icon 1109c-5 also includes the current position 1109c-10 of the stick vacuum cleaner 2. This allows the user to intuitively grasp the current cleaning position within the unit cleaning area. Also, a cleaning start button 1109c-9 is displayed on the display screen 1109c, and pressing this button causes the cleaning management unit 107 to transition the display from FIG. 19C to FIG. 19D.

[0133] 19D is a screen that has been changed from display screen 1109c by cleaning management unit 107 in response to the operation and performance of stick vacuum cleaner 2. Display screen 1109d displays cleaning history 1109d-1, which is the same as cleaning history 1109c-1. In cleaning history 1109d-1, the cleaning time and coverage rate have been counted up since cleaning started.

[0134] Similarly to the display screen 1109c, the display screen 1109d also displays, as captured images of the cleaning-related information, a floor surface 1106d-2, a wall surface 1106d-3, and a vacuum cleaner 1109d-7 to be cleaned. Similarly to the display screen 1109c, the cleaning-related information also displays a movement trajectory 1109d-9, which is AR information. Again, the movement trajectory 1109d-9 may display the boundary between cleaned positions and uncleaned positions using a gradation.

[0135] Furthermore, the display screen 1109d displays an operating map icon 1109d-4, similar to the display screen 1109c. This operating map icon 1109d-4 includes the current position 1109d-5 and movement trajectory 1109d-6 of the stick vacuum cleaner 2. In other words, the movement trajectory 1109d-6 has been added to the display screen 1109c. This was added because the cleaning has progressed. This operating map icon 1109d-4 allows the user to intuitively grasp the current cleaning position and cleaning progress status within the unit cleaning area. The display screen 1109d also displays a pause button 1109d-10 and a cleaning end button 1109d-11. When the pause button 1109d-10 is pressed, the cleaning management unit 107 temporarily suspends acquisition of the operating status, and when pressed again, resumes this. Furthermore, when the cleaning end button 1109d-11 is pressed, step S22 is terminated. This concludes the explanation of FIG. 18, and we will return to FIG. 9B and explain step S23 and subsequent steps.

[0136] Furthermore, in step S23, cleaning management unit 107 determines whether cleaning of the corresponding unit cleaning area has been completed. To do this, cleaning management unit 107 makes a determination by accepting an input from the user confirming completion, or by determining the amount or proportion of the movement trajectory. Then, in step S24, cleaning management unit 107 causes output unit 110 to display a display screen indicating that cleaning has been completed. Below, an example of the cleaning completion process in step S24 will be described using the display screen. Here, the description will be divided into a case where the game function is used (when steps S2202 to S2207 are executed) and a case where the game function is not used (when steps S2208 to S2210 are executed).

[0137] First, a description will be given of the case where the game function is used. As cleaning progresses as described above in FIG. 19B, the cleaning management unit 107 determines that cleaning of the corresponding unit cleaning area has been completed. In this case, the cleaning management unit 107 transitions the display from FIG. 19B to FIG. 20A. For example, the cleaning management unit 107 can determine that cleaning has been completed if the movement trajectory accounts for a predetermined percentage or more of the area of ​​the cleaning area map. The display screen 1110a in FIG. 20A displays confirmation text 1110a-1 confirming the completion of cleaning. Below this, an operating map icon 1110a-2 including the movement trajectory is displayed. This operating map icon 1110a-2 is displayed larger than the operating map icon 1109a-1 and the operating map icon 1109b-1, and also indicates the possibility of deviation from the actual movement trajectory.

[0138] Furthermore, below this operating map icon 1110a-2, a continue button 1110a-3 and an end button 1110a-4 are displayed. When the continue button 1110a-3 is pressed, the cleaning management unit 107 returns the display to FIG. 19B to continue cleaning. In other words, the process transitions to step S2206. When the end button 1110a-4 is pressed, the cleaning management unit 107 transitions the display from FIG. 20A to FIG. 20B.

[0139] The display screen 1110b shown in Fig. 20B is a screen that indicates that cleaning, i.e., the game function, has ended. Therefore, the display screen 1110b displays explanatory text 1110b-1 indicating that the cleaning has ended. Below that, instruction text 1110b-2 is displayed, urging the user to remove the user terminal 1 from the terminal holder 20. Below that, a Next button 1110b-3 is displayed. When this Next button 1110b-3 is pressed, the cleaning management unit 107 transitions the display from Fig. 20B to Fig. 20C.

[0140] The display screen 1110c shown in FIG. 20C is a screen that indicates that the cleaning has been completed. Therefore, the display screen 1110c displays evaluation text 1110c-1 indicating the evaluation of the cleaning. Below that, an active map icon 1110c-2 including the movement trajectory is displayed. Furthermore, a points display area 1110c-3 is displayed showing the number of acorns collected and the points awarded for sucking up dust monsters. These allow the user to understand the evaluation of the cleaning. Furthermore, a next button 1110c-4 is displayed below the points display area 1110c-3, and pressing this button ends step S24. This concludes the explanation of when the game function is used.

[0141] Next, a description will be given of the case where the game function is not in use. As cleaning progresses as described above in FIG. 19D, the cleaning management unit 107 determines that cleaning of the corresponding unit cleaning area has been completed. The display screen 1110d shown in FIG. 20D is the screen to which the cleaning management unit 107 changes from display screen 1109d in response to operation of the cleaning end button 1109d-11. Display screen 1110d displays explanatory text 1110d-1 including the cleaning status. Below that, an active map icon 1110d-2 including the coverage rate is displayed. Further below that, a next button 1110d-3 is displayed. Pressing this next button 1110d-3 ends step S24. This concludes the description of steps S23 and S24.

[0142] Then, in step S25, the cleaning management unit 107 registers various information related to the performed cleaning in the storage unit 111. For example, the cleaning management unit 107 registers the cleaning record, including the created AR information and movement trajectory, in the cleaning management information 113. The cleaning management unit 107 also registers the calculated points in the game information 116. The history information creation unit 108 also registers the cleaning record created in step S2204 or step S2208 in the cleaning management information 113. At this time, the date and time corresponding to the cleaning management information 113 are also registered.

[0143] This concludes the description of the first embodiment. In the first embodiment, the user can enjoy cleaning while using the game function, depending on their selection. In this way, motivation for cleaning can be ensured. However, the game function is not essential, and it is not necessary to provide the game function. Even in this case, in the first embodiment, cleaning can be performed while understanding the situation, and more appropriate cleaning can be performed. Furthermore, it is not necessary to disable the game function, and it is also possible to always use the game function.

[0144] In Example 2, a function of recommending a cleaning position is added to Example 1. This feature will be mainly described below. First, the configuration and information of Example 2 are the same as those of Example 1, and therefore a description thereof will be omitted. Furthermore, a characteristic process of Example 2 is step S22. Therefore, the corresponding description in Figures 9A and 9B will be omitted, and step S22 will be described in detail.

[0145] FIG. 21 is a flowchart showing details of the cleaning-related information display process (step S22) in Example 2. Example 2 will be described below with reference to FIG. 21. First, in step S2211, the cleaning management unit 107 identifies a recommended area in the unit cleaning area for which cleaning is recommended. To do this, the cleaning management unit 107 uses the cleaning history in the cleaning management information 113. For example, an area that is less frequently cleaned based on the cleaning history is identified as a recommended area. Here, an infrequently cleaned area is identified from a location that is cleaned less frequently than a predetermined threshold, a location that is in the top (e.g., 10%) of the unit cleaning areas and is cleaned less frequently, a location where a predetermined period of time has passed since the last cleaning, etc.

[0146] As in the first embodiment, in step S2201, the cleaning management unit 107 determines whether or not to use the game function. If the game function is to be used, the process proceeds to step S2212. If the game function is not to be used, the process proceeds to step S2219.

[0147] Then, in step S2212, the cleaning management unit 107 identifies AR icons to be displayed and calculates the number of AR icons. While the number of AR icons was calculated according to the unit cleaning area in the first embodiment, the number of AR icons can be calculated according to at least one of the area of ​​the recommended area identified in step S2211, the cleaning frequency, and the most recent cleaning time. In the second embodiment, the attributes of the unit cleaning area (room type), the area, the cleaning frequency, and the most recent cleaning time may also be used. Furthermore, the number of AR icons may be determined according to the user, as in the first embodiment.

[0148] Furthermore, in step SS2213, the cleaning management unit 107 determines the arrangement of the identified icons in the target unit cleaning area. That is, the arrangement of the AR icons in the cleaning area map is determined. This arrangement is preferably such that the number (density) of icons per unit area in the recommended area is as large as possible. For example, the cleaning management unit 107 allocates the calculated number of AR icons to the recommended area and the non-recommended area according to the ratio of the area of ​​the recommended area to the unit cleaning area. At this time, a density is determined in advance for the recommended area and the non-recommended area, and the number of AR icons is allocated using this density. The density of the recommended area and the non-recommended area is recorded in the game information 116.

[0149] In step S2214, the cleaning management unit 107 creates AR information by superimposing the movement trajectory and an AR icon on the captured image, similar to step S2204 in Example 1. In step S2215, the cleaning management unit 107 superimposes the movement trajectory on the corresponding cleaning area map, similar to Example 1. In addition, in this case, the history information creation unit 108 uses the movement trajectory to create cleaning results, which are an example of history information, similar to Example 1.

[0150] Then, in step S2216, the cleaning management unit 107 creates cleaning-related information by superimposing the recommended area on each of the AR information and the cleaning area map. Here, the cleaning-related information is information including the AR information on which the recommended area is superimposed and an active map icon indicating the recommended area. Note that the recommended area may be superimposed on either the AR information or the active map icon.

[0151] In response to this, in step S2217, the cleaning management unit 107 causes the cleaning-related information created in step S2216 to be displayed on the output unit 110. An example of this display will be described using cleaning-related information display screens 1111a and 1111b shown in Figures 22A and 22B.

[0152] First, the display screen 1111a in FIG. 22A is a screen displayed when cleaning starts. The display screen 1111a corresponds to the display screen 1109a (FIG. 19A) of Example 1. The display screen 1111a includes a recommended area 1111a-6 and a recommended area 1111a-7 in addition to the display screen 1109a. The recommended area 1111a-6 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information and is displayed separately from other areas (non-recommended areas). This makes it easy to identify that the recommended area 1111a-6 is an area that requires cleaning. The boundary between the recommended area 1111a-6 and the other areas may be displayed with a gradation. The recommended area 1111a-7 is superimposed on the active map icon and indicates the recommended area within the entire unit cleaning area. This allows the user to grasp the areas that require cleaning from a bird's-eye view.

[0153] Furthermore, the display screen 1111b in Fig. 22B is a display screen transitioned from the display screen 1111a in Fig. 22A. In other words, the display screen 1111b in Fig. 22B corresponds to the display screen 1109b (Fig. 19B) of Example 1. Therefore, the operating map icon 1111b-1 to the movement trajectory 1111b-4 on the display screen 1111b are the same as the operating map icon 1109b-1 to the movement trajectory 1109b-4 on the display screen 1109b, respectively. Furthermore, a recommended area 1111b-5 and a recommended area 1111b-6 have been added to the display screen 1111b.

[0154] Here, the recommended area 1111b-5 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information, and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1111b-5 is an area that needs to be cleaned highly. The boundary between the recommended area 1111b-5 and other areas may be displayed with a gradation. Furthermore, the recommended area 1111b-6 is superimposed on the active map icon, and indicates the recommended area within the entire unit cleaning area. As a result, the user can grasp the areas that need to be cleaned highly from a bird's-eye view. This concludes the explanation of step S2217, and we will return to FIG. 21 and explain the processing from step S2218 onwards.

[0155] In step S2218, similar to step S2207 in the first embodiment, the cleaning management unit 107 calculates points for the game function according to the cleaning status. For example, when an AR icon is sucked in, the cleaning management unit 107 calculates points according to the AR point using the game information 116. Here, more AR icons (e.g., dust monsters) are placed in the recommended area. Therefore, by prioritizing cleaning of the recommended area, the user can obtain more points. As a result, it is possible to encourage the user to prioritize cleaning of the recommended area. This concludes the description of steps S2212 to S2218, which are the processing when the game function is used.

[0156] Returning to FIG. 21, the processing from step S2219 onwards, which is the processing when the game function is not used, will be described.

[0157] In step S2219, the cleaning management unit 107 creates AR information by superimposing the recommended area and the movement trajectory on the captured image. Furthermore, in step S2220, similar to step S2209 in the first embodiment, the cleaning management unit 107 superimposes the movement trajectory on the cleaning area map of the unit cleaning area being cleaned. At this time, similar to step S2214, the history information creation unit 108 uses the movement trajectory to create cleaning records, which are an example of history information.

[0158] Then, in step S2221, the cleaning management unit 107 causes the output unit 110 to display cleaning-related information including the AR information created in step S2219 and an operating map icon indicating the cleaning area map on which the movement trajectory is superimposed in step S2210. Note that the cleaning-related information may also include a cleaning history and various explanations and instructions.

[0159] An example of this display will be described using cleaning-related information display screens 1112a and 1112b shown in Figures 23A and 23B. First, display screen 1112a in Figure 23A corresponds to display screen 1109c in Figure 19C of Example 1. Therefore, cleaning history 1112a-1 to current location 1112a-10 on display screen 1112a are the same as cleaning history 1109c-1 to current location 1109c-10 on display screen 1109c, respectively. Furthermore, recommended area 1112a-11 and recommended area 1112a-12 have been added to display screen 1112a.

[0160] The recommended area 1112a-11 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1112a-11 is an area that is highly in need of cleaning. The boundary between the recommended area 1112a-11 and other areas may be displayed using a gradation.

[0161] Furthermore, the recommended area 1112a-12 is superimposed on the operating map icon 1112a-5, and indicates the recommended area within the entire unit cleaning area, allowing the user to grasp, from a bird's-eye view, the areas that are most in need of cleaning.

[0162] Furthermore, display screen 1112b in Fig. 23B is a display screen transitioned from display screen 1112a in Fig. 23A. In other words, display screen 1112b in Fig. 23B is a display screen corresponding to display screen 1109d (Fig. 19D) of Example 1. Therefore, cleaning history 1112a-1 to cleaning end button 1112ad-11 on display screen 1112b are the same as cleaning history 1109d-1 to cleaning end button 1109d-11 on display screen 1109d, respectively. Furthermore, recommended area 1112b-12 and recommended area 1112b-13 have been added to display screen 1112b.

[0163] Here, the recommended area 1112b-12 is superimposed on the cleaning target (such as the floor) in the captured image in the AR information, and is displayed to be distinguished from other areas (non-recommended areas). As a result, it is easy to understand that the recommended area 1112b-12 is an area that needs to be cleaned highly. The boundary between the recommended area 1112b-12 and other areas may be displayed with a gradation. Furthermore, the recommended area 1112b-13 is superimposed on the active map icon, and indicates the recommended area within the entire unit cleaning area. As a result, the user can grasp the areas that need to be cleaned highly from a bird's-eye view.

[0164] In the second embodiment, the recommended area is displayed on the display screen during cleaning, but may also be displayed on display screens 1110a to 1110d during the cleaning completion process. This concludes the description of FIG. 21 , i.e., the description of the second embodiment. In this embodiment, the recommended area is displayed, making it easier for the user to grasp areas that are highly in need of cleaning. Also, in the second embodiment, the game function does not necessarily have to be disabled, and the game function may always be used. Furthermore, superimposing the recommended area on the AR information is not essential and may be omitted. In particular, the display of the active map icon may represent the density of the AR icon using a shade of color or the like.

[0165] In Examples 1 and 2, the cleaning area presentation process is performed by the user terminal 1 alone, but in Example 3, various functions are realized in cooperation with a cleaning management device 3 realized by a server or the like. Fig. 24 is a system configuration diagram of the cleaning management system in Example 3. In Fig. 24, the cleaning management system has a user terminal 1 held by the terminal holder 20 of a stick vacuum cleaner 2, and a cleaning management device 3, which are connected via a network 5. In this case, the user terminal 1 may be connected to the network 5 via a router 4, or may be connected via another device, or may be connected to the network 5 directly.

[0166] In the third embodiment, among the functions of the user terminal 1 described in the first and second embodiments, the functions and configurations executed by the cleaning management device 3 can be omitted. For example, at least one of the cleaner identification unit 103, the operation status detection unit 104, the cleaning area map creation unit 105, the AR information creation unit 106, the cleaning management unit 107, and the history information creation unit 108 can be omitted. Furthermore, at least one of the information stored in the memory unit 111 can be omitted.

[0167] Furthermore, the user terminal 1 and the cleaning management device 3 may be connected to the user terminal 1-a used by the relevant person described in Example 1 via the router 4 and the network 5. The user terminal 1-a may be located in the same house as the user terminal 1 or outside the house, and may connect to the user terminal 1 and the cleaning management device 3 via the router 4 or the network 5, and share the cleaning results, including points, with the user terminal 1. The user terminal 1 and the stick vacuum cleaner 2 are the same as those in Example 1. The user terminal 1-a may have the same functions as the user terminal 1, or may be configured such that the information stored in the vacuum cleaner identification unit 103 to the cleaning management unit 107, the history information creation unit 108, and the memory unit 111 is omitted.

[0168] Next, a description will be given of the cleaning management device 3, which is a characteristic configuration of Example 3. The cleaning management device 3 is used by manufacturers and distributors of stick vacuum cleaners 2, and these distributors manage users as members. Next, a description will be given of the configuration of the cleaning management device 3.

[0169] First, the functional blocks of the cleaning management device 3 will be described with reference to Fig. 24. The cleaning management device 3 has a communication unit 301, a cleaner identification unit 302, an operation status detection unit 303, a cleaning area map creation unit 304, an AR information creation unit 305, a cleaning management unit 306, a history information creation unit 307, and a storage unit 308. First, the communication unit 301 connects to each user terminal and the router 4 via the network 5.

[0170] Furthermore, the cleaner identification unit 302 to the history information creation unit 307 execute the same processes as the cleaner identification unit 103 to the history information creation unit 108 in Examples 1 and 2, respectively. Furthermore, the storage unit 308 stores user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314. Each of these pieces of information will be described later.

[0171] Next, a description will be given of the hardware configuration of one implementation example of the cleaning management device 3. The cleaning management device 3 can be realized by a computer such as a server or a cloud system. Fig. 25 is a hardware configuration diagram of the cleaning management device 3 in Example 3. In Fig. 25, the cleaning management device 3 has a processing device 31, a communication device 32, a memory 33, and a secondary storage device 34, which are connected to each other via a communication path.

[0172] First, the processing device 31 can be realized by a processor such as a CPU, and executes calculations in accordance with a cleaning management program 35 stored in a secondary storage device 34 (described later). The communication device 32 corresponds to the communication unit 301 in Fig. 24, connects to the network 5, and communicates with other devices such as the user terminal 1.

[0173] 24 . The memory 33 and the secondary storage device 34 store the cleaning management program 35 and information used for processing by the processing device 31, which are stored in the secondary storage device 34. The secondary storage device 34 can be implemented as a so-called storage device. The secondary storage device 34 stores the cleaning management program 35, user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314. The secondary storage device 34 may be implemented as various storage media such as an external hard disk drive (HDD), solid state drive (SSD), or memory card, or may be implemented as a device separate from the cleaning management device 3, such as a file server.

[0174] The cleaning management program 35 is composed of a vacuum cleaner identification module 351, an operating status detection module 352, a cleaning area map creation module 353, an AR information creation module 354, a cleaning management module 355, and a history information creation module 356, for each of its functions. Note that each of these modules may be realized as an individual program or a partial combination. In particular, the AR information creation module 354 may be realized as a program with an augmented reality function separate from the cleaning management program. Furthermore, the cleaning management program 35 may be realized as one function of an integrated management application that manages multiple home appliances.

[0175] 24, which executes the same functions as each module, is as follows: Vacuum cleaner identification module 351: vacuum cleaner identification unit 302 Operation status detection module 352: operation status detection unit 303 Cleaning area map creation module 353: cleaning area map creation unit 304 AR information creation module 354: AR information creation unit 305 Cleaning management module 355: cleaning management unit 306 History information creation module 356: history information creation unit 307 Therefore, the processing device 31 executes the processes of the vacuum cleaner identification unit 302, operation status detection unit 303, cleaning area map creation unit 304, AR information creation unit 305, cleaning management unit 306, and history information creation unit 307 in accordance with the cleaning management program 35.

[0176] Here, the cleaning management program 35 executes the same processes as the cleaning management program 16 of the user terminal 1. The cleaning management program 35 can be stored in the secondary storage device 34 or other storage media.

[0177] Next, the user management information 310, cleaning management information 311, cleaning area map information 312, guidance information 313, and game information 314 will be described. FIG. 26 is a diagram showing the user management information 310 used in Example 3. The user management information 310 is information about the user of the vacuum cleaner and related parties. It has the same items as the user management information 112 in Examples 1 and 2, but in comparison, it handles information about multiple users. For example, information about other family members such as user B is added.

[0178] 27 is a diagram showing cleaning management information 311 used in Example 3. The cleaning management information 311 is information for managing the cleaning results of multiple users using vacuum cleaners. The cleaning management information 311 has the same items as the cleaning management information 113 in Examples 1 and 2, but in comparison, it handles multiple stick vacuum cleaners. For example, information on stick vacuum cleaner x has been added.

[0179] 28 is a diagram showing cleaning area map information 312 used in Example 3. The cleaning area map information 312 is map information that defines the layout of a usage location, such as a house or an office, where multiple cleaners, such as the stick vacuum cleaner 2 and the stick vacuum cleaner x, are used, i.e., each unit cleaning area that makes up the usage location. The cleaning area map information 312 has the same items as the cleaning area map information 114 in Examples 1 and 2, but, unlike the cleaning area map information 114, handles information on multiple users (usage locations). For example, information on user B has been added.

[0180] 29 is a diagram showing guidance information 313 used in Example 3. The guidance information 313 is information showing various kinds of guidance, such as terms of use and guidance on how to use the device, including user operations, for executing the cleaning management process and preparation process in Example 3. The guidance information 313 has the same items as the guidance information 115 in Examples 1 and 2, but compared to these, it handles a larger number of stick vacuum cleaners. For example, information on stick vacuum cleaner x has been added.

[0181] Next, a process flow for cleaning with the stick vacuum cleaner 2 in Example 3 will be described. Here, as in Examples 1 and 2, a process flow including a map creation process and a cleaning management process, which are examples of preparation processes for cleaning, will be described. Fig. 30 is a flowchart showing the process flow for cleaning in Example 3. Below, the configuration of the user terminal 1 will be described using the configuration shown in Fig. 3, and the configuration of the cleaning management device 3 will be described using the configuration shown in Fig. 24.

[0182] First, in steps S101 and S102, the user terminal 1 executes the same processes as steps S1 and S2 of the first embodiment. If it is determined in step S102 that this is the first use, the process proceeds to step S103, and if it is determined that this is the second or subsequent use, the process proceeds to step S111. Then, in step S103, the cleaning management unit 107 transmits a registration request for the user or related person to the cleaning management device 3 via the communication unit 101. This registration request includes the user / related person, address, contact information, and owned devices from the user management information 310.

[0183] In step S301, the communication unit 301 of the cleaning management device 3 receives the transmitted registration request.

[0184] In step S302, the cleaning management unit 306 transmits the terms of use to the user terminal 1 via the communication unit 301. Here, the cleaning management unit 306 searches the guidance information 313 for the terms of use corresponding to the owned device for which registration is requested, and transmits them.

[0185] In step S104, the communication unit 101 of the user terminal 1 receives the terms of use, and the cleaning management unit 107 displays them on the output unit 110. The display screen at this time is as described in the first embodiment. The display screen in the third embodiment is the same as in the first and second embodiments. Then, when consent to the terms of use is received from the user via the input unit 109, in step S105 the cleaning management unit 107 transmits a confirmation notice to the cleaning management device 3 indicating that the consent has been confirmed.

[0186] In response to this, when the communication unit 301 of the cleaning management device 3 receives the confirmation notification, in step S303 the cleaning management unit 306 performs registration processing in response to the registration request. Specifically, the cleaning management unit 306 registers the user / related person, address, contact information, and owned devices included in the registration request in the user management information 310.

[0187] Furthermore, in step S304, the cleaning management unit 306 determines whether the user management information 310 and cleaning management information 311 for the corresponding stick vacuum cleaner 2 have already been registered by another user terminal or another related party. If they have already been registered, the process proceeds to step S305. If they have not yet been registered, the process proceeds to step S306. Note that when proceeding to step S305, it is desirable for the cleaning management unit 306 to add the user / related party information registered in step S303 to the registered record. For this reason, step S303 may be executed after this step.

[0188] In step S305, the cleaning management unit 306 transmits the usage instructions to the user terminal 1 via the communication unit 301. To do this, the cleaning management unit 306 searches for the functions and guidance content of the corresponding stick vacuum cleaner 2 from the guidance information 313. The cleaning management unit 306 then transmits these as usage instructions via the communication unit 301.

[0189] Then, when the communication unit 101 of the user terminal 1 receives the usage instructions, in step S110, the cleaning management unit 107 displays the usage instructions on the output unit 110. This display screen is the same as in step S5 of Example 1. Also, in step S109, the cleaning management unit 107 displays a home screen on the output unit 110, similar to step S14 of Example 1.

[0190] In step S306, the cleaning management unit 306 transmits instructions for use and a map creation instruction to the user terminal 1 via the communication unit 301.

[0191] Then, when the communication unit 101 of the user terminal 1 receives the instruction for use and map creation, in response to this, in step S106, the cleaning management unit 107 causes the output unit 110 to display an instruction screen for use and map creation. That is, the same processes as in steps S5 and S6 of the first embodiment are executed.

[0192] Furthermore, in step S107, a map creation process is executed. That is, the processes described in steps S6 to S12 of FIG. 9A in the first embodiment are executed. Then, in step S108, the cleaning area map creation unit 105 transmits a registration request including the created cleaning area map to the cleaning management device 3 via the communication unit 101. Note that the created cleaning area map may be stored in the storage unit 111 of the user terminal 1, and steps S108 to S308 may be omitted.

[0193] Furthermore, when the communication unit 301 of the cleaning management device 3 receives a registration request transmitted from the user terminal 1, in step S307 the cleaning management unit 306 registers the accepted cleaning area map in the cleaning area map information 312. Then, in step S308, the cleaning management unit 306 transmits a registration notification to the user terminal 1 via the communication unit 301 indicating that the cleaning area map has been registered.

[0194] When the communication unit 101 of the user terminal 1 receives the registration notification, step S109 is executed. Furthermore, in step S111, a cleaning management process is executed. That is, the processes described in steps S15 to S24 of FIG. 9B in the first embodiment are executed. Of course, step S22 in this cleaning management process can be realized by the cleaning-related information display process shown in FIG. 18 or FIG. 21.

[0195] Furthermore, in step S112, the cleaning management unit 107 sends a data update request including various information resulting from the cleaning execution to the cleaning management device 3 via the communication unit 101. This information includes, for example, cleaning results and calculated points. Then, in step S309, the communication unit 301 of the cleaning management device 3 accepts the data update request. In response to this, in step S310, the cleaning management unit 306 registers the various information included in the data update request in the storage unit 308.

[0196] This concludes the explanation of the flowchart in FIG. 30 . However, in addition to the processing described here, related information may be sent to the user terminal 1-a of the relevant party. The related information includes the cleaning record and calculated points included in the data update request. Furthermore, various display screens in the map creation processing of step S107 and the cleaning management processing of step S11 may be shared between the user terminal 1 and the user terminal 1-a. Furthermore, the user terminal 1 and the user terminal 1-a can access their own information among the user management information 310, cleaning management information 311, cleaning area map information 312, and guidance information 313 stored in the memory unit 308 of the cleaning management device 3.

[0197] This concludes the description of Example 3. According to Example 3, the above-described process is executed in cooperation with a user terminal 1 such as a smartphone and a cleaning management device 3 such as a server. This allows appropriate resources to be used, enabling efficient processing. Furthermore, the execution entity of each step shown in FIG. 30 is merely an example, and some of the steps may be executed by another device (the user terminal 1 or the cleaning management device 3).

[0198] The user terminal 1 and the cleaning management device 3 in Examples 1 to 3 are examples of information processing devices for realizing the cleaning management system. Therefore, the cleaning management system may be configured to include either the user terminal 1 or the cleaning management device 3.

[0199] Next, a fourth embodiment relating to the structure of the terminal holder 20 will be described. Fig. 31 is a diagram schematically illustrating a side view of the stick vacuum cleaner 2 in the fourth embodiment when the terminal holder 20 is attached thereto. As also described in Fig. 1 , the stick vacuum cleaner 2 has a vacuum cleaner body 21 and a vacuum cleaner head 24, which are connected via an extension tube 23. The vacuum cleaner body 21 is provided with a handle 22 that is held by the user. The lower part of the vacuum cleaner body 21 has a tubular or annular connector 211 for connecting to the extension tube 23. The stick vacuum cleaner 2 in the fourth embodiment has a communication function and other information processing functions, and may be linked to a user terminal 1, a cleaning management device 3, etc.

[0200] Here, the terminal holder 20 is detachably installed on the vacuum cleaner main body 21, more preferably on the connection part 211. For example, the terminal holder 20 is configured in a ring-like or tubular shape and is fixed via an adapter or the like that wraps around the connection part 211. The terminal holder 20 may be attached to a part other than the extension tube 23 or the connection part 211. Furthermore, when the extension tube 23 of the stick vacuum cleaner 2 is removed and a replacement nozzle (tube) is provided, the terminal holder 20 may be connected to the nozzle. The replacement nozzle is a nozzle that is shorter than the extension tube 23 and includes a brush nozzle and a tapered nozzle. Furthermore, it is desirable for the replacement nozzle to have a cableless structure that omits a cable that controls the vacuum cleaner head 24.

[0201] Next, Fig. 32 is an enlarged view of Fig. 31 showing the state in which the terminal holder 20 in Example 4 is attached. In Fig. 32, the terminal holder 20 is fixed to the connection part 211 via the attachment part 200. At this time, it is desirable to position the holding part 20-1 of the terminal holder 20 so that it is at an angle of approximately 100° from the direction of the connection part 211 and the extension tube 23. However, this angle can be fine-tuned by moving the holding part 20-1 or the arm part 20-2, which will be described later. This allows the imaging part 102 of the user terminal 1 to capture images of the cleaner head 24 and the floor surface to be cleaned.

[0202] Here, the terminal holder 20 has at least a holding portion 20-1, an arm portion 20-2, and a base 20-3, and is attached to the stick vacuum cleaner 2. This attachment is preferably performed via an attachment portion 200 that the terminal holder 20 has.

[0203] First, the holding unit 20-1 securely holds the user terminal 1 by clamping it, for example. The base 20-3 and the attachment unit 200 are connected, and the terminal holder 20 is fixedly attached to the stick vacuum cleaner 2. The arm unit 20-2 then connects the holding unit 20-1 to the base 20-3. In the fourth embodiment, the base 20-3 or the terminal holder 20 and the attachment unit 200 are configured separately. However, these may be configured as an integrated unit, or may be configured as an integrated unit or separable unit. In other words, the terminal holder 20 of the fourth embodiment may be configured with the holding unit 20-1, the arm unit 20-2, and the base 20-3, or may be configured with the attachment unit 200 added to these. This allows the terminal holder 20 to be shared with multiple vacuum cleaners, particularly vacuum cleaners with different specifications such as size. As described above, in the fourth embodiment, the terminal holder 20 having the holding unit 20-1 is detachably attached to the vacuum cleaner body of the stick vacuum cleaner 2.

[0204] Next, an example of the structure of the mounting portion 200 according to the fourth embodiment will be described with reference to Fig. 33 and Fig. 34. Fig. 33 is a diagram showing a first example of the mounting portion 200 according to the fourth embodiment. Here, Fig. 33 is a view seen from the direction A-A shown in Fig. 31. The same applies to Fig. 34.

[0205] In Figure 33, the bottom of the terminal holder 20 is connected to the mounting portion 200. This mounting portion 200 has an upper mounting portion 200-1, a lower mounting portion 200-2a, and a lower mounting portion 200-2b, and its cross section is configured to be annular. Here, by moving in the direction of the arrows (left and right), the lower mounting portion 200-2a and the lower mounting portion 200-2b, together with the upper mounting portion 200-1, sandwich the connection portion 211. For this reason, the upper mounting portion 200-1 and the lower mounting portion 200-2a and the lower mounting portion 200-2b are movably connected by pins.

[0206] Then, the lower mounting portion 200-2a and the lower mounting portion 200-2b are fixedly connected with screws or nails, and a force is generated in the left direction in the figure. This causes the mounting portion 200 to tighten the connection portion 211. As a result, the terminal holder 20 connected to the mounting portion 200 is fixed to the connection portion 211.

[0207] Next, Figure 34 is a diagram showing a second example of the mounting portion 200 in Example 4. In Figure 34, the mounting portion 200 is also connected to the lower part of the terminal holder 20. This mounting portion 200 has an upper mounting portion 200-3, a lower mounting portion 200-4a, and a lower mounting portion 200-4b, and its cross section is configured to be annular. Here, by moving in the direction of the arrows (left and right), the lower mounting portion 200-4a and the lower mounting portion 200-4b, together with the upper mounting portion 200-3, sandwich the connection portion 211.

[0208] Here, upper mounting portion 200-3 is connected to lower mounting portion 200-4a and lower mounting portion 200-4b via springs that generate force in the tightening direction. In this way, mounting portion 200 tightens connecting portion 211. As a result, terminal holder 20 connected to mounting portion 200 is fixed to connecting portion 211.

[0209] With the configuration of the fourth embodiment described above, the user terminal 1 can be fixedly held in the terminal holder 20. As a result, the user of the stick vacuum cleaner 2 can use the user terminal 1 and benefit from various information processing functions, including information processing related to cleaning.

[0210] Next, a fifth embodiment, which is different from the fourth embodiment and relates to the structure of the terminal holder 20, will be described. The fifth embodiment illustrates a structure in which the terminal holder 20 is connected to a handy brush 25 that can be attached to and detached from the stick vacuum cleaner 2. The handy brush 25 is a type of accessory, and is tubular, but is a shorter pipe than the extension tube 23. The handy brush 25 may also be referred to as a shelf brush, round brush, brush nozzle, etc. Various types of nozzles without brushes may also be used. Examples of such nozzles include crevice nozzles and water-absorbing nozzles. The stick vacuum cleaner 2 of the fifth embodiment has communication functions and other information processing functions, and may be linked to the user terminal 1, the cleaning management device 3, etc.

[0211] FIG. 35 is a diagram schematically illustrating a side view of the stick vacuum cleaner 2 according to Example 5 when the terminal holder 20 is attached thereto. As described in FIG. 1 , the stick vacuum cleaner 2 has a vacuum cleaner body 21 and a vacuum cleaner head 24, which are connected via an extension tube 23. The vacuum cleaner body 21 is provided with a handle 22 that is held by a user. A handy brush 25 is attached to the bottom of the vacuum cleaner body 21. The extension tube 23 is connected to the vacuum cleaner body via the handy brush 25. Note that "via the handy brush 25" includes the following aspects: (1) the handy brush 25 and the extension tube 23 may each be connected to the vacuum cleaner body 21, and the extension tube 23 may extend into the hollow portion of the handy brush 25; or (2) the extension tube 23 may be connected to the extension tube 23, and the handy brush 25 may be connected to the vacuum cleaner body 21.

[0212] Here, the connection between the terminal holder 20 and the handy brush 25 will be described. FIG. 36 is an external view showing the state in which the terminal holder 20 and the handy brush 25 are connected in Example 5. In FIG. 36, the terminal holder 20 is installed so as to be detachable from the handy brush 25. For this purpose, the terminal holder 20 is connected to the handy brush 25 via the attachment part 20-4, which is fixed (locked) and released (released) by the elastic force of a spring provided in the attachment part 20-4. Note that, in addition to the spring, the terminal holder 20 and the handy brush 25 may be fixed by a claw such as a so-called side release buckle, or may be fixed by generating frictional force using a groove provided in the handy brush 25.

[0213] Next, Fig. 37 is an enlarged view of Fig. 35 showing the state in which the terminal holder 20 in Example 5 is attached. In Fig. 37, the terminal holder 20 is fixed to the vacuum cleaner body 21 via the handy brush 25. At this time, it is desirable to position the holding part 20-1 of the terminal holder 20 so that it is at approximately 100° from the direction of the handy brush 25 and the extension tube 23. However, this angle can be finely adjusted by moving the holding part 20-1 or the arm part 20-2. This allows the image capturing unit 102 of the user terminal 1 to capture images of the vacuum cleaner head 24 and the floor surface to be cleaned.

[0214] Here, the terminal holder 20 has at least a holding part 20-1, an arm part 20-2, and an attachment part 20-4, and is attached to the stick vacuum cleaner 2 via the handy brush 25. In this way, in the fifth embodiment, an accessory such as the handy brush 25 is used to physically connect the user terminal 1 to the stick vacuum cleaner 2, so that the accessory can be used efficiently.

[0215] Next, a sixth embodiment will be described regarding the structure of the terminal holder 20. In the sixth embodiment, the holding portion 20-1, the arm portion 20-2, and the base 20-3 of the terminal holder 20 in the fourth and fifth embodiments are integrated together. In the sixth embodiment, the terminal holder 20-10 is referred to as the terminal holder 20-10.

[0216] First, Figure 38 is a perspective view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In Figure 38, the terminal holder 20-10 is provided with upper ribs 20-11a, 20-11b and a lower rib 20-13 to hold the user terminal 1. The upper ribs 20-11a, 20-11b and the lower rib 20-13 make it difficult for the user terminal 1 placed on the terminal holder 20-10 to move toward the front right in the drawing, making it difficult for the user terminal 1 to come off the terminal holder 20-10. Note that it is sufficient to provide at least one of the upper ribs 20-11a, 20-11b and the lower rib 20-13.

[0217] Additionally, receiving portions 20-12a and 20-12b are provided inside the upper ribs 20-11a and 20-11b of the terminal holder 20-10 (receiving portion 20-12b is not shown due to a blind spot). These receiving portions 20-12a and 20-12b accommodate both sides of the user terminal 1. For this reason, receiving portions 20-12a and 20-12b are implemented as grooves or gaps extending in the thickness direction of the user terminal 1. This makes it less likely that the terminal holder 20-10 will interfere with switches (buttons) provided on the side of the user terminal 1. In other words, when the user terminal 1 is placed in the terminal holder 20-10, accidentally pressing a switch can be prevented or suppressed. For this reason, it is desirable to make the width of the grooves or gaps in receiving portions 20-12a and 20-12b slightly shorter than the thickness of the user terminal 1 (for example, the thickness of a typical smartphone). This allows the front and back of the user terminal 1 to be fixed with these, and the switches can be placed in the grooves or gaps and not in contact with the terminal holder 20-10.

[0218] Furthermore, the grooves and gaps in the receiving portion 20-12a and the receiving portion 20-12 make it difficult for the user terminal 1 to come off the terminal holder 20-10. Note that it is sufficient to provide either the receiving portion 20-12a or 20-12b. Furthermore, it is also possible to provide either the upper ribs 20-11a, 20-11b and the lower rib 20-13, or the receiving portions 20-12a and 20-12b.

[0219] The terminal holder 20-10 is connected to the handy brush 25 below (only a portion of the handy brush 25 is shown in the figure). The configuration of the terminal holder 20-10 will be described in detail below. Figure 39A is a side view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. The upper rib 20-11b and the lower rib 20-13 can be seen on the side of the terminal holder 20-10. In particular, the lower rib 20-13 extends upward as circled in the figure, which makes it difficult for the user terminal 1 to come off. Next, a top view (direction A) and a bottom view (direction B) of the terminal holder 20-10 will be described.

[0220] Fig. 39B is a top view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In Fig. 39B, the upper rib 20-11a and the upper rib 20-11b each extend toward the center of the terminal holder 20-10 as circled in the figure, which makes it difficult for the user terminal 1 to come off.

[0221] Furthermore, receiving portions 20-12a and 20-12b are configured as grooves into which both ends of the user terminal 1 fit, preventing or inhibiting switch malfunction. Furthermore, receiving portions 20-12a and 20-12b are tapered (shaped like the letter "V" in the figure), meaning that receiving portions 20-12a and 20-12b are located approximately in the center of the "V" shape. As a result, the user terminal 1 can be more securely attached to receiving portions 20-12a and 20-12b, which are located in the center. Therefore, switches on the side of the user terminal 1 are positioned in the spaces of the grooves and gaps, preventing malfunction.

[0222] Furthermore, the terminal holder 20-10 is provided with an opening / closing section 20-14 realized by a hinge structure near the upper rib 20-11a. This opening / closing section 20-14 acts as an axis, allowing one end of the terminal holder 20-10 to open (rotate) in the direction of the arrow in the figure. As a result, the user terminal 1 can be attached (placed) or removed from the terminal holder 20-10. In this way, the rotation of one end of the terminal holder 20-10 allows for easy attachment and removal of the user terminal 1 with a simple structure.

[0223] FIG. 39C is a bottom view showing the terminal holder 20-10 in Example 6 connected to the handy brush 25. In FIG. 39C, the terminal holder 20-10 is provided with a circular protrusion 20-15 (the Ω-shaped protrusion within the dashed line in the figure). The circular protrusion 20-15 is made of an elastic material such as metal, and is fixedly attached to the handy brush 25 by its elastic force. Furthermore, in Example 6, a structure is provided for fixedly connecting the terminal holder 20-10 and the handy brush 25. This will be explained next.

[0224] 40A is a rear view showing the state in which the terminal holder 20-10 in Example 6 is connected to the handy brush 25. At the location shown in the CC cross section of this rear view, a claw portion is provided on the terminal holder 20-10, and by combining with a protrusion of the handy brush 25, the terminal holder 20-10 is fixedly installed.

[0225] Figure 40B is a cross-sectional view taken along CC in Figure 40A in Example 6. In Figure 40B, the terminal holder 20-10 is provided with a claw portion 20-16. In contrast, the handy brush 25 is formed with a protrusion 25-1. The claw portion 20-16 and the protrusion 25-1 are interlocked with each other, so that the claw portion 20-16 does not easily come off the protrusion. As a result, the handy brush 25 is fixedly attached to the terminal holder 20-10.

[0226] In the sixth embodiment described above, the terminal holder 20-10 is attached to the handy brush 25, but the present invention is not limited to this. For example, the terminal holder 20-10 may be attached to the vacuum cleaner body 21 or the extension tube 23. Furthermore, each of the above-described configurations may be provided in the terminal holder 20 as in the fourth or fifth embodiment.

[0227] Next, a seventh embodiment will be described, which illustrates a modified example of the map creation process. In the first to third embodiments, the cleaning area map is created by moving the stick vacuum cleaner 2 to which the user terminal 1 is attached. In the seventh embodiment, an example will be described in which the cleaning area map is created by operating the user terminal 1.

[0228] FIG. 41 is a diagram for explaining the map creation process in Example 7. More specifically, FIG. 41 shows a display screen for explaining the operation of the user terminal 1 in creating a cleaning area map. Before explaining FIG. 41 , the premise of Example 7 will first be explained. In Example 7, the above-mentioned marker unit is used. However, the marker unit may be provided somewhere other than the cleaner head 24. For example, it may be provided on the charging base 26, or more preferably on the base member 29. Furthermore, it is desirable to use a so-called two-dimensional code as the marker unit. Below, the map creation process of Example 7 will be explained with reference to FIG. 41.

[0229] First, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for starting map creation. Unlike in Examples 1 to 3, this display screen does not prompt the user to attach the user terminal 1, but rather prompts the user to read a two-dimensional code, which is an example of a marker, as shown in FIG. 41(a). If the marker is successfully read, the cleaning area map creation unit 105 causes the output unit 110 to display evidence of this (for example, a white marker). It is desirable to display the display content for the successful case, as shown in FIG. 41(a).

[0230] Next, in response to a user operation or the like, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining the creation of the map shown in Fig. 41(b). Here, when creating the map, the cleaning area map creation unit 105 causes the output unit 110 to display an AR image created by the AR information creation unit 106 by superimposing a marker on the captured image. Therefore, the cleaning area map creation unit 105 displays an explanation instructing the user to virtually place the marker of the AR image in the corner of a room (an example of a unit cleaning area) as shown in Fig. 41(b).

[0231] Next, in response to a user's operation, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining map creation, as shown in FIG. 41(c). FIG. 41(c) illustrates an operation for creating a cleaning area map that combines multiple rooms. Here, a prompt to move the location, i.e., the room, and place a marker is displayed. The middle section of FIG. 41(c) displays an image of the cleaning area map being created. It also displays that if the area under an obstacle, such as a table, is to be included in the cleaning area, no markers need to be placed. Furthermore, FIG. 41(c) displays a prompt to create a cleaning area map that combines multiple rooms by tapping around the room in a certain direction (e.g., clockwise or counterclockwise).

[0232] Then, in response to a user's operation, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen for explaining the creation of the map shown in FIG. 41(d). FIG. 41(d) indicates that the cleaning area map is completed when the complete button is pressed. Then, when FIG. 41(d) is pressed, the cleaning area map creation unit 105 starts creating the cleaning area map. To this end, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning area map creation unit 105. Then, during the map creation process, the cleaning area map creation unit 105 displays information in accordance with the operation content shown in FIG. 41 described above. For example, the display may include a marker, which is AR information, as shown in FIGS. 41(a) and (b), or an image of the cleaning area map being created or created, as shown in FIGS. 41(c) and (d).

[0233] Next, Example 8 is an example of a display format of AR information including AR icons and cleaning-related information, and an example of dynamically changing the display format. Thus, Example 8, like Examples 1 to 3, performs a display process for cleaning-related information including AR information. For example, in one specific example of Example 8, the recommended area of ​​Example 2 is displayed. However, like Examples 1 and 3, the display of the recommended area can be omitted, and Example 8 can be combined with any of Examples 1 to 3. Note that Example 8 will be described using substantially the same configuration and information as Examples 1 and 2. Therefore, the following description will focus on the differences, particularly the processing flow and display screen.

[0234] Fig. 42 is a flowchart for explaining the details of the display process in Example 8. The process up to this flowchart is executed in the same way as up to S20 in Fig. 9B. Then, in step S41, similar to step S21 in Fig. 9B, the operational status detection unit 104 detects the operational status of the stick vacuum cleaner 2. However, in step S41, the operational status detected includes the stick vacuum cleaner 2 (particularly the position of the cleaner head 24), the position that has been cleaned, the movement speed, the amount of dust collected, the remaining charge (or the amount of charge), and whether the cleaner head has been replaced. Furthermore, the cleaning performance history can also be used as the operational status.

[0235] Furthermore, in step S42, the cleaning management unit 107 creates determination information related to display based on the operating status. Creating this determination information includes calculating and creating the following: Identifying the AR icon to be displayed; The distance between the AR icon and the stick vacuum cleaner 2 (particularly the cleaner head 24); The number of contacts between the AR icon and the stick vacuum cleaner 2 (particularly the cleaner head 24); The distance of the movement trajectory of the stick vacuum cleaner (particularly the cleaner head 24); The area indicated by this movement trajectory; The coverage rate of the movement trajectory relative to the recommended area; The amount of exercise and calories burned due to cleaning; Power consumption. Furthermore, the operating status itself, such as the remaining charge, may be used as the determination information.

[0236] Furthermore, in step S43, the cleaning management unit 107 specifies the display format in accordance with the determination information. To this end, the cleaning management unit 107 determines whether or not a predetermined AR icon is to be displayed and whether or not the display format of the AR icon, etc., is to be changed. Then, the cleaning management unit 107 specifies the AR icon to be displayed and the display format to be changed. In step S43, the AR icons to be displayed, their display formats, the number of icons, the movement trajectory and its display format, the cleaning history and its display format, etc. are specified. The determination in step S43 includes using a threshold value previously stored in the storage unit 111. For example, if the distance between the AR icon and the stick vacuum cleaner 2 is within the distance threshold value, the display format of the AR icon is changed. Note that specifying the display format in Example 8 includes specifying the display format (format such as color and size) and whether or not a predetermined display object (such as an AR icon) is to be displayed.

[0237] In response to this, in step S44, the cleaning management unit 107 causes the output unit 110 to display the display format determined in step S43. In this way, in the second embodiment, various displays are executed on the user terminal 1 for the user depending on the cleaning status. As a result, the user's motivation for cleaning can be improved. Below, a specific example of the eighth embodiment will be described, but these can be implemented in combination with each other. Furthermore, at least a part of steps S42 to S44 may be executed by the AR information creation unit 106, or may be executed in cooperation between the cleaning management unit 107 and the AR information creation unit 106.

[0238] <Specific Example 1> In Specific Example 1, the position of the vacuum cleaner head 24 is detected as the position of the stick vacuum cleaner 2, and a dust monster, which is a type of AR icon, is displayed according to that position. Therefore, Specific Example 1 is premised on the game functions of Examples 1 to 3. Furthermore, in Specific Example 1, the display format of the dust monster is changed according to the positional relationship between the position of the vacuum cleaner head 24 and the dust monster. Details of this will be explained below.

[0239] 43 is a flowchart showing details of the display process in specific example 1 of Example 8. In step S20, the photographing unit 102 is activated and starts photographing in response to an instruction from the cleaning management unit 107. In addition, in step S21, the operating status detection unit 104 starts detecting the operating status of the stick vacuum cleaner 2. These are the same processes as in Example 1, but as described above, the detection of the operating status in Example 8 is not limited to the cleaned position, whereas in specific example 1, the cleaned position, i.e., the position of the vacuum cleaner head 24, which is an example of the position of the stick vacuum cleaner 2, is used. Here, the cleaning management unit 107 uses the detection and identification of the cleaning position by the operating status detection unit 104 as determination information.

[0240] Also, in step S401, the cleaning management unit 107 specifies, as the display format, a screen of the dust monster and the cleaner head 24 of the captured image taken by the image capture unit 102. Then, the cleaning management unit 107 displays the dust monster and the cleaner head 24 on a screen of the output unit 110. This display is similar to the display screen 1109a in FIG. 19A of the first embodiment. That is, the dust monster 1109a-2 and the cleaner 1109a-3 are displayed on the same screen on the display screen 1109a. Here, the cleaner 1109a-3 in FIG. 19A includes the cleaner head 24 of the stick cleaner 2 included in the captured image.

[0241] Furthermore, the dust monster 1109a-2 is displayed to allow the user to acquire points in the game function. Therefore, the dust monster 1109a-2 acts as an inducement for cleaning, that is, as a motivation for pointing the vacuum cleaner head 24 at it. By displaying the dust monster 1109a-2 and the vacuum cleaner head 24 on the same screen, the user will naturally move the vacuum cleaner head 24 toward the dust monster 1109a-2 to clean. Note that while Specific Example 1 may end at step S401, it is desirable to execute the processing from step S402 onward in order to further increase motivation for cleaning.

[0242] In step S402, the cleaning management unit 107 calculates the virtual distance between the dust monster 1109a-2 and the cleaner head 24, i.e., the distance on the display screen, as determination information. To do this, the cleaning management unit 107 calculates the position of the cleaner head 24 (cleaner 1109a-3) on the display screen based on the position where cleaning has been completed. The cleaning management unit 107 then calculates the distance between the dust monster 1109a-2 and the cleaner head 24 on the display screen. This calculation is performed continuously while this flowchart is being executed.

[0243] In step S403, the cleaning management unit 107 determines whether the calculated interval is equal to or less than the interval threshold value stored in the memory unit 111. In other words, it determines whether the user's operation of the stick vacuum cleaner 2 has caused the cleaner head 24 to approach the interval threshold value or less relative to the dust monster 1109a-2. If the result shows that the interval is equal to or less than the interval threshold value (YES), the process proceeds to step S404. If the result shows that the interval is not equal to or less than the interval threshold value (NO), the process proceeds to step S405.

[0244] In step S404, the cleaning management unit 107 changes the expression of the dust monster as a display format. In specific example 1, the dust monster 1109a-2 in Fig. 19A has a normal expression (e.g., a poker face), but is changed to a troubled expression (troubled face). The cleaning management unit 107 then displays the dust monster 1113a-2 with the changed expression and the vacuum cleaner 1113a-3 including the vacuum cleaner head 24 on one display screen 1113a of the output unit 110.

[0245] 44A is a diagram showing a display screen 1113a in specific example 1 of working example 8. In Fig. 44A, the display screen 1113a displays an operating map icon 1113a-1, a dust monster 1113a-2, a vacuum cleaner 1113a-3, and a point display area 1113a-4, similar to Fig. 19A. First, the operating map icon 1113a-1 shows the cleaned area, including the cleaning position and movement trajectory, according to the progress of cleaning, compared to the operating map icon 1109a-1.

[0246] Furthermore, the dust monster 1113a-2's facial expression changes from that of the dust monster 1109a-2 when the vacuum cleaner head 24 represented by the vacuum cleaner 1113a-3 approaches within the distance threshold. In the example shown in FIG. 44A , the dust monster's facial expression changes to a troubled one. However, the changes to the dust monster 1113a-2 are not limited to this and may include a panicked expression, an aggressive expression, a change in color (warning color), a flashing display, a change in size, and so on. Furthermore, these changes may be combined. Note that, like the point display area 1109a-4, the point display area 1113a-4 shows the accumulated points calculated by the dust monster 1113a-2 virtually sucking in dust, etc., during the cleaning. Here, in this example, if a certain number of dust monsters are sucked in, the dust monster can obtain an item called an acorn, and the number of items obtained is displayed as points.

[0247] In step S405, the cleaning management unit 107 continues displaying the dust monster while maintaining its facial expression, which is its display format. At this time, the distance between the dust monster 1109a-2 and the vacuum cleaner 1109a-3, etc., is changed as appropriate in accordance with the movement of the vacuum cleaner head 24. Then, the process returns to step S403 and continues.

[0248] Furthermore, in step S406, it is determined whether the vacuum cleaner 1113a-3 and the dust monster 1113a-2 have come into contact on the display screen based on the distance calculated in step S402. In step S406, as in steps S402 and S403, the position of the vacuum cleaner head 24 (vacuum cleaner 1109a-3) on the display screen is calculated based on the position where cleaning has been completed. Then, based on this position, the cleaning management unit 107 determines whether the vacuum cleaner head 24 has come into contact with the dust monster 1113a-2, that is, whether the distance between them has become zero.

[0249] If the contact is detected (YES), the process proceeds to step S407. If the contact is not detected (NO), the process proceeds to step S408.

[0250] Also, in step S407, the cleaning management unit 107 changes the display format of the dust monster so that it appears as if it is being sucked into the cleaner head 24, and causes the output unit 110 to display this. At this time, the dust monster changes its shape over time, as if it is being sucked into the suction port of the cleaner head 24, and is eventually erased. As a result, the cleaning management unit 107 finally causes the output unit 110 to display a display screen from which the dust monster has been erased, as shown in FIG. 19B.

[0251] Furthermore, in step S408, the cleaning management unit 107 continues displaying the dust monster 1113a-2 while maintaining the troubled face that is its display format. At this time, the distance between the dust monster 1113a-2 and the vacuum cleaner 1113a-3, etc., is changed as appropriate in accordance with the movement of the vacuum cleaner head 24. Then, the process returns to step S406 and continues. However, if this distance becomes greater than or equal to the distance threshold, it is desirable for the cleaning management unit 107 to return the dust monster's facial expression to the normal expression shown in FIG. 19A. In this case, the process returns to step S403.

[0252] In step S409, the cleaning management unit 107 determines whether the number of contacts between the dust monster and the cleaner head 24 is equal to or greater than the contact threshold value previously stored in the storage unit 111. If the number of contacts is equal to or greater than the contact threshold value (YES), the process proceeds to step S410. If the number of contacts is not equal to or greater than the contact threshold value (NO), the process proceeds to step S411.

[0253] Furthermore, in step S410, the cleaning management unit 107 adds, for example, 1 to the number of points (i.e., the number of items indicated by acorns) in the point display area 1109a-4 of FIG. 19A or the point display area 1113a-4 of FIG. 44A. For example, the number of items may be displayed numerically, as in the point display area 1109b-3 of FIG. 19B, or may indicate that all dust monsters have been sucked up, such as MAX. Then, in response to the user's operation of the stick vacuum cleaner 2, the process returns to step S401. The cleaning management unit 107 records the added number of items (points) in the user management information 112. As a result, the points in the user management information 112 are updated. Furthermore, the size of the point display area 1109b-3 itself may be increased or the color may be changed according to the added number of items.

[0254] In step S411, the cleaning management unit 107 continues processing while maintaining the number of points in the point display area 1109a-4 and the point display area 1113a-4, i.e., the number of items. In this case, too, the process returns to step S401 again in response to the user's operation on the stick vacuum cleaner 2.

[0255] As described above, in steps S402 to S410 of specific example 1, the display mode is specified and changed depending on the proximity situation. Note that specific example 1 may also be executed as follows. The determination information in step S21 may be an image or display of the cleaner head 24, or whether a preset display position of a dust monster is included within the display screen. Alternatively, the determination may be made based on whether a predetermined time has elapsed since the start of cleaning.

[0256] Furthermore, specific example 1 also includes the following modified example. In the above, when the number of contacts is equal to or greater than the contact threshold, the number of items is incremented, but an icon other than the dust monster is displayed. This may be performed in place of step S410, or may be performed in conjunction with it. This modified example will be described below.

[0257] In step S410, the cleaning management unit 107 displays a dust monster other than the dust monster on the output unit 110. That is, a dust monster is displayed in place of the sucked-in dust monster. FIG. 44B is a diagram showing a display screen 1113b in a specific example 1 (variant) of Example 8. In FIG. 44B, n dust monsters 1113b-1 to 1113b-n are displayed on the display screen 114a. In addition to the dust monsters 1113b-1 to 1113b-n, the display screen 1113b also displays an operating map icon 1113b-3, a vacuum cleaner 1113b-4, and a point display area 1113b-5. These are similar to the operating map icon 1113a-1, vacuum cleaner 1113a-3, and point display area 1113a-4 described above. However, the operating map icon 1113a-1 may be a graph showing the cleaning amount instead of a map.

[0258] These trash monsters 1113b-1 to 1113b-n appear when the player inhales the dust monster at a contact threshold or more, so their appearance probability is lower than that of the dust monster. The trash monsters can be treated as so-called rare characters. Furthermore, the points per trash monster 1113b-1 to 1113b-n, i.e., the number of items, will be greater than that of the trash monsters. Alternatively, it may be possible to obtain other, more valuable items.

[0259] Here, it is desirable that the number of dust monsters 1113b-1 to 1113b-n displayed on the display screen 1113b, which is one screen, is greater than the number of dust monsters, but this is not limited to this. Also, it is more desirable that n be around 2 to 4. It is also possible to display multiple dust monsters on one screen.

[0260] Furthermore, the change in display mode according to the proximity situation in steps S402 to S410 may be applied to the dust monster. This will be described below with reference to FIG. 44C. First, if it is determined in steps S402 to S403 that the distance between the dust monster and the cleaner head is equal to or less than the distance threshold, the process proceeds to step S404. In step S404, the cleaning management unit 107 causes the output unit 110 to display the display screen 1113c of FIG. 44C. That is, the dust monster is trembling uncontrollably, and its expression has changed to one of distress.

[0261] Furthermore, as a result of operating the stick vacuum cleaner 2, the cleaner head 24 approaches the dust monster and comes into contact with it. In this case, the cleaning management unit 107 causes the output unit 110 to display display screen 1113d of Fig. 44C. That is, the display transitions from display screen 1113c to display screen 1114d. As a result, in step S406, the cleaning management unit 107 determines that the cleaner head 24 has come into contact with the dust monster.

[0262] Then, in step S407, the cleaning management unit 107 causes the output unit 110 to display the display screen 1113e in which the contacted trash monster has been erased. At this time, points (number of items) are added to the point display area 1113b-5, and the cleaning management unit 107 records this result in the user management information 112. As a result, the points in the user management information 112 are updated. Furthermore, as described above, the size of the point display area on the display screen 1113e itself may be increased or the color may be changed according to the number of items added.

[0263] <Specific Example 2> Next, specific example 2 realizes a display in a display format that corresponds to the cleaning operation speed. To this end, specific example 2 describes an example in which the display format is specified according to the movement speed of the stick vacuum cleaner 2 or the user terminal 1. Here, the processing content will be described using FIG. 42 . First, in step S41, the operation status detection unit 104 detects the movement speed of the user terminal 1 or the cleaner head 24. To this end, an acceleration sensor or a speed sensor may be provided in the cleaner head 24 or the user terminal 1, and the operation status detection unit 104 may acquire this detection result. Alternatively, the movement speed may be detected by performing image analysis on images captured by the imaging unit 102. Furthermore, the operation status detection unit 104 may detect the cleaned position for each hour.

[0264] Furthermore, in step S42, the cleaning management unit 107 uses, as determination information, the movement speed detected by the operation status detection unit 104. When the moved position is used, the cleaning management unit 107 calculates the movement speed from the change in the cleaned position over time.

[0265] In step S43, the cleaning management unit 107 determines which of the speed information stored in advance in the storage unit 111 corresponds to the movement speed identified in step S42. Here, the speed information is information that associates a display format with each movement speed (range). As a result, the cleaning management unit 107 identifies a display format that corresponds to the identified movement speed. This display format includes the display format of the surrounding display area and the movement trajectory.

[0266] Then, in step S44, the cleaning management unit 107 displays the surrounding display area and the movement trajectory in the display format determined in step S43 on the output unit 110. In step S44, at least one of the surrounding display area and the movement trajectory may be displayed in a display format according to the movement speed.

[0267] First, an example of the peripheral display area will be described using FIG. 45 . FIG. 45 is a diagram illustrating a display screen 1113a including a peripheral display area in specific example 1 of Example 8. FIG. 45 illustrates a display screen 1113a based on FIG. 44A. Therefore, the display screen 1113a of FIG. 45 displays a peripheral display area 1113s-5 in addition to an active map icon 1113a-1, a dust monster 1113a-2, a vacuum cleaner 1113a-3, and a point display area 1113a-4. This is an area displayed along the outer edge of the display screen 1113a. Depending on the movement speed, either the color or the width of the peripheral display area 1113s-5 is changed. For example, the faster the movement speed, the redder the color. Furthermore, the wider the width, the faster the movement speed.

[0268] Next, an example of a movement trajectory will be described using FIG. 19B of Example 1. In FIG. 19B, the color, length, and width of each line segment of the movement trajectory 1109b-4 are changed according to the movement speed. For example, the faster the movement speed, the redder the color. Furthermore, the longer the line segment, the faster the movement speed. Furthermore, the wider the line segment, the faster the movement speed. Note that in FIG. 19B, the movement trajectory 1109b-4 is displayed as a collection of multiple line segments, but it may also be displayed as a surface. In this case, the color of the movement trajectory 1109b-4 and the length of the line segments are changed according to the movement speed. This change is the same as in the example above. This concludes the description of Example 2.

[0269] <Specific Example 3> Next, specific example 3 can realize display in a display format according to the cleaning volume. To this end, specific example 3 describes an example in which the display format is specified according to the cleaning volume, such as the driving time of the stick vacuum cleaner 2, the moving distance, the cleaning area, the coverage rate of a predetermined area such as a unit cleaning area, etc., as the determination information. In specific example 3, the processing content will also be described using FIG. 42 .

[0270] First, in step S41, the operating status detection unit 104 detects at least one of the following: The time elapsed since the stick vacuum cleaner 2 started cleaning; The positions that have been cleaned by the stick vacuum cleaner 2 (cleaner head 24) for each hour.

[0271] The elapsed time is measured by implementing the operation status detection unit 104 as a timer. The cleaned position can be detected in the same manner as in the first and second specific examples.

[0272] In step S42, the cleaning management unit 107 identifies the cleaning amount, which is the determination information, from the operation status detected by the operation status detection unit 104 as follows.

[0273] The cleaning management unit 107 specifies the elapsed time as the operating time of the stick vacuum cleaner 2. The cleaning management unit 107 also calculates the travel distance of the stick vacuum cleaner 2 by accumulating the amount of change (interval) in the cleaned position for each hour. The cleaning management unit 107 also calculates the cleaning area of ​​the stick vacuum cleaner 2 from the travel distance based on the cleaned position for each hour and the width of the cleaner head 24. Furthermore, the cleaning management unit 107 calculates the coverage rate from the cleaning area and the area of ​​a predetermined area such as a unit cleaning area. Here, the area of ​​a predetermined area such as a unit cleaning area can be specified from the cleaning area map information 114.

[0274] Furthermore, in step S43, the cleaning management unit 107 determines to which of the amount information stored in advance in the storage unit 111 the cleaning amount identified in step S42 corresponds. Here, amount information is information that associates a display format with each cleaning amount (range). Furthermore, as a result, the cleaning management unit 107 identifies a display format that corresponds to the identified cleaning amount. This display format includes a display of the number of icons (points) that have been added. In other words, the number of icons increases as the cleaning amount increases. This can be achieved in the same way as in specific example 1. The cleaning management unit 107 records the number of items (points) that have been added in the points of the user management information 112.

[0275] Then, in step S44, the cleaning management unit 107 causes the output unit 110 to display the surrounding display area and the movement trajectory in the display format determined in step S43. In step S44, for example, the display is as shown in point display area 1109b-3 in FIG. 19B. Furthermore, the size of point display area 1109b-3 itself may be increased according to the number of added items. This concludes the explanation of specific example 3.

[0276] <Specific Example 4> Next, a specific example 4 will be described, in which a recommended area is displayed as a display format similar to that of the second embodiment. In this case, the cleaning record is used to create the recommended area, but the amount of dust collected and the remaining charge (or the charge amount) may also be used. Details of this will be described below with reference to FIG. 43 .

[0277] First, in step S41, the operating status detection unit 104 acquires the recommended area identified by the cleaning management unit 107 when starting cleaning, etc. At this time, it is desirable that the recommended area be identified using the previous cleaning record. Note that starting cleaning includes any of steps S15 to S20 in FIG. 9B. Furthermore, in step S42, as in specific example 1, the cleaning management unit 107 uses the detection and identification of the cleaning position by the operating status detection unit 104 as determination information.

[0278] Then, in step S43, the cleaning management unit 107 creates a display screen including the created recommended area. Then, in step S44, the cleaning management unit 107 displays this on the output unit 110. The display screen is similar to that shown in Fig. 22A or 23A in the second embodiment.

[0279] In addition, in specific example 4, the recommended area can be identified using the dust collection amount and the remaining charge amount. These examples are described below. In either case, the user terminal 1 can acquire the dust collection amount and the remaining charge amount from the stick vacuum cleaner 2 or the charging base 26 using a communication function or the like. First, an example using the dust collection amount will be described. To use the dust collection amount, the stick vacuum cleaner 2 is provided with a dust collection sensor.

[0280] In this example, the cleaning management unit 107 calculates the amount of collected dust for each position using the results of detection by the dust collection sensor.

[0281] The dust collection sensor can detect the weight of the dust collection chamber or the amount of dust sucked in per hour. Even when the former is used, the amount of dust collected per hour can be determined by calculating the amount of change in the weight of the dust collection chamber per hour. The cleaning management unit 107 then compares the date and time in the cleaning management information 113 with the amount of dust collected per hour to calculate the amount of dust collected for each coordinate of the cleaned position in the cleaning management information 113.

[0282] As a result, the cleaning management unit 107 identifies a recommended area based on the dust collection amount for each coordinate. For example, the recommended area is created by connecting coordinates that indicate the highest dust collection amounts within a unit cleaning area. As described above, in this example, the cleaning management unit 107 identifies a recommended area using the cleaning position information in the cleaning management information 113 and the dust collection amount detected by the dust collection sensor. As a result, it is possible to give priority to cleaning areas with a lot of dust.

[0283] When the dust collection amount is used, the display format, such as the color of the recommended area, may be changed depending on the dust collection amount for each position. For example, the position with a larger dust collection amount may be displayed in a more noticeable color such as red, a darker color, or a three-dimensional display or contour line display according to the dust amount.

[0284] Next, an example using the remaining charge will be described. When specifying the recommended area, the operation status detection unit 104 acquires the remaining charge of the stick vacuum cleaner 2 from the stick vacuum cleaner 2. The cleaning management unit 107 then determines whether the recommended area of ​​Example 2 or the recommended area specified based on the dust collection amount described above can be cleaned with the acquired remaining charge. For example, the memory unit 111 stores the cleanable area per unit charge, and the cleaning management unit 107 can make this determination by comparing this with the area of ​​the recommended area.

[0285] If the result of this determination is that cleaning is possible, the recommended area is used as is. On the other hand, if cleaning is not possible, the cleaning management unit 107 narrows the recommended area to create a new recommended area. In this case, the original recommended area may be narrowed evenly, or positions with lower dust collection amounts or cleaning frequencies may be deleted to create a new recommended area.

[0286] In this example, the recommended area in Example 2 and the recommended area determined based on the dust collection amount described above were used, but these may be omitted. That is, the cleaning management unit 107 calculates the cleanable area from the remaining charge, and determines the range of the calculated area from the current position of the stick vacuum cleaner 2 as the recommended area, or determines a unit cleaning area (such as a room) with a relatively high coverage rate within the calculated area as the recommended area. Note that a high coverage rate includes being able to clean the entire unit cleaning area. Furthermore, a priority area (e.g., a floor) can be recorded in the cleaning area map information 114, and the priority area can be determined as the recommended area. Furthermore, the charge amount may be used instead of the remaining charge amount. In this case, the user terminal 1 obtains the charge amount from the stick vacuum cleaner 2 or the charging base 26, and the cleaning management unit 107 estimates the remaining charge amount from the charging time and charge amount.

[0287] Furthermore, in specific example 4, it is desirable to output a response when the recommended area has been cleaned. To this end, cleaning management unit 107 compares the recommended area with the history information (cleaning results in cleaning management information 113) created by history information creation unit 108. As a result, if it is determined that an area equal to or greater than a predetermined standard has been cleaned, cleaning management unit 107 outputs praise words. For example, it outputs words or phrases indicating a successful cleaning, such as "nice cleaning." In this case, cleaning management unit 107 may output text or images via output unit 110, or may output audio via a speaker provided in the user terminal.

[0288] <Specific Example 5> Next, specific example 5, in which health information is displayed as a display format, will be described with reference to FIG. 43 . First, in step S41, the operating status detection unit 104 acquires the history information (cleaning record in the cleaning management information 113) created by the history information creation unit 108 when, for example, cleaning is completed. At this time, it is desirable that the history information be identified using the most recent cleaning record. Note that it is desirable that the completion determination in step S23 of FIG. 9B be used to indicate when cleaning is completed. Furthermore, in step S42, the cleaning management unit 107 determines that the cleaning completion display (step S24) in FIG. 9B has started to be displayed as determination information.

[0289] Then, in step S43, the cleaning management unit 107 creates a display screen including the created recommended area. At this time, the cleaning management unit 107 calculates the distance traveled during the most recent cleaning from the history information. The cleaning management unit 107 then estimates the user's walking distance from the calculated distance traveled. In response to this, the cleaning management unit 107 creates health information including the user's calorie expenditure and / or exercise amount according to the walking distance.

[0290] For this reason, it is desirable to record physical information, including the user's weight, in the user management information 112. The cleaning management unit 107 can then calculate the calories burned and / or the amount of exercise using the walking distance and physical information. As a result, health information related to the user's health and exercise can be created. The cleaning management unit 107 may also be linked to a health management function (health management app) provided in the user terminal 1. For example, the health management function may use a pedometer to calculate the calories burned and / or the amount of exercise during cleaning, or the health management function may use history information to output the calories burned and / or the amount of exercise during cleaning separately from other activities.

[0291] <Specific Example 6> Next, a specific example 6 of displaying the area to be cleaned will be described. This area to be cleaned includes a movement trajectory, a movement history (hereinafter simply referred to as a movement trajectory), and a cleaning area map. Here, various variations regarding the movement trajectory will be described.

[0292] (1) First, an example of managing the movement trajectory for each accessory of the stick vacuum cleaner 2 will be described. For this purpose, the cleaning performance in the cleaning management information 113 is recorded for each accessory. Here, the accessory refers to an accessory that has a suction port, such as a brush nozzle, a tapered nozzle, or the cleaner head 24.

[0293] For this purpose, the stick vacuum cleaner 2 may be provided with an identification device for identifying accessories, or accessories may be identified in response to a user's instruction via the user terminal 1. Furthermore, the accessories may be identified based on image processing of the image captured by the image capturing unit 102. As a result, when the movement trajectory is displayed in Figures 19B, 19D, 20C, 20D, etc., the display will be in accordance with the attached accessories.

[0294] (2) Next, cleaning records in the cleaning management information 113 for identifying a movement trajectory are created in accordance with the movement of the user terminal 1. The cleaning management unit 107 detects the position of the user terminal 1 at each time using a position sensor, acceleration sensor, or the like that the user terminal 1 has, and records this as cleaning records in the cleaning management information 113. As a result, a movement trajectory is created in accordance with the movement of the user terminal 1. Note that at this time, the user terminal 1 does not need to be placed in the terminal holder 20, and may be held in the user's pocket, for example.

[0295] (3) Next, the display of the cleaning area map will be described. In the created map icon 1106g-4 in FIG. 15G, locations that are difficult to clean, such as the back of a television or a table, are accepted, and the cleaning area map creation unit 105 identifies these as areas excluded from cleaning. The cleaning area map creation unit 105 then creates a cleaning area map that distinguishes the areas excluded from cleaning from other areas. As a result, a cleaning area map that includes the areas excluded from cleaning can be displayed in the created map icon 1106g-4. Furthermore, to create such a cleaning area map, the user terminal 1 may detect three-dimensional positions and designate these as areas excluded from cleaning.

[0296] Furthermore, by creating such a cleaning area map, a cleaning area map including the cleaning excluded area can be displayed during cleaning. For example, this can be done in Figures 19A, 19B, 19C, 19D, 20A, etc. Furthermore, when displaying a history such as that in Figures 16A and 16B, the cleaning excluded area can also be included in the display. Note that the cleaning area map may be displayed without including the cleaning excluded area.

[0297] (4) Naturally, the above-mentioned movement trajectory can be superimposed on each display screen of Figures 44A to 45 as history information. Furthermore, this movement trajectory itself may be recorded in the cleaning management information 113. In this case, the cleaning management unit 107 may compare the movement trajectory of the cleaning currently being performed with the movement trajectory that is the history information, and change the display format to display an uncleaned area indicating the difference on the output unit 110 (for example, by changing the color of the uncleaned area). Then, when the stick vacuum cleaner 2 (cleaner head 24) moves through this uncleaned area, the cleaning management unit 107 changes the display format, such as the color of the area that has been moved, to display it.

[0298] As a result, cleaning can be performed by referring to past cleanings. Even more preferably, the cleaning management unit 107 may provide guidance (navigation) for cleaning by providing guidance information that changes the display format of the history information or uncleaned areas as time passes. As a result, it is possible to prevent missing or overlooking cleaning tasks.

[0299] This concludes the description of each specific example of Example 8. Note that in Example 8, common functions that can be applied to the entire example (each specific example) may be provided. The common functions will be described below.

[0300] First, the display of the cleaning area map, i.e., the display format of the active map icon, can be changed. That is, the cleaning management unit 107 controls whether or not the active map icon is displayed, and changes its size. To this end, the cleaning management unit 107 accepts a display instruction from the user and changes the display format of the active map icon in accordance with this display instruction.

[0301] An example of controlling whether or not to display a display format will now be described with reference to FIG. 46 . FIG. 46 is a diagram illustrating an example of controlling whether or not to display an operating map icon 1114a-1 in Example 8. In FIG. 46 , the operating map icon 1114a-1 is displayed on the display screen 1114a, but the cleaning management unit 107 erases the operating map icon 1114a-1 in accordance with an instruction to erase it. As a result, the cleaning management unit 107 displays the display screen 1114 on the output unit 110. Conversely, the cleaning management unit 107 can also display the display screen 1114a on the display screen 1114b by receiving an instruction to display the operating map icon.

[0302] Note that the erase instruction or display instruction may be implemented by tapping the screen. Furthermore, the cleaning management unit 107 enlarges or reduces the display of the active map icon 1114a-1 in response to a zoom-in / zoom-out instruction. This zoom-in / zoom-out instruction can be implemented by a so-called pinch operation. Note that although AR icons are not shown in FIG. 46, AR icons (such as dust monsters) can be displayed on the display screens 1114a and 1114b. Furthermore, such a change in display format can also be implemented in other embodiments. For example, this can be implemented on the display screens shown in FIGS. 19A, 19B, 19C, 19D, and 20A. Furthermore, this can also be implemented on the display screens shown in FIGS. 16A and 16B.

[0303] Also, as a common function, a function of displaying a drive instruction display screen that instructs the stick vacuum cleaner 2 to drive when starting cleaning will be described. More preferably, before displaying the display screens during cleaning in Example 8, such as display screen 1113a in FIG. 44A to display screen 1113a in FIG. 45, the cleaning management unit 107 displays a drive instruction display screen such as that shown in FIG. 47. Furthermore, the drive instruction display screen may be displayed before displaying cleaning-related information (steps S2206, S2210, S2217, S2221). In Example 8, two examples of the drive instruction display screen are shown in FIGS. 47(a) and (b).

[0304] 47(a) and (b) have in common the fact that they display text instructing the user to turn on the vacuum cleaner, i.e., the stick vacuum cleaner 2. In FIG. 47(a), the numbers "3," "2," and "1" count down as time passes in a countdown format. Furthermore, in FIG. 47(a), a game character for the game function is also displayed. Furthermore, in FIG. 47(b), a drive instruction display screen is displayed on a single screen without a countdown. Note that in FIGS. 47(a) and (b), an AR icon such as a dust monster may be displayed on the display screen.

[0305] Furthermore, as a common function, the display screen (cleaning end display screen) for the cleaning end display (step S24) when cleaning is ended will be described. In the eighth embodiment, the display screens shown in Figs. 20A to 20D described in the first embodiment are also displayed. Furthermore, as the cleaning end display, a display screen including a cleaning history as shown in Fig. 48 may be displayed. The display screen 1115a in Fig. 48 includes a cleaning end map icon 1115a-1. The cleaning end map icon 1115a-1 shows the cleaning history of the cleaning that has been performed.

[0306] Furthermore, although display screen 1110c in FIG. 20C is shown as a display screen for indicating the cleaning completion display showing points (number of icons), this is not limiting. A display screen different from display screen 1110c is shown in FIG. 49. FIG. 49 is a diagram showing display screen 1116a for indicating the cleaning completion display including points (number of icons) in Example 8. In FIG. 49, display screen 1116a displays an acquired item display area 1116a-1, a number of acquired items display area 1116a-2, and a game character 1116a-3.

[0307] First, the acquired item display area 1116a-1 displays items (acorns) acquired through cleaning. The acquired item count display area 1116a-2 displays the number of items acquired through cleaning, i.e., the number of items (30) displayed in the acquired item display area 1116a-1. Furthermore, the game character 1116a-3 displays the character used by the user in the game function.

[0308] In addition, in each of the embodiments including the eighth embodiment, the image capturing unit 102 of the user terminal 1 is used. However, the image capturing unit 102 may be configured as a device separate from the user terminal 1, such as a so-called wearable camera (regardless of type, such as a wristwatch type, eyeglass type, or type worn on the body). In this case, the image capturing unit and the user terminal 1 can communicate with each other.

[0309] This concludes the description of the eighth embodiment. According to the eighth embodiment, it is possible to improve motivation for cleaning and to display information in a display format that corresponds to the cleaning status.

[0310] Next, Example 9 shows an example of cleaning support, such as creating a cleaning area map using a two-dimensional code and cleaning management processing. Example 9 can be realized by using the same configuration and information as Example 1. For example, the functional block shown in FIG. 3 can be realized by the hardware shown in FIG. 4. Therefore, the processing of Example 9 can be executed using the configuration of FIG. 3 below. In other words, the processing of Example 9 can be executed in accordance with the cleaning management program 16.

[0311] 3 to 8, any configurations and information not mentioned below may be omitted. For example, the operating status detection unit 104 and the game information 116 may be omitted. Example 9 will be described below, focusing on the differences from the above-described examples.

[0312] Fig. 50A is a flowchart showing the process flow of the map creation process in Example 9. Here, steps up to step S8 are executed in the same manner as in Fig. 9A of Example 1, but these are not essential and other processes may be performed, or these steps may be omitted.

[0313] First, in Fig. 50A, similar to Fig. 9A, the process transitions from step S6 to step S7. Also, in step S7, the photographing unit 102 is activated and begins photographing in response to an instruction from the cleaning area map creation unit 105. Then, in step S8, the cleaning area map creation unit 105 causes the output unit 110 to display a display screen 1106 for map creation including images photographed by the photographing unit 102. At this time, it is assumed that the photographing unit 102 is facing the cleaning target, such as the floor.

[0314] In step S1001, the cleaning area map creation unit 105 reads the recorded content of the two-dimensional code captured by the image capture unit 102 in response to a user's operation. The two-dimensional code is used to identify the position of the stick vacuum cleaner 2 or the user terminal 1. The two-dimensional code may be provided on the charging base 26, more preferably on the base member 29. The two-dimensional code may be provided horizontally or vertically with respect to the floor surface. The two-dimensional code may also be provided diagonally with respect to the floor surface. If the two-dimensional code is provided vertically or diagonally, the cleaning area map creation unit 105 and the cleaning management unit 107 will detect a plane when they operate.

[0315] A two-dimensional code may be provided on the floor surface to be cleaned. The two-dimensional code is an example of the above-mentioned marking section, and a barcode, RFID, AR marker, or the like may also be used.

[0316] The two-dimensional code may store any information as long as it can be used to estimate the relative position. For example, the two-dimensional code may store information indicating the operation mode of the stick vacuum cleaner 2. For example, the cleaning target (room, store, etc.) may be recorded. In this case, the stick vacuum cleaner 2 can be operated in normal mode if in a room, and in demo mode if in a store. Furthermore, the two-dimensional code may store identification information for the stick vacuum cleaner 2 and the two-dimensional code itself.

[0317] In step S1002, the cleaning area map creation unit 105 detects the coordinates and direction of the cleaning target, for example, the floor surface, using the results of reading the two-dimensional code. The cleaning area map creation unit 105 then displays on the output unit 110 that the two-dimensional code has been read and that map creation processing can begin. In response, the cleaning area map creation unit 105 receives a command from the user via the input unit 109 to start map creation.

[0318] In response to this, in step S1003, the cleaning area map creation unit 105 starts plane detection for the cleaning target using the image captured by the image capture unit 102. Here, the plane detection method will be explained. First, as a premise, as described above, the user terminal 1 is pointed in the direction of the floor surface that is the cleaning target. Then, the cleaning area map creation unit 105 acquires feature points from the image captured by the image capture unit 102 and detects a plane by connecting feature points that are approximately the same height.

[0319] Plane detection requires approximately one second to acquire multiple feature points. Therefore, upon receiving a command to start map creation, cleaning area map creation unit 105 performs a countdown and displays it on output unit 110, while plane detection processing is performed in the background. This reduces the user's perceived wait time for plane detection.

[0320] In step S1004, the cleaning area map creation unit 105 determines whether the plane detection in step S1003 has been completed. If the plane detection has been completed (YES), the process proceeds to step S1005. If the plane detection has not been completed (NO), the plane detection in step S1003 continues.

[0321] Furthermore, once a plane is detected, in step S1005, the cleaning area map creation unit 105 displays a target for creating a cleaning area map on the output unit 110. More specifically, a target serving as a reference for creating a cleaning area map, such as a designated position (to be described later), is superimposed on the cleaning target, such as the photographed floor surface. Note that this step can be omitted. Then, as described in FIG. 9A , a cleaning area map is created in step S9. Note that an example of this step may be realized in Example 10 (to be described later). The cleaning area map created above can be identified by relative coordinates with the coordinates of the two-dimensional code. When Example 10 is used, the cleaning area map can be identified by the relative position between the coordinates of the two-dimensional code and the designated position. Note that the cleaning area map created in this manner is stored in the memory unit 111 by the cleaning area map creation unit 105 as cleaning area map information 114.

[0322] Next, the cleaning management process in Example 10 will be described. Figs. 50B and 50C are flowcharts showing the processing flow of the cleaning management process in Example 9. Fig. 50B is a flowchart when cleaning is performed in normal mode, and Fig. 50C is a flowchart when cleaning is performed in simple mode (easy mode). Note that these cleaning management processes can be replaced with step S22 (particularly step S2205 and step S2210) of the cleaning management process in Fig. 9B. The cleaning management process of Example 9 will be described below using each flowchart.

[0323] First, in step S1011 of Fig. 50B, the cleaning management unit 107 photographs the cleaning area using the photographing unit 102. As described in Fig. 50A, at this time, the user terminal 1 faces the cleaning area. Also, in step S1012, the cleaning management unit 107 causes the output unit 110 to display a display screen 1106 for creating a map including the image photographed by the photographing unit 102.

[0324] Also, in step S1013, similar to step S1001, the cleaning management unit 107 reads the recorded content of the two-dimensional code captured by the imaging unit 102 in response to a user's operation. Also, in step S1014, similar to step S1002, the cleaning area map creation unit 105 detects the coordinates and direction of the cleaning target, for example, the floor surface, using the read result of the two-dimensional code.

[0325] Also, in step S1015, similar to step S1003, the cleaning management unit 107 starts plane detection for the cleaning target using the image captured by the image capture unit 102. Then, in step S1016, similar to step S1004, the cleaning management unit 107 determines whether plane detection in step S1015 is complete. As a result, if it is completed (YES), the process proceeds to step S1017. If it is not completed (NO), the plane detection in step S1015 continues.

[0326] Then, in step S1017, cleaning management unit 107 reads out the cleaning area map created by cleaning area map creation unit 105 and displays it on output unit 110. At this time, cleaning management unit 107 may superimpose the cleaning area map on the photographed cleaning area. Alternatively, the cleaning area map may be omitted from display and the content photographed by photographing unit 102 may be displayed. In response to this, the user operates stick vacuum cleaner 2 to start cleaning.

[0327] When cleaning is started (for example, when the cleaning button is turned on), in step S1018, the cleaning management unit 107 draws and displays the movement trajectory of the cleaner head 24 of the stick vacuum cleaner 2 on the output unit 110. It is desirable that this movement trajectory be displayed superimposed on the photographed cleaning area. Specific examples of drawing the activation trajectory will be described after the description of the contents of the simple mode.

[0328] Next, the cleaning management process in the simple mode will be described with reference to Fig. 50C. Here, the normal mode supports cleaning using the created cleaning area map, and steps S1011 and S1012 are the same processes as those in the normal mode of Fig. 50B.

[0329] Furthermore, steps S1013 and S1014 in normal mode are skipped, and in step S1015, the cleaning management unit 107 starts plane detection for the cleaning target using the image captured by the image capture unit 102. Then, in step S1016, the cleaning management unit 107 determines whether plane detection in step S1015 is complete. As a result, if it is completed (YES), the process proceeds to step S1017. If it is not completed (NO), the plane detection in step S1015 continues.

[0330] Then, when cleaning starts, step S1017 is skipped and in step S1018 the cleaning management unit 107 draws and displays the movement trajectory of the cleaner head 24 of the stick vacuum cleaner 2 on the output unit 110. This completes the description of the cleaning management process in simple mode.

[0331] <Trajectory Drawing> Next, a specific example of trajectory drawing (step S1018) in the cleaning management process of FIGS. 50B and 50C will be described. This can also be substituted for step S2205 or step S2210 in FIG. 18 described above. In this example, the movement trajectory of the cleaner head 24 is displayed as an ellipse. In this example, the movement trajectory is drawn only when the cleaner head 24 (suction nozzle) is in contact with a cleaning target such as a floor. In other words, when the cleaner head 24 is lifted from the cleaning target, drawing of the movement trajectory is suppressed. However, either one of these steps may be omitted. Details thereof will be described below.

[0332] First, FIG. 51 illustrates the concept of trajectory drawing in Example 9. In Example 9, the user terminal 1, or in other words, its imaging unit 102, is assumed to be facing the floor surface 510 to be cleaned. For this purpose, the user terminal 1 is assumed to be installed in the terminal holder 20. The cleaning management unit 107 then draws a virtual ray 511 (straight line) from the user terminal 1 (imaging unit 102) to the floor surface 510 and determines the intersection of this ray 511 with the floor surface. The cleaning management unit 107 then uses this result to draw a movement trajectory. Specifically, the cleaning management unit 107 (1) displays an elliptical movement trajectory including this intersection on the output unit 110 of the user terminal 1, and (2) draws the movement trajectory if the length of the ray 511, i.e., the distance from the user terminal 1 to the floor surface 510, is within a certain distance. Regarding (1), it is more preferable that the intersection be near the center of the vacuum cleaner head 24. Regarding (2), in other words, if the distance from the user terminal 1 to the floor surface 510 is equal to or greater than a certain distance, the drawing of the movement trajectory is suppressed.

[0333] For example, a stick vacuum cleaner 2 with a user terminal 1 installed may tilt parallel to the floor surface. In this case, the cleaner head 24 of the stick vacuum cleaner 2 will tend to point upward in response to the tilt. If the cleaner head 24 tilts beyond a certain level, the suction nozzle of the cleaner head 24 will come partially off the floor surface. Furthermore, if the stick vacuum cleaner 2 with a user terminal 1 installed moves above the floor surface by more than a certain amount, the suction nozzle will come off the floor. In other words, the stick vacuum cleaner 2 will be unable to clean or will not be able to clean sufficiently (hereinafter referred to as a situation where cleaning is not possible). In this way, (2) can detect the lifting of the cleaner head 24 (suction nozzle), etc.

[0334] As described above, when the length of Ray 511 (the distance between the intersection and the user terminal 1) exceeds a predetermined distance, the cleaner head 24 (suction nozzle) moves away from the floor surface. Therefore, in Example 9, when it is detected that the length of Ray 511 (the distance between the intersection and the user terminal 1) exceeds a predetermined distance, it is determined that cleaning is not possible and the drawing of the movement trajectory is suppressed. In other words, when the length of Ray 511 (the distance between the intersection and the user terminal 1) is within a predetermined distance, the movement trajectory is drawn in a limited manner.

[0335] Below, (1) drawing of the movement trajectory and (2) details of detecting lifting of the vacuum cleaner head 24 (suction nozzle) will be described. (1) Fig. 52 is a diagram showing a display screen 1106h on which a movement trajectory is drawn in Example 9. In Fig. 52, the display screen 1106h constituting the output unit 110 of the user terminal 1 displays the floor surface 1106h-1, wall surface 1106h-2, and vacuum cleaner 1106h-3 to be cleaned. These are photographed by the photographing unit 102, and the cleaning management unit 107 displays them. The cleaning management unit 107 then draws a movement trajectory 1106h-4 around the vacuum cleaner head of the vacuum cleaner 1106h-3.

[0336] Here, the movement trajectory 1106h-4 is formed around the intersection 512 in FIG. 51, i.e., the cleaner head on the display screen 1106h, and its size is approximately the same as the cleaner head on the display screen 1106h. In this case, the major axis of the ellipse is approximately the same as the width of the cleaner head, and the minor axis is approximately the same as the height of the cleaner head. The shape of the movement trajectory also includes so-called ovals, i.e., egg shapes, oval shapes, rounded rectangles, etc.

[0337] In FIG. 52, the vacuum cleaner 1106h-3 has not yet moved, so a single shape is shown as the movement trajectory 1106h-4, but as the vacuum cleaner 1106h-3 moves, the movement trajectory 1106h-4 is formed in a continuous manner.

[0338] Next, the process for drawing a movement trajectory will be described. First, the cleaning management unit 107 calculates the intersection (intersection position) of the floor surface and the ray 511 extended from the user terminal 1 (image capture unit 102). The cleaning management unit 107 also acquires the yaw angle of the user terminal 1 (image capture unit 102) relative to the floor surface. To do this, the cleaning management unit 107 estimates the direction in which the vacuum cleaner head 24 (suction nozzle) is facing from the orientation of the user terminal 1 and uses this. The cleaning management unit 107 then displays the movement trajectory on the output unit 110 so that it is positioned at the calculated intersection position as an ellipse object with the orientation of the yaw angle of the user terminal 1. In other words, the vacuum cleaner head 24 (suction nozzle) and the ellipse object are displayed so that their longitudinal directions coincide. As a result, a movement trajectory 1106h-4 can be displayed as shown in FIG. 52.

[0339] (2) Next, the process for detecting the lifting of the vacuum cleaner head 24 (suction nozzle) will be described. First, as in (1), the cleaning management unit 107 calculates the intersection position of the floor surface and the ray 511 extended from the user terminal 1 (photographing unit 102). The cleaning management unit 107 also calculates the distance from the intersection position to the user terminal 1 (photographing unit 102). If the calculated distance is within a predetermined distance (reference distance) stored in the memory unit 111, the cleaning management unit 107 draws the movement trajectory 1106h-4. If the calculated distance is longer than the predetermined distance, the cleaning management unit 107 suppresses the drawing of the movement trajectory 1106h-4. Here, the predetermined distance can be a distance corresponding to the length of the extension tube 23 of the stick vacuum cleaner 2. For example, 87 cm can be used. Therefore, if a brush nozzle or the like is used, a different distance can be used as the reference. In this case, the cleaning management unit 107 may select the reference constant distance depending on the image processing of the image captured by the photographing unit 102, the user's specification of the extension tube, etc. Furthermore, the cleaning management unit 107 may calculate the distance from the intersection position at the start of cleaning and use this as a fixed distance (reference).

[0340] This concludes the explanation of Example 9. However, in Example 9, the game function may be used as in Example 1, or may be omitted. Furthermore, the distance being longer than a certain distance also includes a case where the intersection position cannot be calculated. This can be used when the stick vacuum cleaner 2 is tilted, the user terminal 1 is not facing the floor, and the suction nozzle is also facing away from the floor.

[0341] Next, Example 10 shows an example of creating a cleaning area map. Example 10 can also be realized by using the same configuration and information as Example 1. For example, the functional block shown in FIG. 3 can be realized by the hardware shown in FIG. 4. Therefore, the processing of Example 10 can be executed using the configuration of FIG. 3 below. In other words, the processing of Example 10 can also be executed according to the cleaning management program 16. The following description will focus on the differences from the above-mentioned examples.

[0342] In Example 10, the cleaning area map creation unit 105 creates a cleaning area map in the form of a polygon, which is an example of an area formed by lines connecting positions (designated positions) designated by the user via the input unit 109. For example, FIG. 53 shows a state in which a cleaning area map 501-1 indicated by a rectangle is created on the display screen of the user terminal 1 based on the designated positions designated by the user in the order of "1" to "4." In this way, a cleaning area map formed by a polygon with these designated positions as corners is created by connecting the designated positions "1" to "4" with lines, which are an example of lines, in the designated order. Note that the areas in Example 10 include not only polygons but also various closed shapes such as ellipses and crescents.

[0343] On the other hand, if the line segments connecting the specified positions and forming a polygon intersect (self-intersect), a shape like 501-2 in FIG. 53 will result (the black squares in the figure indicate self-intersections). When self-intersections occur like this, the problem arises that it becomes difficult for the user to clean. This is because multiple cleaning areas are created, and the user ends up moving outside the cleaning area. Furthermore, when multiple self-intersections occur, as in 501-3 in FIG. 53, it becomes impossible or difficult to determine which areas are within the cleaning area. For example, in the figure, it is difficult to determine whether 501-3a and 501-3b are within or outside the cleaning area.

[0344] Therefore, in the tenth embodiment, the cleaning area map creation unit 105 determines whether the line segments constituting the polygon self-intersect. This determination includes a final line segment determination as to whether the final line segment candidate connecting the last specified final position candidate and the first specified starting position self-intersects with another line segment. In the case of 501-1 in FIG. 53, it is determined whether the line segment "4"-"1" self-intersects with another line segment.

[0345] In Example 10, the cleaning area map creation unit 105 controls the execution of an end instruction to complete the cleaning area map and a final line segment determination in cooperation with each other. For example, the cleaning area map creation unit 105 executes the final line segment determination on the condition that an end instruction is received. Furthermore, if the final line segment determination determines that the final line segment candidate self-intersects with another line segment, the cleaning area map creation unit 105 disables the acceptance of the end instruction. Specific examples showing these details will be described below. Note that in Example 10, these designated positions are designated on the display screen 1116 of the user terminal 1, and a cleaning area map such as that shown in FIG. 53 is displayed. Note that in Example 10, each designated position and the position of the cleaning area map identified thereby can be specified using the two-dimensional code of Example 9 in terms of relative coordinates with the installation position of the two-dimensional code, but Example 10 may use a coordinate system other than that of Example 9.

[0346] <Specific Example 1> Specific Example 1 is an example in which a final line segment determination is performed on the condition that an end instruction has been received. The processing flow of Specific Example 1 will be described below. Fig. 54 is a flowchart showing the cleaning area map creation processing in Specific Example 1. Here, the processing flow shown in Fig. 54 can be realized as the map creation processing of step S9 in Example 9 or Fig. 9A, but may also be realized as processing independent of the processing in Fig. 9A.

[0347] First, in step S901, the cleaning area map creation unit 105 accepts a designation of a designated position from the user via the input unit 109. At this time, it is assumed that the cleaning area map creation unit 105 is displaying, on the output unit 110, a cleaning target such as a floor in the room photographed by the photographing unit 102, as in step S7. The user then designates a designated position in relation to the output display content. For example, when using the user terminal 1 shown in FIG. 4, the touch panel 12 accepts the designated position by tapping on the touch panel 12.

[0348] Here, the cleaning area map creation unit 105 identifies the coordinates of the floor surface corresponding to the position specified in step S901 in the captured image of the cleaning target displayed on the output unit 110. These coordinates correspond to the coordinates in the cleaning area map information 114 in Fig. 7. Note that these coordinates can be treated as relative coordinates between the installation position of the two-dimensional code and the specified position using the two-dimensional code of Example 9.

[0349] In step S902, the cleaning area map creation unit 105 determines whether the specified position in step S901 is the first specification in creating the cleaning area map. If it is the first specification, the process proceeds to step S903, and if it is the second or subsequent specification, the process proceeds to step S904.

[0350] In step S903, the cleaning area map creation unit 105 sets the coordinates specified in step S901 as the starting point. This starting point indicates the coordinate "1" shown in Fig. 53, that is, the initial position when creating the cleaning area map. Then, the process returns to step S901, and the cleaning area map creation unit 105 accepts the next specified position.

[0351] In step S904, the cleaning area map creation unit 105 creates a newest line segment, which is a line segment connecting the most recently specified position and the specified position immediately before that. In other words, the newest line segment is a line segment connecting consecutive specified positions. Here, this line segment is preferably a straight line, but may also be a curved line.

[0352] In step S905, the cleaning area map creation unit 105 determines whether an end instruction for inputting the designated position has been received from the user via the input unit 109. As a result, if an end instruction has been received (YES), the process proceeds to step S906. If an end instruction has not been received (NO), the process returns to step S901 and the input of the next designated position is received. Note that whether an end instruction has not been received can also be determined by whether the next designated position has been input.

[0353] In step S906, the cleaning area map creation unit 105 creates a final position candidate, which is the most recently accepted specified position, and a final line segment candidate, which is a line segment connecting the starting point. In step S907, the cleaning area map creation unit 105 determines whether either the final line segment candidate or the most recent line segment created in step S904 self-intersects. If the result shows that either line segment self-intersects (YES), the process proceeds to step S908. If none of these line segments self-intersects (NO), the process proceeds to step S909.

[0354] In step S908, the cleaning area map creation unit 105 displays an alert indicating that a self-intersection has occurred on the output unit 110. The content of the alert can be assumed to be something like "An invalid point has been specified."

[0355] Then, in step S909, the cleaning area map creation unit 105 creates a polygon made up of line segments connecting the specified positions as a cleaning area map. As a result, the final line segment candidate created in step S906 becomes the final line segment. At this time, the cleaning area map creation unit 105 stores the created cleaning area map in the memory unit 111 as cleaning area map information 114. At this time, the coordinates of the cleaning area map information 114 can be the coordinates of each specified position.

[0356] This concludes the description of Specific Example 1, but in step S908, the following guidance display may be performed. The cleaning area map creation unit 105 displays the most recently specified position on the self-intersecting line segment on the output unit 110. As a result, the user can confirm the specified position that caused the self-intersection. Furthermore, if the cleaning area map creation unit 105 determines that the final line segment candidate is self-intersecting, it displays guidance information on the output unit 110 indicating a final position candidate where no self-intersection occurs.

[0357] <Specific Example 2> Next, specific example 2 will be described, illustrating another example in which the final line segment determination is performed on the condition that an end instruction has been received. FIG. 55 is a flowchart showing the cleaning area map creation process in specific example 2. Here, the process flow shown in FIG. 55 can be realized as the map creation process of step S9 in Example 9 or FIG. 9A, as in FIG. 54, but it may also be realized as a process independent of the process in FIG. 9A. Furthermore, the same reference numerals are used for the steps in FIG. 55 to indicate processes that are the same as those in FIG. 54. Here, when similar processes are performed using different information, they are treated as similar processes.

[0358] 55, steps S901 to S904 are executed in the same manner as in Specific Example 1. Furthermore, in step S911, the cleaning area map creation unit 105 determines whether the created latest line segment self-intersects with other line segments. If the result shows that the line segment self-intersects with another line segment (YES), the process proceeds to step S908. If the line segment does not self-intersect with another line segment (NO), the process proceeds to step S905.

[0359] In step S905, the cleaning area map creation unit 105 determines whether an end instruction has been received, as in Specific Example 1. Then, as in Specific Example 1, if an end instruction has been received (YES), the process proceeds to step S906, and if an end instruction has not been received (NO), the process returns to step S901 and receives input of the next specified position.

[0360] In step S906, the cleaning area map creation unit 105 creates a final line segment candidate, as in Specific Example 1. In step S912, the cleaning area map creation unit 105 determines whether the final line segment candidate self-intersects with another line segment. As a result, if there is a self-intersection (YES), the process proceeds to step S909. If there is no self-intersection (NO), the process proceeds to step S908. In step S908, the cleaning area map creation unit 105 displays an alert on the output unit 110 indicating that a self-intersection has occurred, as in Specific Example 1. Here, the same content may be displayed when transitioning from step S911 and when transitioning from step S912, or different content may be displayed. In the latter case, the designated position (most recently designated position, final designated position candidate) that caused the self-intersection may be displayed.

[0361] Then, in step S909, the cleaning area map creating unit 105 creates a cleaning area map in the same manner as in Example 1, and this processing flow ends. This concludes the description of Example 2.

[0362] <Specific Example 3> Specific Example 3 is an example in which acceptance of an end instruction is disabled when a final line segment candidate self-intersects with another line segment. FIG. 56 is a flowchart showing the process of creating a cleaning area map in Specific Example 3. Here, the process flow shown in FIG. 56 can be realized as the map creation process of step S9 in Example 9 and FIG. 9A, similar to FIGS. 54 and 55, but it may also be realized as a process independent of the process in FIG. 9A. Furthermore, the same reference numerals are used for the steps in FIG. 56 to indicate processes that are similar to those in FIGS. 54 and 55. Here, when similar processes are performed using different information, they are treated as similar processes.

[0363] 56, steps S901 to S904 are executed in the same manner as in specific examples 1 and 2. Furthermore, in step S911, the cleaning area map creation unit 105 determines whether the created latest line segment self-intersects with other line segments. If the result indicates a self-intersection (YES), the process proceeds to step S908. If the result indicates a non-self-intersection (NO), the process proceeds to step S909.

[0364] In step S911, the cleaning area map creation unit 105 determines whether the latest line segment self-intersects with another line segment, as in Example 2. If the result indicates that the latest line segment self-intersects with another line segment (YES), the process proceeds to step S906. If the result indicates that the latest line segment does not self-intersect with another line segment (NO), the process proceeds to step S909.

[0365] In step S906, the cleaning area map creation unit 105 creates a final line segment candidate, similar to specific examples 1 and 2. In step S912, the cleaning area map creation unit 105 determines whether the final line segment candidate self-intersects with other line segments, similar to specific example 2. As a result, if the final line segment candidate self-intersects with another line segment (YES), the process proceeds to step S913. If the final line segment candidate does not self-intersect with another line segment (NO), the process proceeds to step S905.

[0366] In step S913, the cleaning area map creation unit 105 invalidates the end instruction. Invalidating the end instruction includes refusing the end instruction from the user. As an example of this invalidation, the end instruction button on the touch panel 12, which is an example of the input unit 109, is deactivated so that input from the user via this button cannot be accepted. Then, the process proceeds to step S908.

[0367] Then, in step S908, the cleaning area map creation unit 105 displays an alert on the output unit 110. Here, in step S908 of specific example 3, the display content may be the same as the message "self-intersection has occurred" when transitioning from step S911 and when transitioning from step S913, but it is preferable to change the display content. In the latter case, when transitioning from step S913, a self-intersection has also occurred, so it is preferable to display a message that the end instruction button has been disabled. In this case, the cleaning area map creation unit 105 may further display a method for canceling the disablement on the output unit 110. The method for canceling the disablement can be realized by displaying guidance information indicating final position candidates where no self-intersection occurs, as described in specific example 1.

[0368] Also, in step S905, the cleaning area map creation unit 105 determines whether an end instruction has been received, similar to Specific Examples 1 and 2. Then, similar to Specific Examples 1 and 2, if an end instruction has been received (YES), the process proceeds to step S909, where the cleaning area map creation process is executed. Also, similar to Specific Examples 1 and 2, if an end instruction has not been received (NO), the process returns to step S901, where the input of the next specified position is received. This concludes the description of Example 10.

[0369] Next, in Example 11, a process for correcting distortion or distortion of a cleaning area map (hereinafter simply referred to as distortion) will be described. If a cleaning area map is displayed as is using the granularity (size) of the unit area currently used for various detections in creating the cleaning area map, there is a risk that the shape will be distorted. In other words, misalignment may occur in the horizontal and vertical directions of the screen. For example, FIG. 57 shows distortion of a cleaning area map 1106i-1 in Example 11. In FIG. 57, each side of the cleaning area map 1106i-1 is tilted with respect to each side of the display screen 1106i. In other words, distortion occurs in the cleaning area map 1106i-1.

[0370] Therefore, in Example 11, the cleaning area map creation unit 105 corrects the distortion (tilt) by making the granularity of the unit area coarser, that is, larger, than that of the coordinate system used at the time of detection. Fig. 58 is a diagram showing the downward tilt (distortion) of the top side of the cleaning area map 1106i-1 in Example 11. In Fig. 58, an upper side 583 is formed by connecting end points 581 and 582. These are spaced apart by 0.01 mm in the AR space. 2 Here, the specified positions in the tenth embodiment can be used as the end points 581 and 582, for example.

[0371] For such a cleaning area map 1106i-1, the slope can be made gentler by making the unit areas coarser. For example, as shown in FIG. 59A, the slope can be made gentler by making the endpoints and the lines connecting them into coarser unit areas (areas filled with vertical lines) than the rectangles shown in the figure. Here, the cleaning area map creation unit 105 corrects the upper edge by making the unit areas coarser than the coordinate system used during detection and filling in each rectangle (unit rectangle) through which the upper edge 583 passes.

[0372] The process for this purpose will be described in detail below. First, coordinate system information consisting of a plurality of grids for the floor surface to be cleaned is stored in advance in the storage unit 111. This coordinate system information is shown by arranging unit areas in a matrix, as shown in Figures 58 and 59A.

[0373] The cleaning area map creation unit 105 then identifies a ray from the specified position (a position in a virtual space having a certain height) specified by the user toward the floor. As described in Example 10, the cleaning area map is formed as a polygon by the line segments connecting the specified positions. Therefore, the specified positions indicate the corners (vertices) of the polygonal cleaning area map.

[0374] The cleaning area map creation unit 105 also identifies the intersection of the identified Ray and the floor surface to be cleaned. The cleaning area map creation unit 105 also acquires the intersection coordinates of the identified intersection, and identifies the grid closest to the intersection coordinates from the stored coordinate system information.

[0375] The cleaning area map creation unit 105 also places corner objects in the identified grid. Corner objects are parts that will become the corners (vertices) of the cleaning area map, which is represented by a polygon. The cleaning area map creation unit 105 then places edge objects in the specified order so as to connect the corner objects (the filled-in areas in FIG. 59A). As a result, the cleaning area map creation unit 105 can create a grid map 1106i-1a, which is a modified cleaning area map as shown in FIG. 59B, and display it on the display screen 1106i. Hereinafter, such a modified cleaning area map will be referred to as a "grid map."

[0376] In the eleventh embodiment, a grid map may be used, but instead of or in addition to this, a "dot map" may be created. As shown in FIG. 59B, the grid map 1106i-1a has a slight distortion (tilt). Therefore, the cleaning area map creation unit 105 further corrects the grid map 1106-1i so that each side is horizontal and vertical. If the tilt is greater than a predetermined value, the correction may be performed to create a step. As a result, the cleaning area map creation unit 105 can create a dot map 1106i-1b as shown in FIG. 59C, which is displayed on the display screen 1106i. Comparing the cleaning area map 1106i-1 with the corrected grid map 1106i-1a and dot map 1106i-1b, the distortion (tilt) is reduced or eliminated, and the sides of the cleaning area map are thicker.

[0377] The cleaning area map creation unit 105 may correct distortion of the cleaning area map 1106i-1 as follows. FIG. 60 is another diagram showing the correction of the downward right slope (distortion) of the top side of the cleaning area map 1106i-1 in Example 11. In this example, the cleaning area map creation unit 105 moves the end points 581a and 582a to the center of the unit area in a coordinate system in which the unit area is coarser than the coordinate system used during detection, correcting the top side 583a. In a more desirable mode, the cleaning area map creation unit 105 increases the thickness of the top side 583a. This makes the slope more gentle.

[0378] This concludes the explanation of Example 11, but Example 11 can be applied to map creation in Example 1 as well as map creation in Example 9. Furthermore, the coarse unit area in Example 11 is a square, and one side of the square is preferably equal to or greater than the width of the cleaner head 24, and preferably a value in which a predetermined value is added to the width. For example, if the width of the cleaner head 24 is 25 cm, one side of the unit area can be 30 cm.

[0379] Furthermore, when cleaning (cleaning management processing) is performed with the stick vacuum cleaner 2, the cleaning management unit 107 preferably displays at least one of the cleaning area map 1106i-1, the grid map 1106i-1a, and the dot map 1106i-1b on the output unit 110. The display will be described below.

[0380] FIG. 61A is a diagram showing a display screen 1106j for cleaning-related information in Example 11. In FIG. 61A, cleaning-related information includes a cleaning area map 1106i-1, a grid map 1106i-1a, and a dot map 1106i-1b, along with the image captured by the image capture unit 102, such as a vacuum cleaner 1106j-2 and a movement trajectory 1106j-3. In this figure, the cleaning area map 1106i-1, the grid map 1106i-1a, and the dot map 1106i-1b are displayed from left to right slightly above the center of the display screen 1106j, and each map also displays the cleaned area (circled in the figure) as the cleaning status. This allows the user to intuitively grasp the cleaning status. The movement trajectory 1106j-3 may be elliptical, as shown in Example 9, or may have another shape.

[0381] Here, the display screen that displays the cleaning area map may also display the movement status, such as the speed, of the cleaner head 24. An example of this is shown in Fig. 61B. Fig. 61B is a diagram showing a display screen 1106k for cleaning-related information in Example 11. In Fig. 61B, the suction tip lift, speed, and acceleration are further displayed as cleaning-related information and as the movement status of the cleaner head 24.

[0382] 61B, in addition to the display contents of FIG. 61A, the cleaning management unit 107 displays a floating suction nozzle warning lamp 1106j-4, a speed warning lamp 1106j-5, and an acceleration warning lamp 1106j-6. When the cleaning management unit 107 detects that the cleaner head 24 (suction nozzle) has left the floor surface as described in Example 9, the cleaning management unit 107 changes the color of the display to red or the like, changes the intensity, or turns the lamp on.

[0383] Furthermore, when the speed of the cleaner head 24 (cleaner 1106j-2) exceeds a predetermined speed such as 0.8 m / s, the cleaning management unit 107 changes the color of the display to red or other colors, changes the intensity, or turns on the speed warning lamp 1106j-5. Furthermore, when the acceleration of the cleaner head 24 (cleaner 1106j-2) exceeds 1.4 m / s, the speed warning lamp 1106j-6 2 If the speed exceeds a predetermined value, the cleaning management unit 107 changes the display color to red, changes the intensity, or turns on the corresponding light. This technology can be used. Note that each of these warning lamps (1106j-4 to 1106j-6) may be configured to have only one, or a display indicating whether the warning lamp is for "suction nozzle float," "speed," or "acceleration" may be displayed near each warning lamp. Furthermore, the display screen 1106k may omit the display of at least one of the cleaning area map 1106i-1, grid map 1106i-1a, dot map 1106i-1b, and movement trajectory 1106-k3. Note that these warning lamps may be displayed on the output unit 110 by the cleaning area map creation unit 105 during the map creation process.

[0384] Example 12 illustrates adjustment of the movement trajectory of Examples 1 and 9. This adjustment includes changing the shape and shifting the position. Details will be described below with reference to FIGS. 62A to 62D. FIG. 62A is a diagram showing a display screen 1106l (part 1) for adjusting the movement trajectory in Example 12. On the display screen 1106l in this figure, the cleaning management unit 107 displays an adjustment button 1106l-1, a reset button 1106l-2, a vacuum cleaner 1106l-3, and a movement trajectory 1106l-4a.

[0385] The cleaning management unit 107 shifts the display position of the drawn movement trajectory 1106l-4a up, down, left, or right in response to a user's specification using the adjustment button 1106l-1. This is because the movement trajectory 1106l-4a is displayed superimposed on the image captured by the image capture unit 102, and the display position may shift from the vacuum cleaner 116l-3. Therefore, the display position of the drawn movement trajectory 1106l-4a can be adjusted using the adjustment button 1106l-1. The amount of this shift (adjustment) can be set to 2.5 cm, 5 cm, 10 cm, 15 cm, 20 cm, etc. each time the button is pressed. Alternatively, the shift may be set to a fixed amount.

[0386] Furthermore, when the reset button 1106l-2 is pressed, the cleaning management unit 107 returns the shifted display position of the movement trajectory 1106l-4a to its original position. Note that in Fig. 62A, the shape of the movement trajectory 1106l-4a is displayed as a rectangle. However, the shape of the movement trajectory is not limited to a rectangle, and it can also be displayed as a polygon including a quadrangle or the ellipse described above.

[0387] Fig. 62B is a diagram showing a display screen 1106l (part 2) for adjusting a movement trajectory in Example 12, and shows an example in which a movement trajectory 1106l-4b is displayed in an elliptical shape. The rest is the same as Fig. 62A. As a result, the user can display the movement trajectory in a shape that is easy for the user to understand.

[0388] Furthermore, the display effect of the movement trajectory can be changed. For example, a blurring effect, a glowing effect, or a change in transparency can be applied. Fig. 62C is a diagram showing a display screen 1106l (part 3) for adjusting the movement trajectory in Example 12, and shows an example in which a blurring effect is added to the movement trajectory 1106l-4c.

[0389] Furthermore, in Example 12, the size of the movement trajectory and its drawing start position can also be adjusted. Fig. 62D is a diagram showing a display screen 1106l (part 4) for adjusting the movement trajectory in Example 12, and shows an example of a screen for adjusting the size of the movement trajectory and its drawing start position. In Fig. 62D, a movement trajectory size adjustment button 1106l-5 and a drawing start position adjustment button 1106l-6 have been added to Figs. 62A to 62C.

[0390] When the user presses the movement trajectory size adjustment button 1106l-5, the cleaning management unit 107 changes the size of the movement trajectory. Furthermore, when the drawing start position adjustment button 1106l-6 is pressed, the cleaning management unit 107 moves the drawing start position of the movement trajectory. For example, it moves as indicated by the cross marks in the figure. Note that these cross marks may be displayed on the display screen 1106l, or their display may be omitted. Note that the drawing start position indicates the intersection point with the floor surface in Ray 511 in FIG. 52. Furthermore, although not shown, the color and shade of the movement trajectory may also be adjustable.

[0391] Example 13 is an example of display processing in cleaning management processing using a cleaning area map created during cleaning. Fig. 63 is a diagram showing a display screen 1106m during cleaning management processing in Example 13. In Fig. 63, the cleaning management unit 107 displays a minimap area 1106m-1, an actual map 1106m-2, and an actual trajectory 1106m-3 on the display screen 1106m.

[0392] First, a minimap including a mini-trajectory showing the movement trajectory is displayed in the minimap area 1106m-1. The minimap is a kind of thumbnail image that is a reduced version of the created cleaning area map. However, the mini-trajectory is also drawn in accordance with the cleaning (movement) of the stick vacuum cleaner 2.

[0393] Furthermore, the real map 1106m-2 shows an image in which the boundaries (edges) of the created cleaning area map are superimposed on the floor surface photographed by the photographing unit 102. The real map 1106m-2 shows a grid map or dot map, and each rectangle in the figure indicates a unit area (grid). Furthermore, grids of different colors indicate grids at specified positions or grids at vertices. Furthermore, the real trajectory 1106m-3 shows a movement trajectory superimposed on the floor surface photographed by the photographing unit 102. By using the display screen 1106m as described above, the user can intuitively grasp the cleaning status because the cleaning area map and movement trajectory are virtually displayed in the photographed image.

[0394] Next, a process for displaying such a display screen 1106m will be described. In Example 13, the above display is realized by virtual photography using a virtual camera. The idea is to photograph an object, which is a grid placed on the floor, with a virtual camera virtually placed above. That is, an object placed on the floor is photographed with a virtual camera placed above, and the image is displayed in a minimap area 1106m-1 in the upper right corner of the display screen 1106m.

[0395] Fig. 64A is a diagram for explaining the placement of a virtual camera in Example 13. In Fig. 64A, the cleaning management unit 107 sets the average of the corners 641 (multiple coordinates) of the room specified by the user in the cleaning area map 642 as the central coordinate 643 of the room, that is, the cleaning area map 642. The cleaning management unit 107 then virtually places a virtual camera facing downward above the central coordinate 643. Here, the cleaning management unit 107 sets the height of the virtual camera so that the corner of the room farthest from the central coordinate 643 fits within the angle of view.

[0396] FIG. 64B is a diagram illustrating the height direction arrangement of the virtual camera in Example 13. As described above, the height of the virtual camera 644 needs to capture the entire cleaning area map 642, that is, the farthest corner. Therefore, when the angle of view θ (for example, 60 degrees) is set, the cleaning management unit 107 calculates the height h as h = L / Tan(θ / 2). Here, L indicates the distance from the center coordinate 643 to the corner 641 (the farthest corner). The cleaning management unit 107 displays the results of the image captured by the virtual camera 644 on the display screen 1106m.

[0397] Here, the actual trajectory 1106m-3 shown in FIG. 63 is displayed in detail, and the trajectory on the minimap is displayed as a grid. For this reason, the cleaning management unit 107 displays the actual trajectory object for the content captured by the capture unit 102, and does not display the grid trajectory object on the minimap. In other words, the capture unit 102 acquires the actual trajectory in a limited manner. Furthermore, the cleaning management unit 107 does not display the actual trajectory object for the content captured by the virtual camera 644, but displays the grid trajectory object. In other words, the virtual camera 644 acquires the grid trajectory in a limited manner. As a result, a display such as that shown in FIG. 63 is possible.

[0398] Furthermore, on the display screen 1106m in Example 13, there is a risk that the actual trajectory 1106m-3 may extend beyond the real map 1106m-2. This situation is shown in FIG. 65A. This may occur when the stick-type vacuum cleaner 2, such as the cleaner head 24, extends beyond the area indicated by the created cleaning area map due to user operation, or when the drawing position of the actual trajectory 1106m-3 or the real map 1106m-2 is shifted. In Example 13, the actual trajectory 1106m-3 is adjusted in such a case. This adjustment will be explained below. First, the cleaning management unit 107 determines whether the actual trajectory has extended beyond the cleaning area map. This determination will be explained using FIG. 65B. FIG. 65B schematically illustrates the actual trajectory 645 extending beyond the cleaning area map 642. In such a case, the cleaning management unit 107 counts the number of intersections between line segments 646 from the origin of the coordinate system to each actual trajectory 645 and the sides (boundaries) of the cleaning area map 642. If the number of intersections is an even number, the cleaning management unit 107 determines that the corresponding actual trajectory 645 extends outside the cleaning area map 642.

[0399] If one or more actual trajectories 645 extend beyond the cleaning area map 642, the cleaning management unit 107 performs a trajectory adjustment process on each of the actual trajectories 645. Specifically, the cleaning management unit 107 moves the coordinates of each actual trajectory 645 toward the center coordinate 643 of the cleaning area map 642. For example, as shown in FIG. 65C , the actual trajectories 645 are shifted toward the center coordinate 643 by a predetermined percentage (for example, 1%) in both the x and y directions. The cleaning management unit 107 repeats this process until the actual trajectories 645 are within the cleaning area map 642. In this way, when the actual trajectories 645 extend beyond the cleaning area map 642, the actual trajectories 645 can be moved toward the center of the cleaning area map 642 to prevent the actual trajectory from extending beyond the cleaning area map, thereby preventing the trajectory from being drawn outside the cleaning area map.

[0400] Note that, although each actual trajectory 645 is shifted here, adjustment, i.e., shifting, may be limited to the actual trajectory that protrudes. Furthermore, the cleaning management unit 107 may suppress drawing when the actual trajectory protrudes, or may suppress drawing of the actual trajectory 645 that protrudes in a limited manner. Furthermore, in the above example, the actual trajectory 645 is uniformly shifted by a predetermined percentage, but the actual trajectory 645 may be shifted more the farther it is from the central coordinate 643.

[0401] Furthermore, the adjustment of the actual trajectory (movement trajectory) also includes the following aspects: When the directions of the cleaning area map 642 and the actual trajectory 645 do not match and diverge, the cleaning management unit 107 adjusts the direction of the actual trajectory 645 to match the direction of the cleaning area map 642 according to the content read from the two-dimensional code. The direction of the cleaning area map 642 can be, for example, any side of the cleaning area map 642.

[0402] Furthermore, even if the scale of the cleaning area map 642 and the actual trajectory 645 do not match, the cleaning management unit 107 makes adjustments according to the content read from the two-dimensional code. Furthermore, if there is a discrepancy between the actual trajectory 645 and the real space, that is, the content captured by the imaging unit 102, the cleaning management unit 107 makes adjustments using the processes described in Figures 65A to 65C.

[0403] Furthermore, if the cleaning area map 642 is not drawn correctly on the display screen 1106m, the cleaning management unit 107 displays a message urging the user to move the stick vacuum cleaner 2 by tracing the sides of the cleaning area map 642 (actual map 1106m-2 in FIG. 65A). In response to such movement, the cleaning management unit 107 adjusts the sides of the cleaning area map 642 to straight lines. Furthermore, the cleaning management unit 107 receives a specification from the user regarding a corner of the cleaning area map 642, and adjusts the angle to 90 degrees.

[0404] Furthermore, if the self-position of the stick vacuum cleaner 2 deviates, the cleaning management unit 107 resets the position recognition and corrects the position according to the content read from the two-dimensional code. In this case, the origin of the coordinate system is recorded in the two-dimensional code. The cleaning management unit 107 may also suppress drawing of the actual trajectory 645. Furthermore, the cleaning management unit 107 may correct the deviation using the detection results of a gyro sensor or acceleration sensor provided in the user terminal 1. Furthermore, the cleaning management unit 107 may place items such as those described in Example 8 on the display screen 1106m and perform the cleaning management process without drawing an actual trajectory.

[0405] In Example 13, the cleaning management unit 107 displays the minimap in dots or a grid, and displays the movement trajectory and cleaning area map in more detail than the minimap in the AR space (other than the minimap area 1106m-1 in FIG. 63). As a result, the user can roughly check which areas have been cleaned on the minimap, and can check in more detail which areas have been cleaned or not, that is, the cleaning status, between ARs.

[0406] This concludes the description of each embodiment of the present invention, but the present invention is not limited to these embodiments. In particular, the user terminal 1 may be realized as a wearable device such as so-called AR goggles, smart glasses, or a smart watch. In this case, the wearable device does not need to be attached to the stick vacuum cleaner 2. Furthermore, a separate main body (e.g., a smartphone) may be provided from the wearable device. In this case, the main body may create cleaning-related information and send it to the wearable device, or the wearable device may create it. Furthermore, at least two of the embodiments may be combined. Furthermore, the display screens described in embodiments 1 and 2 may be shared with the relevant parties. Sharing may be performed with the manufacturer's engineers or service personnel. Furthermore, the present invention also includes the following various modifications.

[0407] <Variation 1: Adjustment of Minimap> The cleaning management unit 107 rotates the minimap of the minimap area 1106m-1 displayed on the display screen 1106m in Figures 63 and 65A so that it aligns with each side of the display screen 1106m. In other words, if the horizontal and vertical directions of the minimap and the display screen 1106m do not match, the cleaning management unit 107 rotates the minimap so that they match.

[0408] Furthermore, the minimap may be rotated manually. That is, the cleaning management unit 107 displays a rotation button on the display screen 1106m, accepts operation of the rotation button by the user, and rotates the minimap accordingly.

[0409] <Variation 2: Use of Depth Information> Depth information is included in the cleaning area map information 114, and this information is used to control the display content. First, depth information is information that indicates the height of a position (coordinate) on a plane. Therefore, by using the depth information, various objects with height, such as chairs and walls, can be identified.

[0410] Therefore, the cleaning management unit 107 uses the depth information to make the object transparent if there is an object in front of the movement trajectory, cleaning area map, or other object, and displays the movement trajectory or cleaning area map. As a result, it is possible to reduce the sense of incongruity that occurs when an object behind an object such as a wall is displayed. Note that when the position of the movement trajectory is the same as the floor height, it is desirable for the cleaning management unit 107 to set the position (height) at which the movement trajectory is drawn higher than the original height to prevent the movement trajectory from being hidden or revealed by the floor surface.

[0411] <Variation 3: Recognition of the cleaner head 24> When drawing a movement trajectory, the cleaning management unit 107 identifies its reference position. For example, the coordinates slightly below the center of the user terminal 1 are set as the starting position for drawing the movement trajectory. To this end, the cleaning management unit 107 performs image recognition of the cleaner head 24 and draws the movement trajectory from its center position. This reduces the sense of incongruity regarding the position where the movement trajectory is drawn. As a result, it becomes possible to draw movement trajectories for various vacuum cleaners from different manufacturers.

[0412] <Variation 4: Use of Game Function> In Examples 1 and 2, the use and non-use of the game function are controlled. Similar control can be performed in other examples and variations. Note that operation in the former case can be called game mode, and operation in the latter case can be called normal mode.

[0413] In addition to the above-described modifications, the following display examples are also included in the present invention.

[0414] <Display Example 1: Switching Between Game Mode and Normal Mode> Fig. 66 is a diagram showing a display screen 1106n for switching (selecting) between game mode and normal mode. The display screens 1106n-1 and 1106n-2 in Fig. 66 are displayed by the cleaning management unit 107 in step S2201 of Fig. 18 in the first embodiment, for example.

[0415] Since "No Score" is selected on the display screen 1106n-1, the process transitions to step S2208 in the example of Fig. 18. That is, the stick vacuum cleaner 2 operates in normal mode. Since "With Score" is selected on the display screen 1106n-2, the process transitions to step S2202 in the example of Fig. 18. That is, the stick vacuum cleaner 2 operates in game mode.

[0416] <Display Example 2: Map Creation Process> Next, as Display Example 2, the display screen 1106o during the map creation process will be described. Figures 67A to 67C show the screen transitions of the display screen 1106o during the map creation process. First, when the map creation process is started, the display screen 1106o-1 of Figure 67A is displayed. Here, when the user selects "Start cleaning," the display screen transitions to display screen 1106o-2 or display screen 1106o-3. These correspond to display screen 1106n-1 and display screen 1106n-2 of Figure 66.

[0417] When a mode is selected on display screen 1106o-2 or 1106o-3, the display screen transitions to display screen 1106o-4. This display screen 1106o-4 prompts the user to create a cleaning area map, and then transitions to display screen 1106o-5 in FIG. 67B. This transition may occur after a certain time has elapsed since display screen 1106o-4 was displayed, or in response to a user's instruction, such as a tap.

[0418] Additionally, the display screen 1106o-5 displays an explanation that the two-dimensional code must be read in order to create a cleaning area map. Here, the two-dimensional code is shown attached to the charging stand 26. Then, in response to an operation by the user, the photographing unit 102 is activated and begins photographing, and the display screen 1106o-6 is displayed. The photographed content and a message urging the user to read the two-dimensional code are displayed on the display screen 1106o-6.

[0419] In response to this, when the user points the user terminal 1 (photographing unit 102) at the two-dimensional code, a display like that shown on display screen 1106o-7 appears. Here, the user is prompted to acquire the two-dimensional code. Then, when the two-dimensional code is read by the user's operation, the display screen transitions to display screen 1106o-8. Display screen 1106o-8 displays a prompt to input the corner of the room as the designated position (center of the white circle). When this designated position is input, the display screen transitions to display screen 1106o-9 of FIG. 67C, which displays a "marker" indicating the designated position in Example 9 and prompts the user to input the designated position.

[0420] <Display Example 3: Cleaning After Map Creation Processing> Next, as Display Example 3, a display screen 1106p for cleaning after the map creation processing will be described. FIGS. 68A to 68C show screen transitions of the display screen 1106p for cleaning after the map creation processing. First, when the map creation processing is completed or a cleaning instruction is received from the user after completion, display screen 1106p-1 is displayed. Display screen 1106p-1 is similar to display screen 1106n-2 in FIG. 66 and is a screen for selecting between game mode and normal mode for cleaning. Then, when an instruction to perform cleaning is input on this display screen 1106p-1, the screen transitions to display screen 1106p-2.

[0421] Display screen 1106p-2 is a display screen for prompting the user to install the user terminal 1 in the terminal holder 20. When the "Start" button is selected on this display screen 1106p-2, the display screen transitions to display screen 1106p-3. Note that display screen 1106p-2 may be displayed as display screen 1106p-4, visually displaying the installation of the user terminal 1 so that it is easy to intuitively understand.

[0422] When the user points the user terminal 1 (photographing unit 102) at the two-dimensional code, a display such as that shown on display screen 1106p-3 appears, similar to that on display screen 1106o-7.

[0423] Next, display screens 1106p-5 to 1106p-7 in FIG. 68B display instructions for performing cleaning (cleaning management process). These display screens 1106p-5 to 1106p-7 count down, and the character's appearance changes in accordance with this countdown. When the cleaning management process begins, the display screen transitions to display screen 1106p-8. This display screen 1106p-8 is a display screen for cleaning-related information, equivalent to display screens 1106j and 1109k in Example 11. Furthermore, display screen 1106p-8 shows cleaning in game mode, and displays acquired items in the lower left corner.

[0424] Furthermore, when cleaning is completed, display screen 1106p-9 in FIG. 68C is displayed. This display screen 1106p-9 allows the user to confirm whether or not to end the cleaning management process (cleaning). As a result, when the user selects the "End" button, the display screen transitions to display screen 1106p-10. Display screen 1106p-10 displays that cleaning is completed and how to remove user terminal 1 from terminal holder 20. Then, when the "View Results" button is selected on display screen 1106p-10, the display screen transitions to display screen 1106p-11.

[0425] The display screen 1106p-11 displays information to guide the user to the cleaning details that have been performed. Tapping on this display screen 1106p-11 transitions to display screen 1106p-12. The display screen 1106p-12 then displays the evaluation (Nice) of the cleaning details and the points acquired (number of icons).

[0426] 1...user terminal, 11...camera, 12...touch panel, 13...processing device, 14...communication device, 15...storage device, 16...cleaning management program, 161...vacuum cleaner identification module, 162...operation status detection module, 163...cleaning area map creation module, 164...AR information creation module, 165...cleaning management module, 166...history information creation module, 101...communication unit, 102...photography unit, 103...vacuum cleaner identification unit, 104...operation status detection unit, 105...cleaning area map creation unit, 106...AR information creation unit, 107...cleaning management unit, 108...history information creation unit, 109...input unit, 110...output unit, 111...storage unit, 112...user management information, 113...cleaning management information, 114...cleaning area map information, 115...guidance information, 116...game information, 2...stick Type vacuum cleaner, 20... terminal holder, 20-1... holding part, 20-2... arm part, 20-3... base, 20-4... attachment part, 21... vacuum cleaner main body, 22... handle, 23... extension tube, 24... vacuum cleaner head, 25... handy brush, 26... charging stand, 27... holder member, 28... stand member, 29... base member, 3... cleaning management device, 31... processing device, 32... communication device, 33... memory, 34...Sub-storage device, 301...Communication unit, 302...Cleaner identification unit, 303...Operation status detection unit, 304...Cleaning area map creation unit, 305...AR information creation unit, 306...Cleaning management unit, 307...History information creation unit, 308...Memory unit, 310...User management information, 311...Cleaning management information, 312...Cleaning area map information, 313...Guidance information, 314...Game information, 4...Router, 5...Network

Claims

A user terminal, which is a computer attached to a terminal holder of the vacuum cleaner, an input unit that accepts a plurality of designated positions from a user; a cleaning management program that functions as a cleaning area map creation unit that creates a cleaning area map using line segments connecting the specified positions, The cleaning area map creation unit is a cleaning management program that controls the execution of an end instruction to complete the cleaning area map and a determination of self-intersections between the final line segment identified in response to the end instruction and other line segments in cooperation with each other. 　 The cleaning management program according to claim 1 , the input unit accepts the termination instruction, The cleaning area map creation unit creating a final line segment candidate connecting a final position candidate among the specified positions and a starting point among the plurality of specified positions; determining whether the final line segment candidate and any of the latest line segments created by successive designated positions self-intersect with each other; A cleaning management program that creates a polygon made up of line segments connecting the plurality of designated positions as the cleaning area map when the final line segment candidate and any of the plurality of latest line segments do not self-intersect. 　 The cleaning management program according to claim 1 , The cleaning area map creation unit Create multiple latest line segments by specifying consecutive positions, determining whether each of the plurality of latest line segments self-intersects; the input unit accepts the end instruction when each of the plurality of latest line segments does not self-intersect; The cleaning area map creation unit creating a final line segment candidate connecting a final position candidate among the specified positions and a starting point among the plurality of specified positions; determining whether the final line segment candidate and any of the plurality of latest line segments self-intersect; A cleaning management program that creates a polygon made up of line segments connecting the plurality of designated positions as the cleaning area map when the final line segment candidate and any of the plurality of latest line segments do not self-intersect. 　 The cleaning management program according to claim 1 , The cleaning area map creation unit creating a plurality of latest line segments from consecutive designated positions, and determining whether each of the plurality of latest line segments intersects with itself; a cleaning management program that invalidates the end instruction when each of the plurality of latest line segments self-intersects; 　 The cleaning management program according to claim 1 , The cleaning management program causes the user terminal to further function as a cleaning management unit that displays, on an output unit, a movement locus of a cleaner head of the vacuum cleaner in an elliptical shape. 　 The cleaning management program according to claim 1 , A cleaning management program in which the cleaning area map creation unit creates the cleaning area map using the line segments as unit areas larger than the unit areas used when creating the cleaning area map. 　 The cleaning management program according to claim 6, A cleaning management program in which a unit area larger than the unit area used when creating the cleaning area map is equal to or larger than the width of a cleaner head of the vacuum cleaner. 　 The cleaning management program according to claim 1 , The cleaning management program further causes the user terminal to function as a cleaning management unit that determines that the cleaning head of the vacuum cleaner is floating above the cleaning object when the distance between the user terminal and the intersection of the cleaning object of the vacuum cleaner is equal to or greater than a reference distance. 　 The cleaning management program according to claim 8, The reference distance is determined according to an extension tube of the vacuum cleaner. 　 The cleaning management program according to claim 9, The cleaning management program, wherein the cleaning management unit calculates the distance between the user terminal and the intersection of the object to be cleaned by the vacuum cleaner at the start of cleaning, and uses the calculated distance as the reference distance.

Images