User support device, user support method, and user support program

Abstract

To provide a device which appropriately operates a movable type device in a building to make the movable type device support a user of the building without generating a floor plan of the building.SOLUTION: A user support device 10 includes: a data acquisition part 111 which acquires floor plan data which shows a floor plan of a building and which is previously generated; a movable type device control part 112 which, based on the floor plan data, causes the movable type device to move inside the building and causes the movable type device to collect user data indicating at least one of activities and health conditions of a user in the building; and a user support part 113 which supports the user of the building by using the user data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a user support device, a user support method, and a user support program. [Background technology]

[0002] Currently, the Internet of Things (IoT) is becoming widespread in various fields, and mobile devices such as mobile robots are increasingly being used in homes, offices, and the like. For example, in homes, robot vacuum cleaners are becoming more common due to an increase in dual-income households and elderly households. Mobile devices often require a floor plan of a building in order to move and operate appropriately within the building. For example, a method disclosed in Patent Document 1 is one example of a technology for generating a floor plan of a building.

[0003] The above-mentioned method autonomously captures images with a camera while autonomously moving around the house, identifies the space of the house, generates a map of the house divided into a plurality of areas each having room identification information, has a user select a room identification marker, assigns the room identification marker to the room identification information, and causes the mobile robot to move along a planned route to a position corresponding to the designated destination room identification marker. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-103277 A

[0005] However, the above-mentioned method requires the execution of a process for generating a map, which imposes a certain processing load on the device that executes the process. In addition, there are still technical challenges that have yet to be overcome in the technology for automatically generating a floor plan of a building. For example, although the above-mentioned method can generate a map of a house, it cannot automatically recognize the types of rooms on the map. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above circumstances, and has an objective of providing a user assistance device, a user assistance method, and a user assistance program that can accurately operate a mobile device, such as a mobile robot, within a building and cause the mobile device to assist users of the building without generating a floor plan of the building. [Means for solving the problem]

[0007] One aspect of the present invention includes a data acquisition unit that acquires floor plan data that shows the layout of a building and that has been generated in advance; a mobile device control unit that causes a mobile device to move inside the building based on the floor plan data and causes the mobile device to collect user data that shows at least one of the activities and health conditions of users of the building; a user support unit that uses the user data to support users of the building; and an input / output device to which a camera is connected, wherein the data acquisition unit acquires floor plan data that shows the layouts of multiple floors of the building, Incorporated into the mobile device A still image captured by the camera or a still image constituting a moving image By applying object recognition to the still image and identification data used in the process of identifying the floor of the building, which corresponds the still image to the floor on which the still image was taken, and the mobile device control unit uses the identification data to identify the floor of the building on which the mobile device is located, and causes the mobile device to move within the building based on a floor plan of the floor on which the mobile device is located.

[0008] In the above-mentioned user support device, The identification data is data generated by the mobile device, and the mobile device applies object recognition to the still image to search for structures and equipment specific to one floor of the building, thereby associating the still image depicting the structures and equipment with the floor on which the still image was taken. may be also possible.

[0009] In the above-mentioned user support device, The floor plan data indicates a floor plan of at least one of a plurality of floors of a building, and the mobile device control unit causes the mobile device to move around inside the building based on the floor plan data corresponding to the floor of the building on which the mobile device is located, the floor plan data being identified using the identification data. This is also fine.

[0010] In the above-mentioned user support device, The building is a detached house.This is also fine.

[0012] In the above-described user support device, the mobile device control unit may cause the mobile device to collect the user data in a non-contact manner with users of the building.

[0013] In addition, in the above-mentioned user support device, the mobile device may be equipped with at least one of a camera and a sensor, and the mobile device control unit may cause the mobile device to collect the user data using at least one of the camera and the sensor.

[0014] One aspect of the present invention is a method for implementing a method of the present invention, comprising: The floor plan data shows the layout of a plurality of floors of a building and is generated in advance; The vehicle is mounted on a mobile device that moves inside the building. Still images captured by a camera or still images that make up a video By applying object recognition to the still image and The above This is a user assistance method that executes the following steps: a first step of acquiring identification data used in a process of identifying a floor of the building, which corresponds to the floor on which a still image was taken; a second step of using the identification data to identify the floor of the building on which a mobile device is located based on the floor plan data, and moving the mobile device around the interior of the building based on the floor plan on which the mobile device is located; a third step of causing the mobile device to collect user data indicating at least one of the activity and health condition of users of the building; and a fourth step of using the user data to assist users of the building.

[0015] In the above-described user assistance method, the third step may include causing the mobile device to collect the user data in a non-contact manner with respect to users of the building.

[0016] In the above-mentioned user support method, the mobile device includes at least one of a camera and a sensor, and in the third step, the mobile device The mobile device is provided with The user data may be collected by a camera and / or a sensor.

[0017] One aspect of the present invention is to provide a computer with floor plan data that shows floor plans of a plurality of floors of a building and that is generated in advance, The vehicle is mounted on a mobile device that moves inside the building. Still images captured by a camera or still images that make up a video By applying object recognition to Still images and The above This user assistance program realizes: an acquisition function that acquires identification data used in the process of identifying the floor of the building, which is associated with the floor on which a still image was taken; a mobile device control function that uses the identification data to identify the floor of the building on which a mobile device is located based on the floor plan data, and causes the mobile device to move around the interior of the building based on the floor plan of the floor on which the mobile device is located, and causes the mobile device to collect user data indicating at least one of the activities and health status of users of the building; and a user assistance function that uses the user data to assist users of the building. Effect of the Invention

[0018] According to the present invention, it is possible to provide a user assistance device, a user assistance method, and a user assistance program that can operate a mobile device appropriately within a building and assist users of the building without generating a floor plan of the building. [Brief description of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an example of a hardware configuration of a user support device according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a functional configuration of a user support device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing an example of a floor plan shown by floor plan data according to an embodiment of the present invention. [Figure 4] FIG. 1 is a diagram showing an example of a scene in which a moving object according to an embodiment of the present invention is moving inside a building. [Diagram 5] 5 is a flowchart illustrating an example of a process executed by the user support device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] [Embodiment] A user support device according to an embodiment will be described with reference to Fig. 1 to Fig. 4. Fig. 1 is a diagram showing an example of a hardware configuration of a user support device according to an embodiment of the present invention. As shown in Fig. 1, a user support device 10 includes a processor 11, a main memory device 12, a communication interface 13, an auxiliary memory device 14, an input / output device 15, and a bus 16.

[0021] In the following description, the user support device 10 is mounted on a mobile device used in a building. The building may be, for example, a detached house, an apartment building, or an office building. The mobile device may be, for example, a robot vacuum cleaner, a pet robot, or a drone, and includes any robot that may be used in a building.

[0022] In addition, when the user support device 10 is mounted on a mobile device, it may share at least one of the processor 11, main memory device 12, communication interface 13, auxiliary memory device 14, input / output device 15 and bus 16 with the mobile device.

[0023] The processor 11 is, for example, a CPU (Central Processing Unit), which reads and executes a user assistance program to realize each function of the user assistance device 10. The processor 11 may also read and execute a program other than that of the user assistance device 10 to realize a function required to realize each function of the user assistance device 10. Furthermore, the processor 11 may use data received via the communication interface 13 when executing these functions.

[0024] The main storage device 12 is, for example, a RAM (Random Access Memory), and stores in advance the user assistance program and other programs that are read and executed by the processor 11.

[0025] The communication interface 13 is an interface circuit for communicating with other devices via the network NW shown in Fig. 1. The network NW is, for example, a wide area network (WAN), a local area network (LAN), the Internet, or an intranet.

[0026] The auxiliary storage device 14 is, for example, a hard disk drive (HDD: Hard Disk Drive), a solid state drive (SSD: Solid State Drive), a flash memory, or a read only memory (ROM). The auxiliary storage device 14 stores, for example, programs, user data described below, identification data, and other data.

[0027] The input / output device 15 is, for example, an input / output port. For example, an input device 151, an output device 152, a camera 153, and a sensor 154 are connected to the input / output device 15. The input device 151, the output device 152, the camera 153, and the sensor 154 are incorporated in, for example, a mobile device.

[0028] The input device 151 is, for example, a touch panel display, a mouse, or a keyboard, and is used to operate the user support device 10 and input data to the user support device 10. The output device 152 is, for example, a touch panel display, a speaker, and is used by the user support device 10 to present information to the user.

[0029] The camera 153 is used to generate at least one of user data required for the user support device 10 to support a building user and identification data used in a process of identifying a floor of the building.

[0030] For example, the camera 153 captures video that the mobile device uses to recognize the building user's movements and the characteristics of those movements. The building user's movements here are, for example, walking and standing up from a chair or the floor. Furthermore, the characteristics of the building user's movements here are, for example, walking speed and the amount of movement of the trunk when standing up. The mobile device applies object recognition targeting the user to each of the still images constituting the video captured by the camera 153, and generates user data indicating the user's movements and the characteristics of those movements.

[0031] Alternatively, the camera 153 captures still images or video that the mobile device uses to identify a floor of the building. The mobile device applies object recognition to the still images that compose the still images or video captured by the camera 153 to search for structures and equipment that are specific to one floor of the building. The mobile device then generates identification data that associates a still image depicting such structures and equipment with the floor on which the still image was captured. An example of such a structure or equipment is a terrace of a detached house having two or more floors. If the building is a detached house and an image depicting a terrace is captured by the camera 153, the identification data is data indicating that the floor on which the image was captured is the first floor.

[0032] The sensor 154 is used to generate at least one of user data necessary for the mobile device to support users of the building and identification data used in processing to identify a floor of the building. Both the user data and the identification data are data indicating physical quantities measured by the sensor 154.

[0033] The sensor 154 includes at least one of a sensor that measures a physical quantity related to the environment in the building and a sensor that measures a physical quantity related to the building's users. The sensor that measures a physical quantity related to the environment in the building is, for example, a temperature sensor, a humidity sensor, an illuminance sensor, a power sensor, a human presence sensor, a door sensor, and a window sensor. The sensor that measures a physical quantity related to the building's users is, for example, a non-contact thermometer, a pulse wave sensor, and a skin age measuring device.

[0034] The user data generated using the sensor 154 is, for example, data indicating the user's body temperature, the user's pulse wave, or the user's skin age. The identification data generated using the sensor 154 is, for example, data indicating that a humidity sensor included in the sensor 154 has detected humidity exceeding a predetermined threshold on a particular floor due to the presence of a bathroom, kitchen, or the like.

[0035] 2 is a diagram showing an example of a functional configuration of a user support device according to an embodiment of the present invention. As shown in FIG. 2, the user support device 10 includes a data acquisition unit 111, a mobile device control unit 112, and a user support unit 113.

[0036] The data acquisition unit 111 acquires floor plan data that shows a floor plan of a building and that has been generated in advance. FIG. 3 is a diagram showing an example of a floor plan shown by floor plan data according to an embodiment of the present invention. The floor plan data is, for example, data showing the floor plan shown in FIG. 3. The floor plan data is, for example, data in a vector format showing a floor plan created using computer-aided design (CAD). However, the floor plan data may be data in a data format other than the vector format, such as a raster format.

[0037] The floor plan data preferably further indicates at least one of the types of rooms included in the building and the types of facilities installed in the building. The types of rooms referred to here are, for example, a living room, a kitchen, a bathroom, a hallway, a bedroom, a corridor, and a veranda. The types of facilities referred to here are, for example, a sink, a washroom, a bathtub, and a pantry. Furthermore, the types of facilities referred to here may include home appliances and furniture specific to each room, such as a television, a sofa, a refrigerator, a cupboard, a shower, a shoebox, a bed, a handrail, and a clothes drying rack. This information may be written as metadata when creating the floor plan using, for example, a computer-aided design, or may be separately input using an input device connected to the input / output device 15 after the floor plan is created.

[0038] The floor plan data may also show a floor plan depicting a plurality of sections each assigned to a user of the building. Such floor plan data is particularly useful when the building is an apartment building, an office building, etc., and the mobile device is shared by a plurality of households, offices, etc. The sections referred to here are, for example, each dwelling unit in the case of an apartment building, and each office occupied by each business operator in the case of an office building.

[0039] The floor plan data may also show at least one of the floor plans of a building's multiple floors. For example, if the building is three stories tall, the floor plan data may show at least one of the floor plans of the first floor, the second floor, and the third floor. Such floor plan data is particularly useful, for example, when the building is a detached house having multiple floors, or when the building is an apartment building, an office building, or the like, and the same tenant rents dwelling units, offices, or the like across multiple floors.

[0040] The data acquisition unit 111 may further acquire identification data used in a process of identifying a floor of a building.

[0041] The mobile device control unit 112 moves the mobile device inside the building based on the floor plan of the building shown by the floor plan data. For example, the mobile device control unit 112 identifies an area in which the mobile device can move based on the floor plan of the building shown by the floor plan data, and controls the mobile device to move only within that area. Also, for example, the mobile device control unit 112 identifies an object that may be an obstacle when the mobile device moves based on the floor plan of the building shown by the floor plan data, and controls the mobile device to move while avoiding the periphery of the object.

[0042] Fig. 4 is a diagram showing an example of a scene in which a moving object according to an embodiment of the present invention moves inside a building. For example, as shown in Fig. 4, the mobile device control unit 112 causes the mobile device R to move inside a building B. Also, as shown in Fig. 4, the mobile device control unit 112 may cause the mobile device R to move mainly around the periphery of a user H.

[0043] Furthermore, when the data acquisition unit 111 collects the identification data, the mobile device control unit 112 identifies the floor of the building on which the mobile device is located by using the identification data. Then, the mobile device control unit 112 causes the mobile device to move inside the building based on the floor plan of the floor on which the mobile device is located.

[0044] The mobile device control unit 112 then causes the mobile device to collect user data indicating at least one of the activity and health condition of the building users.

[0045] Furthermore, the mobile device control unit 112 may cause the mobile device to collect user data without contacting the building users. For example, the mobile device control unit 112 may photograph the building users with the camera 153 and cause the mobile device to collect the user data. The user data collected in this manner indicates, for example, at least one of the user's posture and movement. Furthermore, the mobile device control unit 112 may cause the mobile device to collect user data using a non-contact sensor among the sensors 154. The user data collected in this manner indicates at least one of the user's body temperature, pulse, skin age, etc.

[0046] The user support unit 113 supports the building user using the user data. For example, the user support unit 113 notifies the user of an exercise effective for improving physical strength using the output device 152 based on the user data indicating the building user's motion and the characteristics of the motion. Alternatively, the user support unit 113 notifies the user that he or she is suspected of being infected with influenza or the like using the output device 152 based on the user data indicating the body temperature of the building user.

[0047] Next, an example of processing executed by the user support device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of processing executed by the user support device according to the embodiment of the present invention.

[0048] In step S1, the data acquisition unit 111 acquires floor plan data that indicates the layout of a building and that has been generated in advance.

[0049] In step S2, the mobile device control unit 112 causes the mobile device to move around the interior of the building based on the floor plan of the building indicated by the floor plan data, and causes the mobile device to collect user data indicating at least one of the activities and health conditions of the building users.

[0050] In step S3, the user support unit 113 uses the user data to support the user of the building.

[0051] The user support device 10 according to the embodiment has been described above. The user support device 10 includes a data acquisition unit 111, a mobile device control unit 112, and a user support unit 113. The data acquisition unit 111 acquires floor plan data that indicates the layout of a building and that has been generated in advance. The mobile device control unit 112 causes the mobile device to move inside the building based on the floor plan of the building indicated by the floor plan data, and causes the mobile device to collect user data indicating at least one of the activity and health condition of the building user. The user support unit 113 uses the user data to support the building user.

[0052] Therefore, since the user support device 10 acquires a floor plan of the building, it is possible to operate the mobile device appropriately within the building and to have the mobile device support the user of the building without generating a floor plan of the building.

[0053] The user support device 10 also acquires floor plan data indicating at least one of the types of rooms included in the building and the types of facilities installed in the building. This allows the user support device 10 to effectively operate the mobile device in accordance with a specific policy of support to be provided to the user, taking into account the types of rooms and facilities included in the building.

[0054] The user support device 10 also acquires floor plan data showing a floor plan depicting a plurality of sections assigned to each user of the building. This allows the user support device 10 to move the mobile device among the plurality of sections and provide support to more users of the building.

[0055] The user support device 10 also acquires floor plan data showing the layout of at least one of the multiple floors of the building. This allows the user support device 10 to move the mobile device across the multiple floors of the building, and provide support to more building users over a wider area.

[0056] The user support device 10 also acquires identification data used in a process of identifying a floor of a building, and identifies the floor of the building on which the mobile device is located using the identification data. The user support device 10 then moves the mobile device inside the building based on the floor plan of the floor on which the mobile device is located. This allows the user support device 10 to accurately move the mobile device based on the characteristics of the floor on which the mobile device is located.

[0057] In the above embodiment, the user support device 10 is mounted on a mobile device used in a building, but the present invention is not limited to this. For example, the user support device 10 may be a device independent of the mobile device used in the building. For example, the user support device 10 may be a server, a stationary device, or a portable device that can communicate with the mobile device via the network NW shown in FIG. 1.

[0058] The embodiment of the present invention has been described above with reference to the drawings. However, the user support device 10 is not limited to the above embodiment, and various modifications, substitutions, combinations, and / or design changes can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0059] 10...user support device, 11...processor, 12...main memory device, 13...communication interface, 14...auxiliary memory device, 15...input / output device, 111...data acquisition unit, 112...mobile device control unit, 113...user support unit, 151...input device, 152...output device, 153...camera, 154...sensor, NW...network

Claims

1. A data acquisition unit that acquires floor plan data that shows a floor plan of a building and that has been generated in advance; a mobile device control unit that causes a mobile device to move inside the building based on the floor plan data and causes the mobile device to collect user data indicating at least one of an activity and a health condition of a user of the building; a user support unit that uses the user data to support users of the building; an input / output device to which the camera is connected; The data acquisition unit is The floor plan data showing floor plans of a plurality of floors of the building; acquiring identification data used in a process of identifying a floor of the building, the identification data being obtained by applying object recognition to a still image captured by the camera incorporated in the mobile device or a still image constituting a moving image, and associating the still image with the floor on which the still image was captured; The mobile device control unit includes: Using the identification data, a floor of the building on which the mobile device is located is identified, and the mobile device is moved inside the building based on a floor plan of the floor on which the mobile device is located. User support equipment.

2. The identification data is data generated by the mobile device, and the mobile device applies object recognition to the still image to search for structures and equipment specific to one floor of the building, thereby associating the still image depicting the structures and equipment with the floor on which the still image was taken. The user support device according to claim 1 .

3. The floor plan data indicates a floor plan of at least one of a plurality of floors of a building, The mobile device control unit causes the mobile device to move inside the building based on the floor plan data corresponding to the floor of the building on which the mobile device is located, the floor plan data being identified using the identification data. The user support device according to claim 2.

4. The user support device according to claim 1 , wherein the building is a detached house.

5. the data acquisition unit acquires the floor plan data indicating at least one of types of rooms included in the building and types of facilities installed in the building; 5. The user support device according to claim 1.

6. The data acquisition unit acquires the floor plan data showing a floor plan in which a plurality of sections assigned to each user of the building are depicted. The user support device according to any one of claims 1 to 5.

7. the mobile device control unit causes the mobile device to collect the user data in a non-contact manner with respect to users of the building; The user support device according to any one of claims 1 to 6.

8. the mobile device includes at least one of a camera and a sensor; The mobile device control unit causes the mobile device to collect the user data using at least one of the camera and the sensor. The user support device according to any one of claims 1 to 7.

9. The computer a first step of acquiring floor plan data that shows floor plans of a plurality of floors of a building and that is generated in advance, and identification data used in a process of identifying a floor of the building, the floor plan data being obtained by applying object recognition to still images or still images that constitute a moving image taken by a camera incorporated in a mobile device that moves inside the building, thereby associating the still images with the floors on which the still images were taken; a second step of identifying a floor of the building on which the mobile device is located using the identification data based on the floor plan data, and moving the mobile device around inside the building based on the floor plan data on which the mobile device is located; a third step of causing the mobile device to collect occupant data indicative of at least one of an activity and a health status of occupants of the building; a fourth step of using the occupant data to assist occupants of the building; A method for assisting users in carrying out the above.

10. In the third step, the user data is collected by the mobile device in a non-contact manner from users of the building. The user support method according to claim 9.

11. the mobile device includes at least one of a camera and a sensor; In the third step, the mobile device collects the user data using at least one of a camera and a sensor provided in the mobile device. The user support method according to claim 9 or 10.

12. On the computer, an acquisition function for acquiring floor plan data that shows floor plans of a plurality of floors of a building and that is generated in advance, and for acquiring identification data used in a process for identifying a floor of the building, the identification data being obtained by applying object recognition to still images or still images that constitute a moving image taken by a camera incorporated in a mobile device that moves inside the building, thereby associating the still images with the floors on which the still images were taken; a mobile device control function for identifying the floor of the building on which the mobile device is located using the identification data based on the floor plan data, and for causing the mobile device to move around the building based on the floor plan of the floor on which the mobile device is located, and for causing the mobile device to collect user data indicating at least one of the activities and health conditions of users of the building; a user assistance function that uses the user data to assist users of the building; A user support program that makes this possible.

Images