System, method, and program

Abstract

To further improve accuracy of a determination as to whether or not a user's hand gets closer to a user's face.SOLUTION: A system comprises: a control device including an image pickup section for acquiring face image data of a user and a control section for analyzing the face image data; a portable terminal including a first detection section mounted in the vicinity of a wrist of the user; and a second detection section mounted in the vicinity of a head of the user. The first detection section acquires distance data with the second detection section. In a case where it is determined that a face and a hand overlap by analyzing the face image data, the control section acquires the distance data and in a case where the distance data are shorter than a predetermined distance which is set beforehand, outputs an alarm to the portable terminal.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] One embodiment of the present invention relates to a system, a control device, a mobile terminal, a method, and a program for preventing infectious diseases.

[0002] In recent years, not only influenza viruses but also novel coronavirus have been raging around the world. Such viruses are known to mainly enter through the mucous membranes of the human face. Conventionally, it has been recommended to prevent the virus from entering the human body by washing hands or gargling, and covering the nose and mouth with a mask.

[0003] However, even if hands are washed frequently, if the face is unconsciously touched with hands that have touched a surface where the virus is attached, there is a risk of infection with the virus. Patent Document 1 discloses a hand recognition device that determines whether there is a hand around a recognized face area by image recognition.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When determining whether a user's hand is approaching the user's face by image recognition, the distance between the face and the hand cannot be determined, and there is a risk of false detection. In this case, even when the distance between the face and the hand is far, it may be determined that there is a hand around the face area, so the accuracy of the determination may decrease.

[0006] In view of the above problems, one of the objectives of one embodiment of the present invention is to further improve the accuracy of determining whether a user's hand is approaching the user's face.

Means for Solving the Problems

[0007] A system according to an embodiment of the present invention includes a control device having an image capturing unit that acquires face image data of a user and a control unit that analyzes the face image data, a mobile terminal having a first detection unit mounted near the user's wrist, and a second detection unit mounted near the user's head. The first detection unit acquires distance data from the second detection unit. When the control unit analyzes the face image data and determines that the face and the hand overlap, the control unit acquires the distance data. When the distance data is smaller than a preset predetermined distance, an alarm is output to the mobile terminal.

[0008] In the above configuration, when the control unit determines that the face and the hand do not overlap, the control unit analyzes the newly acquired face image data.

[0009] In the above configuration, when the acquired distance between the first detection unit and the second detection unit is equal to or greater than a preset predetermined distance, the newly acquired face image data is analyzed.

[0010] In the above configuration, the predetermined distance is the distance between the first detection unit and the second detection unit when the user brings the hand close to the user's eyes.

[0011] A method according to an embodiment of the present invention acquires face image data of a user, analyzes the face image data to determine whether the face and the hand overlap, and when it is determined that the face and the hand overlap, the first detection unit of the mobile terminal mounted near the user's wrist acquires distance data from the second detection unit mounted near the user's head, and outputs an alarm when the acquired distance between the first detection unit and the second detection unit is smaller than a preset predetermined distance.

[0012] In the above method, when the acquired distance between the first detection unit and the second detection unit is equal to or greater than a preset predetermined distance, new face image data is acquired.

[0013] In the above method, when the distance between the newly acquired first detection unit and the second detection unit is equal to or greater than a preset predetermined distance, the newly acquired face image data is analyzed.

[0014] In the above method, the predetermined distance is the distance between the first detection unit and the second detection unit when the user brings the hand close to the eyes.

[0015] A program according to an embodiment of the present invention causes a computer to execute: acquiring face image data, analyzing the face image data, determining whether a part of the face overlaps with a hand based on the analysis result, and when it is determined that a part of the face overlaps with the hand, acquiring the distance between a first detection unit of a mobile terminal worn on the user's wrist and a second detection unit provided near the user's head, and outputting an alarm to the mobile terminal when the acquired distance between the first detection unit and the second detection unit is smaller than a predetermined distance.

Advantages of the Invention

[0016] According to an embodiment of the present invention, it is possible to further improve the accuracy of determining whether the user's hand is approaching the user's face.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0018] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following embodiments, and various modifications can be made within the scope of the gist thereof.

[0019] (1. Overview) A system 1 according to one embodiment of the present invention outputs an alarm so that a user does not unconsciously touch their face when working at a determined position, such as at a supermarket cash register or in factory line work.

[0020] FIG. 1 is a diagram for explaining the overview of a system 1 according to one embodiment of the present invention. The system 1 includes a control device 10, a mobile terminal 20, and a detection unit 30. The control device 10 is connected to the mobile terminal 20 via a wireless or wired network 40. Also, the mobile terminal 20 may be connected to the detection unit 30 by short-range wireless communication or may be connected by the network 40. Short-range wireless communication is, for example, communication using Bluetooth (registered trademark), infrared communication, LAN (Local Area Network), or the like.

[0021] The control device 10 outputs an alarm to the mobile terminal 20 in order to prevent the user 1000 from touching their face. The control device 10 is, for example, a personal computer, a smartphone, or a tablet. The image capturing unit 103 is provided at a position where the face of the user 1000 can be constantly acquired. The image capturing unit 103 acquires the face image data of the user 1000 at predetermined intervals. The control device 10 analyzes the acquired face image data of the user 1000 and determines whether the face of the user 1000, particularly the mucous membrane parts such as the eyes, nose, and mouth, overlaps with the hand of the user 1000. As shown in FIG. 1, the image capturing unit 103 may be a device separate from the control device 10 or may be included in the control device 10.

[0022] In the face image data of the user 1000 acquired by the image capturing unit 103, even if the face and hand of the user 1000 overlap, the distance between the face and hand of the user 1000 cannot be determined. Therefore, it is necessary to detect the distance between the face and hand of the user 1000.

[0023] The user 1000 wears the portable terminal 20 near the hand and wears the detection unit 30 near the head. The portable terminal 20 is provided with a detection unit 207. The portable terminal 20 is, for example, a wearable device such as a wristwatch type or a ring type. The detection unit 30 is installed in the article 31. The article in which the detection unit 30 is installed may be any article that can be worn near the head, such as a helmet, a piercing, a necklace, glasses, a hair accessory, a mask, or a neck-mounted wearable device. When the hand of the user 1000 approaches near the face of the user 1000, the detection unit 207 of the portable terminal 20 detects the distance to the detection unit 30.

[0024] In FIG. 1, it is assumed that the user 1000 wears the portable terminal 20 on the wrist and wears the necklace 31 with the detection unit 30 installed around the neck. At this time, the detection unit 207 of the portable terminal 20 worn on the wrist acquires distance data to the detection unit 30 installed in the necklace 31 every predetermined period and transmits it to the control device 10. When the control device 10 determines that the distance between the portable terminal 20 and the detection unit 30 is smaller than a preset predetermined distance, the control device 10 outputs an alarm. Here, the predetermined distance is the distance for determining that the face and hand of the user 1000 have approached.

[0025] According to system 1, an alarm can be output when user 1000 tries to touch their face unconsciously. It determines whether the face and hand of user 1000 overlap by image recognition and also determines based on the distance between the mobile terminal 20 and the detection unit 30. Therefore, the accuracy of determining whether the hand of user 1000 is approaching the face of user 1000 can be further improved. Thereby, it is possible to suppress the output of unnecessary alarms. Since it is possible to suppress the user from unconsciously touching their face, it is possible to promote the prevention of infectious diseases.

[0026] (2. First Embodiment) In this embodiment, in the control device 10, a case will be described in which it is determined whether the user's hand is approaching the face and an alarm is output so that the user does not touch their face.

[0027] FIG. 2 is a block diagram of the hardware in one embodiment of the present invention. As shown in FIG. 2, the control device 10 includes a control unit 101, a storage unit 102, an image capturing unit 103, a communication unit 104, an operation unit 106, and a display unit 105.

[0028] The control unit 101 includes an arithmetic processing circuit such as a CPU. The control unit 101 executes the program stored in the storage unit 102 by the CPU to realize each function in the control device 10. The functions realized in the control device 10 include, for example, a determination function for executing determination processing. The storage unit 102 is a storage device such as a non-volatile memory. The storage unit 102 stores a program 300 for realizing the above determination function. The determination function will be described later. Also, the storage unit 102 stores face image data and distance data used in the determination processing. Here, the face image data is the face image of the user, and the distance data is the distance between the mobile terminal 20 and the detection unit 30 acquired from the mobile terminal 20. Further, the storage unit 102 may store a database in which distance data (also referred to as a predetermined distance) between the mobile terminal 20 and the detection unit 30 set in advance by the user is stored. The predetermined distance is a distance for determining that the user's hand has approached the user's eyes, nose, or mouth. Since the predetermined distance is different for each user wearing the mobile terminal 20, the predetermined distance is set for each mobile terminal 20. Also, the distance data stored in the database differs depending on the part where the user wears the mobile terminal 20 and the part where the detection unit 30 is worn. For example, the distance data when the mobile terminal 20 (device 0001) is worn on the wrist and the detection unit 30 is worn on the neck may be stored, or the distance data when the mobile terminal 20 (device 0003) is worn on the finger and the detection unit 30 is worn on the helmet may be stored.

[0029] The image capturing unit 103 is a photographing device such as a CCD camera, for example. The image capturing unit 103 captures a face image of the user. The face image data of the face image of the user captured by the image capturing unit 103 is stored in the storage unit 102. The image capturing unit 103 captures the face image of the user every predetermined period (for example, every few milliseconds to 1 second).

[0030] Based on the control of the control unit 101, the communication unit 104 connects to the mobile terminal 20 via wireless communication. The communication unit 104 transmits and receives information to and from the mobile terminal 20, for example, by communication using Bluetooth (registered trademark), infrared communication, LAN (Local Area Network), etc. The communication unit 104 may, for example, acquire distance data from the mobile terminal 20 or transmit an alarm to the mobile terminal 20.

[0031] The operation unit 106 is a device such as a keyboard, a mouse, and a touch sensor, and outputs a signal corresponding to the input operation to the control unit 101. The display unit 105 is a display device such as a liquid crystal display and an organic EL display, and a screen based on the control by the control unit 101 and an image captured by the image capturing unit 103 are displayed. Note that the operation unit 106 and the display unit 105 may form a touch panel integrally.

[0032] Here, an example of the database stored in the storage unit 102 will be described with reference to FIG. 3. FIG. 3 is an example of the database 313. The device ID stored in the database 313 is an ID for identifying the mobile terminal 20. For each device ID, the distance [mm] set by the user is associated. FIG. 3 shows an example in which the distances when the hand approaches the eye, the nose, and the mouth are set respectively. The distance when the hand approaches the eye means the distance between the mobile terminal 20 (detection unit 207) and the detection unit 30 when the hand approaches the eye. Similarly, the distance when the hand approaches the nose is the distance between the mobile terminal 20 (detection unit 207) and the detection unit 30 when the hand approaches the nose, and the distance when the hand approaches the mouth is the distance between the mobile terminal 20 (detection unit 207) and the detection unit 30 when the hand approaches the mouth. The setting of the predetermined distance may be such that the detection unit 207 acquires the distance data in each case of the operations and stores it in the database 313 when the user touches the eye, the nose, and the mouth with the hand. Also, the user may input a predetermined distance from the operation unit 106. In FIG. 3, the distances when the hand approaches the eye, the nose, and the mouth are set, but for the determination process, the largest numerical value among these distances may be used. For example, when the device ID of the mobile terminal 20 is 0001, the distance to the eye is set to 10 cm. In the determination process, the control device 10 may output an alarm when it determines that the distance between the mobile terminal 20 and the detection unit 30 is less than 10 cm. Also, although not shown in FIG. 3, the number of times an alarm has been output may be recorded for each device ID. By displaying the number of times an alarm has been output on the display unit 105 of the control device 10 or the display unit of the mobile terminal 20, the user can be prompted to be careful not to touch the face.

[0033] In addition, in FIG. 2, the control device 10 is illustrated as including a control unit 101, a storage unit 102, an image capturing unit 103, a communication unit 104, an operation unit 106, and a display unit 105. However, an embodiment of the present invention is not limited thereto. For example, the image capturing unit 103 may be an image capturing device (e.g., a camera) provided separately from the control device 10, and the image capturing device may be configured to be connected to the control device 10. Further, the program stored in the storage unit 102 or the database 313 storing predetermined distances may be stored in a storage device separate from the storage unit 102, and the separate storage device may be configured to be connected to the control device 10.

[0034] The mobile terminal 20 includes a control unit 201, a storage unit 202, a communication unit 204, a detection unit 207, and an alarm output unit 208.

[0035] The control unit 201 includes an arithmetic processing circuit such as a CPU. The control unit 201 executes the program stored in the storage unit 202 by the CPU to cause each function to be realized in the control device 10. The storage unit 202 is a storage device such as a non-volatile memory. Further, the storage unit 202 stores the distance data acquired by the detection unit 207.

[0036] The detection unit 207 is a sensor capable of measuring a distance, such as a distance sensor. As the distance sensor, for example, an infrared distance sensor, an optical sensor, an ultrasonic sensor, etc. can be used. The detection unit 207 may include an acceleration sensor and a gyro sensor, etc. The detection unit 207 forms a pair with the detection unit 30, and measures the distance between the detection unit 207 and the detection unit 30. The detection unit 207 acquires the distance data between the detection unit 207 and the detection unit 30 at every predetermined period (e.g., every few milliseconds to 1 second). The distance data is stored in the storage unit 202. The distance data may be transmitted to the control device 10 and stored in the storage unit 102 of the control device 10.

[0037] Based on the control of the control unit 201, the communication unit 204 connects to the control device 10 via wireless communication. The communication unit 204 transmits and receives information with the control device 10 through communication using, for example, Bluetooth (registered trademark), infrared communication, LAN (Local Area Network), etc. The communication unit 204, for example, transmits distance data to the control device 10 or receives an alarm from the control device 10. The timing for the communication unit 204 to transmit distance data to the control device 10 is, for example, every few milliseconds to 1 second.

[0038] The alarm output unit 208 includes, for example, at least one of a sound output unit 281, a light emitting unit 282, and a vibration unit 283. The alarm output unit 208 outputs the alarm received from the control device 10 by at least one of the sound output unit 281, the light emitting unit 282, and the vibration unit 283. Here, the alarm may be, for example, outputting a voice or a warning sound by the sound output unit 281, or vibrating by the vibration unit 283. Also, the alarm may be lighting the LED of the light emitting unit 282 or flashing it. Although not shown in the figure, when the mobile terminal 20 has a display unit, a warning message may be displayed on the display unit as an alarm. When the alarm output unit 208 receives a signal for outputting an alarm, it outputs the alarm by any one of the sound output unit 281, the light emitting unit 282, and the vibration unit 283, or in combination thereof. The period for outputting the alarm may be a predetermined period (for example, a few milliseconds to 1 second), or a signal for stopping the alarm may be transmitted from the operation unit 107 to the control unit 101 by an operation by the user. Also, the period for outputting the alarm may be until the control unit 101 acquires the next face image data.

[0039] Although not shown in FIG. 2, the mobile terminal 20 may have a display unit and an operation unit, similar to the control device 10.

[0040] FIG. 4 is a block diagram showing a determination function in an embodiment of the present invention. Note that part or all of the configuration for realizing each function described below may be realized by hardware.

[0041] The determination function (program 300) for outputting an alarm includes an acquisition unit 301, an image analysis unit 302, a determination unit 303, and an output unit 304.

[0042] The acquisition unit 301 acquires the face image data 311 acquired by the image capturing unit 103 and the distance data 312 acquired by the detection unit 207 of the mobile terminal 20. The timing at which the acquisition unit 301 acquires the face image data 311 and the distance data 312 is every predetermined period (for example, every few milliseconds to 1 second).

[0043] The image analysis unit 302 analyzes the face image data 311. FIG. 5 is a functional block diagram of the image analysis unit 302. The image analysis unit 302 has teacher data 322, a learning unit 323, a machine learning model 324, and an estimation unit 325.

[0044] The image analysis unit 302 has a function of analyzing the face image data of the user acquired by the acquisition unit 301. The teacher data 322 stores two types of images: an image in which the face and the hand overlap, and an image in which the face and the hand do not overlap. The image in which the face and the hand overlap includes, for example, an image in which the eyes and the hand overlap, an image in which the nose and the hand overlap, and an image in which the mouth and the hand overlap. Also, the image in which the face and the hand overlap may be an image in which the cheek and the hand overlap, or an image in which the forehead and the hand overlap, or an image in which the jaw and the hand overlap. Also, it is not necessary for all of the eyes to overlap with the hand, and an image in which a part of the eyes and a part of the hand (finger) overlap may be sufficient. Also, an image in which the eyes and the hand overlap while the user is facing sideways may be sufficient. Also, the image in which the face and the hand do not overlap may be an image of only the face, or an image in which there is a hand near the face, or an image in which there is a hand at a location far from the face. The learning unit 323 constructs the machine learning model 324 by performing machine learning in advance using two types of images, an image in which the face and the hand overlap and an image in which the face and the hand do not overlap, as the teacher data 322. The constructed machine learning model 324 is stored in the storage unit 202.

[0045] The face image data of the user acquired by the image capturing unit 103 is transmitted to the acquisition unit 301. The face image data acquired by the acquisition unit 301 is estimated by the estimation unit 325 using the machine learning model 324 as to whether it is face image data in which the face and the hand overlap or face image data in which the face and the hand do not overlap.

[0046] The determination unit 303 determines whether or not the user's face and the user's hand overlap based on the result of the analysis by the image analysis unit 302. Specifically, the determination unit 303 determines whether or not the user's eyes, nose, and mouth overlap with the user's hand. When the user is wearing a mask, the determination unit 303 determines whether at least the user's eyes overlap with the user's hand. Also, the determination unit 303 determines whether or not the distance data 312 is smaller than a preset predetermined distance.

[0047] Also, the acquisition unit 301 acquires the distance data 312. When the determination unit 303 acquires the distance data 312, it refers to a database in which a predetermined distance is stored and determines whether the acquired distance data 312 is smaller than the predetermined distance.

[0048] When the determination unit 303 determines based on the analysis result of the face image data that the user's face and the user's hand overlap and also determines that the distance data is smaller than the predetermined distance, the output unit 304 outputs an alarm. The control unit 101 transmits the alarm output from the output unit 304 to the mobile terminal 20. When the mobile terminal 20 receives the alarm from the control device 10, it outputs the alarm from the alarm output unit 208.

[0049] Next, the determination process according to an embodiment of the present invention will be described with reference to FIG. 6. The determination process outputs an alarm to the mobile terminal 20 based on the face image data of the user's face acquired by the image capturing unit 103 and the distance data acquired from the mobile terminal 20.

[0050] FIG. 6 is a flowchart for explaining the determination process in an embodiment of the present invention.

[0051] For example, in the mobile terminal 20, when a signal instructing the start of the determination process is output, the control unit 101 of the control device 10 starts a process of determining whether the user's hand is approaching the face. First, the control unit 101 acquires the face image data of the user captured by the image capturing unit 103 (step S401). The timing at which the control unit 101 acquires the face image data of the user captured by the image capturing unit 103 is every few milliseconds to 1 second. Next, the control unit 101 analyzes the face image data (step S402). Next, the control unit 101 determines whether the user's face and the user's hand (or finger) overlap based on the analyzed face image data (step S403). When the control unit 101 determines that the user's face and the user's hand do not overlap in the analyzed face image data (step S403; No), the process proceeds to the process of step S401.

[0052] When the control unit 101 determines that the user's face and the user's hand overlap in the analyzed face image data (step S403; Yes), it acquires distance data (step S404). The control unit 101 determines whether the distance data is smaller than a predetermined distance set in advance (step S404). As the predetermined distance used for the determination in step S405, any one of the distances when the hand approaches the eye, the distance when the hand approaches the nose, and the distance when the hand approaches the mouth in the database shown in FIG. 3 is used. Which of the three distances is used as the predetermined distance may vary for each device. Here, the distance when the hand approaches the eye is used as the predetermined distance. When the control unit 101 determines that the distance data is equal to or greater than the predetermined distance (step S405; No), the process proceeds to the process of step S401.

[0053] When the control unit 101 determines that the distance data is smaller than a predetermined distance (step S405; Yes), the control unit 101 outputs an alarm to the mobile terminal 20 (step S406). When the mobile terminal 20 receives the alarm, the alarm output unit 208 outputs the alarm. The period for outputting the alarm is a predetermined period (for example, several seconds). Next, the control unit 101 determines whether or not an end condition is satisfied (step S407). Here, examples of the end condition include when a predetermined period has elapsed since the start of the determination process, and when a signal for stopping the determination process is received from the operation unit 106. When the control unit 101 determines that the end condition is satisfied (step S407; Yes), the control unit 101 ends the determination process. When the control unit 101 determines that the end condition is not satisfied (step S407; No), the process proceeds to step S401.

[0054] According to the determination process shown in FIG. 6, when it is determined in the control unit 101 that a hand is approaching the user's face, an alarm is output. If the timing from when the control unit 101 acquires the face image data in step S401 to proceeding to step S407 and then to the next step S401 is as short as several milliseconds to 1 second, the alarm will be continuously output while the hand is approaching the user's face. In this case, the output of the alarm stops while the hand is away from the user's face.

[0055] According to the control device 10 in an embodiment of the present invention, it is determined from the user's face image data whether or not the user's hand and face overlap, and when it is determined that the user's hand and face overlap, it is determined whether or not the distance between the mobile terminal 20 and the detection unit 30 is smaller than a predetermined distance. Thereby, it is possible to further improve the accuracy of the timing for outputting an alarm compared to determining whether a user's hand is approaching the face based on only the face image data.

[0056] (3. Second Embodiment) In this embodiment, a case will be described in which, in the mobile terminal 20, it is determined whether the user's hand is approaching the face, and an alarm is output so that the user does not touch the face. Note that parts different from the first embodiment will be described, and descriptions of overlapping parts will be omitted.

[0057] FIG. 7 is a diagram for explaining an outline of a system 1A according to an embodiment of the present invention. In this embodiment, in the mobile terminal 20, a program 300 is executed. In this embodiment, the mobile terminal 20 is a wearable device such as a wristwatch type or a ring type, for example.

[0058] In FIG. 7, the control device 10 transmits face image data acquired by the image capturing unit 103 from the communication unit 104 to the mobile terminal 20.

[0059] In the mobile terminal 20, the control unit 201 executes the program 300 stored in the storage unit 202 by the CPU to realize each function shown in FIG. 4 in the mobile terminal 20. The storage unit 202 stores the face image data 311 received from the control device 10. Further, the storage unit 202 stores a predetermined distance (the distance between the mobile terminal 20 and the detection unit 30) set in advance by the user. In this embodiment, it is sufficient that the storage unit 202 stores a predetermined distance of the user who uses the mobile terminal 20. Further, the storage unit 202 stores distance data 312 between the mobile terminal 20 and the detection unit 30 acquired by the detection unit 207 at each predetermined period. The storage unit 202 may accumulate and store the acquired distance data 312 and store it in the long term. Or the storage unit 202 may temporarily store the acquired distance data 312. When the distance data 312 is temporarily stored, the past distance data may be updated with the new distance data.

[0060] Next, with reference to FIG. 6, the determination process in this embodiment will be described.

[0061] When the mobile terminal 20 outputs a signal instructing the start of the determination process, the control unit 201 starts a process of determining whether the user's hand is approaching the face. First, the control unit 201 acquires the face image data 311 of the user captured by the image capturing unit 103 (step S401). Next, the control unit 201 analyzes the face image data 311 (step S402). Next, the control unit 201 determines whether the user's face and the user's hand (or finger) overlap based on the analyzed face image data 311 (step S403). If the control unit 201 determines that the user's face and the user's hand do not overlap in the analyzed face image data 311 (step S403; No), the process proceeds to the process of step S401.

[0062] If the control unit 201 determines that the user's face and the user's hand overlap in the analyzed face image data (step S403; Yes), it acquires the distance data 312 (step S404). The control unit 201 determines whether the distance data 312 is smaller than a preset predetermined distance (step S404). If the control unit 201 determines that the distance data 312 is equal to or greater than the predetermined distance (step S404; No), the process proceeds to the process of step S401.

[0063] If the control unit 201 determines that the distance data 312 is smaller than the predetermined distance (step S404; Yes), the control unit 201 outputs an alarm to the alarm output unit 208 (step S406), and the control unit 101 ends the determination process.

[0064] According to the mobile terminal 20 in an embodiment of the present invention, it is determined whether the user's hand and face overlap from the user's face image data, and when it is determined that the user's hand and face overlap, it is determined whether the distance between the detection unit 207 and the detection unit 30 of the mobile terminal 20 is smaller than a predetermined distance. Thereby, it is possible to further improve the accuracy of the timing for outputting an alarm compared to determining whether the user's hand is approaching the face only based on the face image data.

[0065] (4. Modification example) The above describes one embodiment of the present invention. However, one embodiment of the present invention can be modified into various forms as follows. Also, the above-described embodiment and the modification examples described below can be applied in combination with each other.

[0066] (1) In the first and second embodiments, the case where the distance between the mobile terminal 20 and the detection unit 30 is determined after determining whether the face and the hand overlap based on the analysis result of the face image data has been described. However, one embodiment of the present invention is not limited to this. After determining the distance between the mobile terminal 20 and the detection unit 30, it may be determined whether the face and the hand overlap based on the analysis result of the face image data.

[0067] FIG. 8 is a flowchart for explaining the determination process in one embodiment of the present invention. Here, the case where the determination process is executed in the control unit 101 of the control device 10 will be described. However, the determination process may be executed in the control unit 201 of the mobile terminal 20.

[0068] For example, when the mobile terminal 20 outputs a signal instructing the start of the determination process, the control unit 101 of the control device 10 starts the process of determining whether the user's hand is approaching the face. First, the control unit 101 acquires distance data from the mobile terminal 20 (step S411). The control unit 101 determines whether the distance data is smaller than a preset predetermined distance (step S412). When the control unit 101 determines that the distance data is equal to or greater than the predetermined distance (step S412; No), the process proceeds to the process of step S411.

[0069] When the control unit 101 determines that the distance data is smaller than a predetermined value (step S412; Yes), the control unit 101 acquires the face image data of the user captured by the image capturing unit 103 (step S413). Next, the control unit 101 analyzes the face image data (step S414). Next, the control unit 101 determines whether or not the user's face and the user's hand (or finger) overlap based on the analyzed face image data (step S415). When the control unit 101 determines that the user's face and the user's hand do not overlap in the analyzed face image data (step S415; No), the process proceeds to step S411.

[0070] When the control unit 101 determines that the user's face and the user's hand overlap in the analyzed face image data (step S415; Yes), the control unit 101 outputs an alarm to the mobile terminal 20 (step S406), and the control unit 101 ends the determination process. When the mobile terminal 20 receives the alarm, the alarm output unit 208 outputs the alarm.

[0071] (2) In the first and second embodiments, an example in which the mobile terminal 20 measures the distance from the detection unit 30 has been described, but an embodiment of the present invention is not limited to this. The detection unit 207 of the mobile terminal 20 alone may measure whether a hand is approaching the user's face or measure the distance.

[0072] (3) In the first and second embodiments, the case where the mobile terminal 20 is a wearable device has been described, but an embodiment of the present invention is not limited to this. The mobile terminal 20 may be a mobile terminal possessed by a user, such as a smartphone or a tablet, for example.

[0073] (4) The systems 1 and 1A are useful when the user works at a determined position. For example, it is also applicable to desk work in an office or when driving a car.

[0074] (5) In the first and second embodiments, the case where an alarm is output from the mobile terminal 20 has been described. However, an embodiment of the present invention is not limited to this. A communication unit and an alarm output unit may be provided in the article where the detection unit 30 is installed. When an alarm is output from the control device 10 or the mobile terminal 20, an alarm may be output from the alarm output unit of the article. For example, when an alarm is output from the control device 10 or the mobile terminal 20, voice or a warning sound may be output by the sound output unit of the article where the detection unit 30 is installed.

Explanation of Reference Numerals

[0075] 1, 1A: System, 10: Control device, 20: Mobile terminal, 30: Detection unit, 40: Network, 101: Control unit, 102: Storage unit, 103: Image capturing unit, 104: Communication unit, 105: Display unit, 106: Operation unit, 201: Control unit, 202: Storage unit, 204: Communication unit, 207: Detection unit, 208: Alarm output unit, 300: Program, 301: Acquisition unit, 302: Image analysis unit, 303: Determination unit, 304: Output unit, 311: Face image data, 312: Distance data

Claims

1. A control device having an image capturing unit that acquires face image data of a user, and a control unit that analyzes the face image data, A mobile terminal having a first detection unit attached near the user's wrist, A second detection unit attached near the user's head, and The first detection unit acquires distance data from the second detection unit, When the control unit determines that the face and the hand overlap by analyzing the face image data, the control unit acquires the distance data, and when the distance data is smaller than a predetermined distance set in advance, outputs an alarm to the mobile terminal. A system.

2. When the control unit determines that the face and the hand do not overlap, the control unit analyzes newly acquired face image data. The system according to claim 1.

3. When the acquired distance between the first detection unit and the second detection unit is equal to or greater than a predetermined distance set in advance, the system according to claim 1 or 2 analyzes newly acquired face image data.

4. The predetermined distance is the distance between the first detection unit and the second detection unit when the user brings the hand close to the user's eyes. The system according to any one of claims 1 to 3.

5. Acquire face image data of the user, and determine whether the face and the hand overlap by analyzing the face image data. When it is determined that the face and the hand overlap, the first detection unit of the mobile terminal worn near the user's wrist acquires distance data from the second detection unit worn near the user's head. A method of outputting an alarm when the acquired distance between the first detection unit and the second detection unit is smaller than a predetermined distance set in advance.

6. When it is determined that the face and the hand do not overlap, analyze newly acquired face image data. The method according to claim 5.

7. When the acquired distance between the first detection unit and the second detection unit is equal to or greater than a predetermined distance set in advance, analyze newly acquired face image data. The method according to claim 5 or 6.

8. The predetermined distance is the distance between the first detection unit and the second detection unit when the user brings the hand close to the eyes. The method according to any one of claims 5 to 7.

9. Acquire face image data of the user, Analyze the face image data, and determine whether a part of the face and the hand overlap based on the analysis result. When it is determined that a part of the face overlaps with the hand, obtain the distance between a first detection unit of a mobile terminal worn on the user's wrist and a second detection unit provided near the user's head, A program for causing a computer to output an alarm to the mobile terminal when the obtained distance between the first detection unit and the second detection unit is smaller than a predetermined distance.

Images