Information processing device and program

Abstract

To provide an information processing apparatus and program for effectively determining whether hand washing is done in a predetermined procedure.SOLUTION: An information processing apparatus, in an action determining system 1, comprises a memory, a camera interface, an input / output interface and a processor. The memory (NVM) stores a plurality of hand-washing patterns related to a procedure of hand washing. The camera interface is connected to a camera for taking an image of a hand. The input / output interface is for inputting an operation. The processor sets up a hand-washing pattern from the plurality of hand-washing patterns based on an operation inputted through the input / output interface, acquires a photographic image of a hand from the camera through the camera interface, determines an action class of the hand reflected in the photographic image based on a model the hand-washing pattern indicates, and determines whether the procedure of hand washing is appropriate based on the action class determined and the hand-washing pattern set up.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] A system for determining whether hand washing has been performed in a predetermined manner is provided. Such a system identifies a user's hand washing behavior and determines whether the identified hand washing behavior conforms to a specific hand washing pattern.

[0003] Furthermore, the required hand washing procedures may differ depending on the area, such as food factories, hospitals, shops, restaurants, nursing care facilities, and educational facilities. Conventionally, systems have sometimes been unable to properly determine whether hand washing has been performed according to the prescribed procedure, depending on the area, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-140678 Summary of the Invention [Problem to be solved by the invention]

[0005] In order to solve the above problem, an information processing device and a program are provided that can effectively determine whether hand washing has been performed in a predetermined procedure. [Means for solving the problem]

[0006] According to an embodiment, an information processing device includes a memory, a camera interface, an input / output interface, and a processor. The memory stores a plurality of hand-washing patterns related to hand-washing procedures. The camera interface is connected to a camera that captures images of hands. The input / output interface receives input of operations. The processor sets a hand-washing pattern from the plurality of hand-washing patterns based on the operations input through the input / output interface, acquires a captured image of the hand from the camera via the camera interface, determines a movement class of the hand captured in the captured image based on a model indicated by the hand-washing pattern, and determines whether the hand-washing procedure is appropriate based on the determined movement class and the set hand-washing pattern. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram schematically illustrating an example of the configuration of a motion determining system according to an embodiment. [Figure 2] FIG. 2 is a diagram schematically illustrating another configuration example of the motion determining system according to the embodiment. [Figure 3] FIG. 3 is a block diagram illustrating an example of the configuration of the movement determining device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a hand-washing pattern according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a pattern setting table according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a feature quantity model table according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of an action class table according to the embodiment. [Figure 8] FIG. 8 is a diagram showing an example of an instruction screen according to the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the operation of the movement determining device according to the embodiment. [Figure 10] FIG. 10 is a flowchart showing an example of the operation of the movement determining device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described with reference to the drawings. An action determination system according to an embodiment determines whether a user has washed their hands in a predetermined procedure. The action determination system captures an image of the user's hands in a sink or the like. The action determination system determines the action class of the user's hands from the captured image. For example, the action determination system determines hand actions classified by the parts of the hand being washed (e.g., the back of the hand, between the fingers, the thumb), etc. The action determination system determines whether the user has washed their hands in a predetermined procedure based on the order of the action classes, the duration of the action, etc.

[0009] 1 is a schematic diagram illustrating an example of the configuration of a motion determination system 1 according to an embodiment. As shown in FIG. 1, the motion determination system 1 includes a camera 3, an environmental sensor 4, an input / output device 5, a human presence sensor 6, a sink 20, and the like.

[0010] The sink 20 is a water tank where a user washes their hands. The sink 20 has a recessed structure. A drain groove is formed in the bottom of the sink 20. In addition, a faucet 21 is installed in the sink 20. The faucet 21 discharges water for a user to wash their hands. The discharged water is discharged from a drain in the sink 20.

[0011] Camera 3 is installed facing downward on top of sink 20. Camera 3 captures an area (captured area) including the user's hand. Camera 3 captures the user's hand from above. Camera 3 captures a two-dimensional image (captured image). Camera 3 outputs color information (RGB (Red Green Blue)) for each dot. The camera 3 may also be equipped with a light that illuminates the user's hands.

[0012] The environmental sensor 4 is a sensor that measures an environmental parameter related to the environment in which the sink 20 is installed. In this example, the environmental sensor 4 measures temperature as the environmental parameter. The environmental sensor 4 outputs a sensor signal that indicates the measured environmental parameter.

[0013] The environmental sensor 4 may measure humidity as an environmental parameter. The environmental sensor 4 may also measure a plurality of environmental parameters. The configuration of the environmental parameters measured by the environmental sensor 4 is not limited to a specific configuration.

[0014] The input / output device 5 is installed in a position visible to the user. In this example, the input / output device 5 is installed in front of the user washing their hands at the sink 20.

[0015] The input / output device 5 is an interface that receives instructions from an operator and displays various information to the operator. The input / output device 5 is made up of an operation unit that receives instructions and a display unit that displays information.

[0016] The input / output device 5 transmits a signal indicating an operation received from an operator as the operation of the operation unit to a later-described operation determination device 10. For example, the operation unit is configured with a touch panel. The operation unit may also include a keyboard or a numeric keypad.

[0017] The input / output device 5 operates as a display unit to display an image from the motion determining device 10. For example, the display unit is configured with a liquid crystal monitor. The display unit may be formed integrally with a touch panel serving as an operation unit.

[0018] The human presence sensor 6 is a sensor that detects the user's hand. The human presence sensor 6 is formed near the faucet 21. The human presence sensor 6 uses infrared rays or the like to detect whether the user's hand has entered a predetermined area. The human presence sensor 6 transmits a detection signal indicating that the user's hand has been detected to the movement determination device 10.

[0019] Next, another example of the movement determination system 1 will be described. FIG. 2 shows a schematic configuration example of a motion determination system 1′, which is another example of the motion determination system 1. As shown in FIG.

[0020] Like the motion determination system 1, the motion determination system 1' includes a camera 3, an environmental sensor 4, an input / output device 5, a human sensor 6, a sink 20, and the like.

[0021] The camera 3 is installed to the side (left side in FIG. 2) of the sink 20. The camera 3 captures an image of the user's hand from the side. The environmental sensor 4 and the input / output device 5 are installed next to the sink 20 . The other configurations are the same as those of the movement determination system 1', and therefore the description thereof will be omitted.

[0022] Next, the control systems of the motion determination systems 1 and 1' will be described. The motion determination systems 1 and 1' have the same configuration, and therefore will be described as motion determination system 1.

[0023] Fig. 3 is a block diagram showing an example of the configuration of a control system of the movement determination system 1. As shown in Fig. 2, the movement determination system 1 includes a camera 3, an environmental sensor 4, an input / output device 5, a human presence sensor 6, and a movement determination device 10. The movement determination device 10, the camera 3, the environmental sensor 4, the input / output device 5, and the human presence sensor 6 are communicatively connected.

[0024] The movement determination device 10 (information processing device) includes a processor 11, a ROM 12, a RAM 13, an NVM 14, a communication interface 15, a camera interface 16, an environment sensor interface 17, an input / output interface 18, a human sensor interface 19, and the like.

[0025] The processor 11 is communicably connected to the ROM 12, RAM 13, NVM 14, communication interface 15, camera interface 16, environmental sensor interface 17, input / output interface 18, and human sensor interface 19.

[0026] The camera interface 16 is communicatively connected to the camera 3. The environment sensor interface 17 is communicatively connected to the environment sensor 4. The input / output interface 18 is communicatively connected to the input / output device 5. The human presence sensor interface 19 is communicatively connected to the human presence sensor 6. The camera 3, the environmental sensor 4, the input / output device 5, and the human sensor 6 are as described above.

[0027] The processor 11 controls the overall operation of the motion determination device 10. That is, the processor 11 controls the overall operation of the motion determination device 10. The processor 11 may include an internal cache and various interfaces. The processor 11 realizes various processes by executing programs stored in advance in the internal cache, the ROM 12, or the NVM 14.

[0028] Some of the various functions realized by the processor 11 executing the programs may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0029] The ROM 12 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 12 are pre-installed in accordance with the specifications of the movement determination device 10. The ROM 12 stores, for example, a program for controlling a circuit board of the movement determination device 10.

[0030] RAM 13 is a volatile memory. RAM 13 temporarily stores data being processed by processor 11. RAM 13 stores various application programs based on instructions from processor 11. RAM 13 may also store data necessary for executing application programs and execution results of application programs.

[0031] The NVM 14 is a nonvolatile memory to which data can be written and rewritten. The NVM 14 is configured, for example, by a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The NVM 14 stores control programs, applications, various data, and the like according to the operational use of the operation determination device 10.

[0032] The NVM 14 also stores in advance a pattern setting table showing a plurality of hand washing patterns, a feature model table showing feature models for determining hand movement classes, and an action class table showing the movement classes. The pattern setting table, feature model table, and action class table will be described later.

[0033] The NVM 14 also stores in advance a feature model for determining hand motion classes (motion classes C1 to C6 in this example). For example, the feature model is a network obtained by machine learning or the like.

[0034] The NVM 14 stores a plurality of feature quantity models corresponding to different shooting environments (such as shooting a hand from above or a hand from the side).

[0035] The communication interface 15 is an interface for transmitting and receiving data to and from an external device. The communication interface 15 connects to the external device via a network or the like. For example, the communication interface 15 supports wired or wireless LAN (Local Area Network) connections.

[0036] Camera interface 16 is an interface for transmitting and receiving data to and from camera 3. For example, camera interface 16 transmits a signal instructing camera 3 to take a photograph, based on the control of processor 11. Camera interface 16 also acquires captured images from camera 3. For example, camera interface 16 may be one that supports a USB (Universal Serial Bus) connection, or one that supports a connection via Camera Link.

[0037] The environmental sensor interface 17 is an interface for transmitting and receiving data to and from the environmental sensor 4. For example, the environmental sensor interface 17 acquires a sensor signal from the environmental sensor 4. The environmental sensor interface 17 transmits the acquired sensor signal to the processor 11. For example, the environmental sensor interface 17 may support a USB connection.

[0038] The input / output interface 18 is an interface for transmitting and receiving data to and from the input / output device 5. For example, the input / output interface 18 receives a signal indicating an operation accepted from an operator from the input / output device 5. The input / output interface 18 transmits the received signal to the processor 11. Furthermore, the input / output interface 18 transmits information indicating a screen to be displayed to the operator to the input / output device 5 under the control of the processor 11. For example, the input / output interface 18 may be one that supports a USB connection or one that supports a connection via a parallel interface.

[0039] The movement determining device 10 may include other components as needed in addition to the components shown in FIG. 3, or certain components may be excluded from the movement determining device 10. For example, the movement determining device 10 is a desktop PC, a notebook PC, a tablet PC, a smartphone, or the like.

[0040] Next, the hand washing pattern will be described. The hand-washing pattern relates to the hand-washing procedure and is a pattern used by the movement determination system 1 to determine whether the hand-washing procedure is appropriate.

[0041] Figure 4 shows an example of a hand-washing pattern. FIG. 4 shows hand washing patterns A, B, and C.

[0042] Hand-washing pattern A is used to determine the hand-washing procedure in factory "A." Hand-washing pattern A indicates that the hand-washing procedure is to continue the action classes C3, C4, and C5 in that order, each for 5 seconds. Hand-washing pattern A is also used when camera 3 captures the user's hands from above (as in Figure 1).

[0043] Like hand-washing pattern A, hand-washing pattern B is used to determine the hand-washing procedure in factory "A." Like hand-washing pattern A, hand-washing pattern B indicates that the hand-washing procedure consists of action classes C3, C4, and C5, each lasting 5 seconds in that order. Hand-washing pattern B is also used when camera 3 captures the user's hands from the side (here, from the left) (as in Figure 2).

[0044] Hand-washing pattern C is used to determine the hand-washing procedure in factory "I." Hand-washing pattern C indicates that the hand-washing procedure consists of action classes C3, C4, C5, and C6, performed in any order, for a total of 20 seconds. Hand-washing pattern C is also used when camera 3 captures the user's hands from above (as in Figure 1).

[0045] Next, the pattern setting table will be described. Fig. 5 shows an example of the configuration of the pattern setting table. As shown in Fig. 5, the pattern setting table stores a "pattern ID," "action sequence ID," "action class ID," "action duration," "feature model ID," and "environmental condition" in association with each other.

[0046] "Pattern ID" is an ID that identifies a hand-washing pattern. Here, "A" indicates hand-washing pattern A, "B" indicates hand-washing pattern B, and "C" indicates hand-washing pattern C.

[0047] The "action order ID" indicates the order of the action class. For example, in the hand-washing pattern A, the pattern setting table indicates "1" as the "action order ID" corresponding to "3" (action class C3) as the "action class ID." That is, in the hand-washing pattern A, the pattern setting table indicates "1" as the order of action class C3.

[0048] Furthermore, the pattern setting table indicates "0" as the "action order ID" corresponding to each "action class ID" in hand-washing pattern C. In other words, the pattern setting table indicates that the order of each action class in hand-washing pattern C is random.

[0049] Additionally, the "action class ID" indicates the action class to be executed in the hand-washing pattern. Here, "1" to "6" indicate action classes C1 to C6, respectively.

[0050] "Action duration" indicates the duration of the corresponding action class. For example, for hand-washing pattern A, the pattern setting table indicates "5 seconds" as the "action duration" corresponding to "3" (action class C3) as the "action class ID." In other words, the pattern setting table indicates that for hand-washing pattern A, action class C3 will continue for 5 seconds.

[0051] Furthermore, the pattern setting table indicates "0" as the "action duration" corresponding to each "action class ID" for hand-washing pattern C. The pattern setting table also indicates "20 seconds" as the "action duration" at the end. In other words, the pattern setting table indicates that each action class will continue for a total of 20 seconds for hand-washing pattern C.

[0052] Furthermore, the "feature model ID" is an ID that identifies the feature model used to determine the action class. The "feature model ID" corresponds to the direction in which the camera 3 photographs the hand. Here, when the camera 3 photographs the hand from above (as in Figure 1), the "feature model ID" indicates "H1." When the camera 3 photographs the hand from the side (as in Figure 2), the "feature model ID" indicates "H2."

[0053] The "environmental conditions" indicate modifications according to the environment in which the sink 20 is installed. That is, the "environmental conditions" indicate modifications to the hand-washing procedure according to the environmental parameters. For example, the "environmental conditions" indicate that the "action duration" should be increased (or decreased) when the temperature is above a predetermined threshold. Here, for hand-washing pattern A, the "environmental conditions" indicate that 5 seconds should be added to each "action duration" when the temperature is 30 degrees or higher. Also, for hand-washing pattern B, the "environmental conditions" indicate that 5 seconds should be added to each "action duration" when the temperature is 25 degrees or higher.

[0054] It is not necessary to set the "environmental conditions." For example, in hand-washing pattern C, the "environmental conditions" are not set.

[0055] The "environmental conditions" may also be used to add or delete an operation class. The "environmental conditions" may also be used to change the order of the operation classes. The configuration of the "environmental conditions" is not limited to a specific configuration.

[0056] The pattern setting table may be updated as needed in accordance with an operator's operation.

[0057] Next, the feature quantity model table will be described. Fig. 6 shows an example of the structure of a feature model table. As shown in Fig. 6, the feature model table stores a "feature model ID," a "corresponding action class ID," and a "shooting environment" in association with each other.

[0058] The "feature model ID" is an ID that identifies a feature model. The "Corresponding Action Class ID" indicates the action class that the feature model can determine. Here, "H1" and "H2" indicate that the action classes indicated by "1" to "8" (action classes C1 to C8) can be determined.

[0059] "Shooting environment" indicates the shooting environment that the feature model corresponds to. Here, the feature model table indicates that "H1" corresponds to a shot image taken from above. Also, the feature model table indicates that "H2" corresponds to a shot image taken from the side (left and right).

[0060] The feature quantity model table may be updated as needed in accordance with an operator's operation or the like.

[0061] Next, the operation class table will be described. Fig. 7 shows an example of the configuration of an action class table. As shown in Fig. 7, the action class table stores an "action class ID," an "action class display name," and an "action class display image" in association with each other.

[0062] "Action class ID" is an ID that identifies an action class. The "action class display name" is a character string that indicates an action class. For example, the "action class display name" indicates "whisk well" as the "action class display name" of the action class C3.

[0063] The "action class display image" is an image that indicates an action class. For example, the "action class display image" is composed of an illustration of the action class and an "action class display name."

[0064] The action class table may be updated as needed in accordance with an operator's operation or the like.

[0065] Next, a description will be given of functions realized by the movement determining device 10. The functions realized by the movement determining device 10 are realized by the processor 11 executing a program stored in the internal memory, the ROM 12, the NVM 14, or the like.

[0066] First, the processor 11 has a function of setting a hand-washing pattern. For example, the operator inputs a hand-washing pattern into the input / output device 5 based on the installation location of the sink 20, the angle of the camera 3, and the like.

[0067] The processor 11 inputs the operation via the input / output interface 18. When the operation is input, the processor 11 sets a hand-washing pattern according to the operation.

[0068] The processor 11 also has the capability to modify the hand-washing pattern based on environmental parameters.

[0069] When a hand-washing pattern is set, processor 11 determines whether the ID of the set hand-washing pattern is included in the pattern setting table.

[0070] When it is determined that the ID of the set hand washing pattern is included in the pattern setting table, the processor 11 measures the environmental parameters through the environmental sensor interface 17. After measuring the environmental parameters, the processor 11 determines whether the measured environmental parameters match the environmental conditions corresponding to the ID of the set hand washing pattern.

[0071] When the environmental parameters match the environmental conditions, the processor 11 modifies the hand-washing pattern based on the environmental conditions.

[0072] The processor 11 also has a function of acquiring a feature model based on a hand-washing pattern.

[0073] If the environmental parameters do not match the environmental conditions, or if the hand-washing pattern is modified, the processor 11 refers to the pattern setting table and acquires from the NVM 14 a feature model corresponding to the ID of the hand-washing pattern.

[0074] Processor 11 also has a function of constructing a determination logic for determining whether the hand-washing procedure is appropriate.

[0075] When the feature quantity model is acquired, the processor 11 refers to the pattern setting table and acquires the action order ID, action class ID, and action duration of the hand washing pattern (or the corrected hand washing pattern).

[0076] When the action order ID, action class ID, and action duration are acquired, the processor 11 constructs a determination logic based on the action order ID, action class ID, action duration, and the like.

[0077] If it is determined that the ID of the set hand washing pattern is not included in the pattern setting table, the processor 11 acquires a default feature quantity model and constructs default determination logic.

[0078] The processor 11 also has a function of displaying on the input / output device 5 an instruction screen showing the hand-washing procedure.

[0079] After constructing the judgment logic, processor 11 refers to the action class table and acquires an action class display image corresponding to the acquired action class ID. After acquiring the action class display image, processor 11 generates an instruction screen in which the action class display images are arranged according to the acquired action order ID. After generating the instruction screen, processor 11 displays the generated instruction screen on input / output device 5 via input / output interface 18.

[0080] Fig. 8 shows an example of an instruction screen. Fig. 8 shows an instruction screen corresponding to hand-washing pattern A. As shown in Fig. 8, the instruction screen includes a display area 31, a display area 32, and an icon 33.

[0081] The display area 31 displays the action class display images according to the action order IDs. Here, the display area 31 displays the action class display images in the order of action classes C3, C4, and C5.

[0082] The display area 32 displays the captured image captured by the camera 3. Here, the display area 32 displays the captured image including the user's hand.

[0083] Icon 33 is an icon for inputting an operation to end the judgment of the hand-washing procedure. The configuration of the instruction screen is not limited to a specific configuration.

[0084] Processor 11 also has a function of determining whether the hand-washing procedure has been performed appropriately based on a determination logic.

[0085] The processor 11 determines whether a user's hand has been detected using the human sensor 6. The processor 11 may detect the user's hand based on an image captured by the camera 3.

[0086] When it is determined that the user's hand has been detected, the processor 11 acquires a captured image from the camera 3 via the camera interface 16.

[0087] When a photographed image is acquired, processor 11 determines a hand movement class based on the feature model. After determining the movement class, processor 11 determines whether the movement class set in the initial order matches the determined movement class.

[0088] If it is determined that the two are consistent, the processor 11 measures the time (motion time) during which the user's hand is determined to belong to the relevant motion class. The processor 11 waits until the measured motion time exceeds the motion duration corresponding to the relevant motion class. If the measured motion time does not exceed the motion duration even after a predetermined time has elapsed since the detection of the hand (or since the hand motion class has changed), the processor 11 determines that the hand-washing procedure was not performed properly. It may be determined that:

[0089] When the measured action time exceeds the action duration, the processor 11 updates the instruction screen. For example, the processor 11 displays information indicating that the action class has been completed on the instruction screen. For example, the processor 11 displays a circle on the action class display image corresponding to the action class.

[0090] When the instruction screen is updated, processor 11 determines whether there is an action class set to be next in the order of the action class. If it determines that there is an action class set to be next in the order of the action class, processor 11 operates as described above for the action class.

[0091] If there is no action class set in the next order, processor 11 determines that the user has washed their hands properly.

[0092] In addition, when the action classes are in no particular order and the total time of each action class is set as the action duration, processor 11 may calculate the total time during which the user's hands are determined to be in any one of the action classes set in the hand-washing pattern. Processor 11 may determine that the user has washed their hands properly when the total time exceeds the action duration.

[0093] Next, an example of the operation of the movement determining device 10 will be described. 9 and 10 are flowcharts for explaining an example of the operation of the movement determining device 10. FIG.

[0094] First, processor 11 of action determination device 10 inputs a hand-washing pattern via input / output interface 18 (S11). When the hand-washing pattern is input, processor 11 determines whether the ID of the input hand-washing pattern is included in the pattern setting table (S12).

[0095] If it is determined that the ID of the input hand washing pattern is included in the pattern setting table (S12, YES), the processor 11 measures environmental parameters through the environmental sensor interface 17 (S13).

[0096] After measuring the environmental parameters, processor 11 determines whether the measured environmental parameters match the environmental conditions (S14). If it determines that the environmental parameters match the environmental conditions (S14, YES), processor 11 modifies the hand-washing pattern based on the environmental conditions (S15).

[0097] If it is determined that the environmental parameters do not match the environmental conditions (S14, NO), or if the hand-washing pattern is modified based on the environmental conditions (S15), the processor 11 refers to the feature model table to obtain a feature model (S16).

[0098] When the feature quantity model is acquired, the processor 11 constructs a decision logic (S17). After constructing the decision logic, the processor 11 displays an instruction screen on the input / output device 5 via the input / output interface 18 (S18).

[0099] If it is determined that the input hand washing pattern ID is not included in the pattern setting table (S12, NO), processor 11 acquires a default feature quantity model (S19). After acquiring the default feature quantity model, processor 11 constructs a default determination logic (S20). After constructing the default determination logic, processor 11 displays a default instruction screen on input / output device 5 via input / output interface 18 (S21).

[0100] When an instruction screen is displayed on the input / output device 5 (S18), or when a default instruction screen is displayed on the input / output device 5 (S21), the processor 11 determines whether the user's hand has been detected through the human sensor interface 19 (S22).

[0101] If it is determined that the user's hand has not been detected (S22, NO), the processor 11 returns to S22.

[0102] When it is determined that the user's hand has been detected (S22, YES), the processor 11 acquires the action class and the action duration according to the action order ID from the determination logic (S23). After acquiring the action class and the action duration, the processor 11 acquires the captured image from the camera 3 via the camera interface 16 (S24).

[0103] When the captured image is acquired, the processor 11 determines the movement class of the user's hand based on the captured image and the feature model (S25). After determining the movement class, the processor 11 determines whether the determined movement class matches the movement class acquired from the determination logic (S26).

[0104] If it is determined that the two action classes do not match (S26, NO), the processor 11 returns to S24.

[0105] If it is determined that both action classes match (S26, YES), the processor 11 adds up the action times (S27). After adding up the action times, the processor 11 determines whether the action time exceeds the action duration (S28).

[0106] If it is determined that the operation time has not exceeded the operation duration time (S28, NO), the processor 11 returns to S24.

[0107] When it is determined that the action time has exceeded the action duration (S28, YES), processor 11 updates the instruction screen (S29). After updating the instruction screen, processor 11 determines whether there is an action class set in the next order (S30).

[0108] If it is determined that an action class set in the next order exists (S30, YES), the processor 11 returns to S23.

[0109] If it is determined that there is no action class set in the next order (S30, NO), processor 11 resets the instruction screen (S31). After resetting the instruction screen, processor 11 determines whether an operation to continue the action has been input via input / output interface 18 (S32).

[0110] If it is determined that an operation to continue the action has been input (YES in S32), processor 11 returns to S22. When it is determined that an operation to continue the operation is not input (S32, NO), processor 11 ends the operation.

[0111] The processor 11 may return to S22 after S31. Furthermore, the processor 11 may display an instruction screen on the input / output device 5 after detecting the user's hand.

[0112] Furthermore, if the process does not proceed to S31 even after a predetermined time has elapsed since the detection of a hand, processor 11 may determine that the hand-washing procedure has not been performed properly. If it is determined that the hand-washing procedure has not been performed properly, processor 11 may display an error on input / output device 5.

[0113] Furthermore, the processor 11 may transmit, via the communication interface 15, to an external device, a determination result indicating whether or not the hand-washing procedure has been properly performed.

[0114] Furthermore, the movement determination system 1 may include a distance sensor that measures distance in an area including the hand. The processor 11 may determine the movement class of the hand based on the measurement result of the distance sensor. The processor 11 may also determine the movement class of the hand based on the measurement result of the distance sensor and a captured image.

[0115] The environmental condition may also be configured as a function that inputs an environmental parameter, for example, a function that inputs a temperature and outputs a value to be added to, subtracted from, divided by, or multiplied by the operation hold time.

[0116] Furthermore, the movement determining device 10, the camera 3, the environmental sensor 4, the input / output device 5 and the human sensor 6 (or some of these) may be configured integrally.

[0117] The movement determination system configured as described above sets a hand-washing pattern based on the area where the sink is installed, etc. The movement determination system determines whether the user has washed their hands appropriately according to the hand-washing pattern. As a result, the movement determination system can effectively determine whether the user has washed their hands appropriately.

[0118] The motion determination system also measures environmental parameters in the environment where the sink is installed. The motion determination system modifies the hand-washing pattern based on the measured environmental parameters. As a result, the motion determination system can set an appropriate hand-washing pattern even in situations where bacteria are likely to grow due to elevated temperatures, etc.

[0119] The action determination system also displays an instruction screen showing hand-washing procedures based on the hand-washing pattern, thereby enabling the action determination system to present appropriate hand-washing procedures to the user.

[0120] The program may be transferred in a state where it is stored in a device, or in a state where it is not stored in a device. In the latter case, the program may be transferred via a network, or in a state where it is recorded on a recording medium. The recording medium is a non-transitory tangible medium. The recording medium is a computer-readable medium. The form of the recording medium is not important as long as it is a medium that can store the program and is computer-readable, such as a CD-ROM or a memory card.

[0121] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. The original claims are set forth below. [1] a memory for storing a plurality of hand washing patterns relating to hand washing procedures; a camera interface that connects to a camera that takes pictures of the hand; an input / output interface for inputting operations; setting the hand-washing pattern from the plurality of hand-washing patterns based on an operation input through the input / output interface; Acquiring a photographed image of a hand from the camera through the camera interface; determining a hand movement class of the hand captured in the captured image based on a model indicated by the hand-washing pattern; determining whether the hand washing procedure is appropriate based on the determined action class and the set hand washing pattern; a processor; An information processing device comprising: [2] the hand-washing pattern indicates an environmental condition related to an environment; an environmental sensor interface for connecting to an environmental sensor for measuring an environmental parameter; the processor modifies the hand-washing pattern based on the measured environmental parameters and the environmental conditions. [1] The information processing device according to [1]. [3] the environmental parameter is temperature; The information processing device according to [2], wherein the processor increases the operation duration of the operation class based on the temperature. [4] The hand-washing pattern indicates the order and duration of the action classes. An information processing device according to any one of [1] to [3]. [5] the input / output interface is connected to a monitor; the processor displays, via the input / output interface, an instruction screen on the monitor that shows hand-washing procedures based on the hand-washing pattern; [1] The information processing device according to [1]. [6] A program executed by a processor, the processor, A function of setting a hand washing pattern from a plurality of hand washing patterns related to hand washing procedures based on input operations; A function to capture images of hands, a function of determining a hand movement class of the hand captured in the captured image based on a model indicated by the hand washing pattern; a function of determining whether the hand washing procedure is appropriate based on the determined action class and the set hand washing pattern; A program to make this happen. [Explanation of symbols]

[0122] 1...motion determination system, 1'...motion determination system, 3...camera, 4...environmental sensor, 5...input / output device, 6...human presence sensor, 10...motion determination device, 11...processor, 12...ROM, 13...RAM, 14...NVM, 15...communication interface, 16...camera interface, 17...environmental sensor interface, 18...input / output interface, 19...human presence sensor interface, 20...sink, 21...faucet, 31...display area, 32...display area, 33...icon

Claims

1. a memory for storing a plurality of hand washing patterns relating to hand washing procedures; a camera interface that connects to a camera that takes pictures of the hand; an input / output interface for inputting operations; an environmental sensor interface for connecting to an environmental sensor for measuring an environmental parameter; selecting a hand-washing pattern that indicates an environmental condition from the plurality of hand-washing patterns based on an operation input through the input / output interface; modifying the hand washing pattern based on the measured environmental parameters and the environmental conditions; Acquiring a photographed image of a hand from the camera through the camera interface; determining a hand movement class of the hand captured in the captured image based on a model indicated by the hand-washing pattern; determining whether the hand washing procedure is appropriate based on the determined action class and the set hand washing pattern; a processor; An information processing device comprising:

2. the environmental parameter is temperature; The information processing device according to claim 1 , wherein the processor increases an operation duration of the operation class based on the temperature.

3. The hand-washing pattern indicates the order and duration of the action classes.

3. The information processing device according to claim 1 or 2.

4. the input / output interface is connected to a monitor; the processor displays, via the input / output interface, an instruction screen on the monitor that shows hand-washing procedures based on the hand-washing pattern; The information processing device according to claim 1 .

5. A program executed by a processor, the processor, A function of setting a hand washing pattern indicating environmental conditions from a plurality of hand washing patterns relating to hand washing procedures based on input operations; modifying the hand washing pattern based on the measured environmental parameters and the environmental conditions; A function to capture images of hands, a function of determining a hand movement class of the hand captured in the captured image based on a model indicated by the hand washing pattern; a function of determining whether the hand washing procedure is appropriate based on the determined action class and the set hand washing pattern; A program to make this happen.

6. The environmental parameter is temperature, The program product of claim 5 , further comprising: causing the processor to execute a function of increasing an operation duration of the operation class based on the temperature.

Images