Information processing device and program

Abstract

To provide an information processing device which evaluates a series of predetermined operations while considering variation between individuals.SOLUTION: An information processing device comprises: an identification information interface (I / F); a memory: an image I / F; a reporting I / F; and at least one processor. The at least one processor executes an operation threshold setting mode for setting an individual parameter for each user using a common acceptance threshold for each operation included in a series of predetermined operation, and an operation determination mode for determining whether the series of work has been executed using the individual parameters. The at least one processor reports the common acceptance threshold for each operation to a user in the operation threshold setting mode, determines a continuation length for each operation of the user on the basis of an image in which an object region is imaged, and associates a parameter based on a ratio of the continuation length for each operation and the common acceptance threshold with identification information identifying the user as the individual parameter related to the user and stores them in the memory.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an information processing device and a program. [Background technology]

[0002] A system for determining whether hand washing has been performed in a predetermined sequence is provided, which identifies a user's hand washing actions and determines whether the identified hand washing actions conform to a specific hand washing pattern.

[0003] In such a situation, in a system that uses images to evaluate a series of predetermined tasks such as hand-washing actions, for example, even when multiple users perform the same hand-washing action, the results of the action classification may differ due to individual variability. Here, the variability between individuals includes variability in the sense of time between multiple users and misclassification of the action due to differences in the actions between multiple users. In this way, if the judgment results differ between users or are not judged correctly despite the user performing a predetermined hand-washing action, the user may be dissatisfied with the judgment result. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to evaluate a predetermined series of tasks while taking into account inter-personal variability. [Means for solving the problem]

[0005] The information processing device according to the embodiment includes an identification information interface, a memory, an image interface, a notification interface, and at least one processor. The identification information interface acquires identification information for identifying a user. The memory stores a work pattern indicating the actions of a predetermined series of tasks, the common pass threshold for each action included in the series of tasks, and an order. The image interface acquires a captured image of a target area. The notification interface notifies the user. The at least one processor executes an action threshold setting mode in which a personal parameter for each user is set using the common pass threshold, and an action determination mode in which the personal parameter is used to determine whether the series of tasks has been performed. In the action threshold setting mode, the at least one processor notifies the user of the common pass threshold for each action, determines a duration of each action of the user based on the captured image, and stores a parameter based on a ratio between the determined duration of each action and the common pass threshold in association with the identification information in the memory as a personal parameter for the user identified by the identification information. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a motion determining device according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of another configuration of the motion determining device according to the embodiment. [Diagram 3] FIG. 3 is a block diagram showing an example of a hardware configuration of the motion determining device according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a hand-washing pattern according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a pattern setting table according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a feature quantity model table according to the embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of an action class table according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of an identification information table according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a functional configuration of a motion determining device according to an embodiment. [Figure 10] FIG. 10 is a diagram showing an example of an instruction screen for the operational threshold setting mode according to the embodiment. [Figure 11] FIG. 11 is a diagram showing an example of an instruction screen for the motion determination mode according to the embodiment. [Figure 12] FIG. 12 is a flowchart showing an example of the flow of processing executed in the movement determining device according to the embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the process flow of each mode in FIG. [Figure 14] FIG. 14 is a flowchart showing an example of the process flow of the motion determination mode of FIG. [Figure 15] FIG. 15 is a flowchart showing an example of the process flow of the operational threshold setting mode of FIG. [Figure 16] FIG. 16 is a diagram illustrating another example of the identification information table according to the embodiment. [Figure 17] FIG. 17 is a diagram for explaining another example of the operation threshold setting mode according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an information processing device, a display terminal, a system, an information processing method, a program, and a recording medium according to embodiments will be described with reference to the drawings.

[0008] The embodiment described below is an embodiment of an information processing device, a display terminal, a system, an information processing method, a program, and a recording medium, and does not limit the configuration, specifications, etc. The embodiment described below is an example of application to a system that uses images to evaluate the hand-washing action of a person washing their hands.

[0009] The motion determination device (system) according to the embodiment is an example of a system that determines whether a user's hand washing has been performed in a predetermined procedure. The motion determination device captures an image of the user's hand in a sink or the like. The motion determination device determines a motion class of the user's hand from the captured image. For example, the motion determination device determines hand motions classified according to the parts being washed (back of the hand, between the fingers, thumb, etc.) as motion classes. The motion determination device determines whether the user's hand washing has been performed in a predetermined procedure based on the order of the motion classes, the duration of the motion, etc.

[0010] 1 is a diagram illustrating an example of the configuration of a motion determining device 1 according to an embodiment. The motion determining device 1 includes a camera 3, an input / output device 5 (display terminal), a reader 6, and a sink 20, as shown in FIG.

[0011] Sink 20 is a water tank in which a user washes his / her hands. Sink 20 has a recessed structure. A drain groove is formed in the bottom of sink 20. Faucet 21 is installed in sink 20, for example, at a position facing the drain groove, i.e., so as to be able to discharge water into a space formed by the recessed structure of sink 20. Faucet 21 discharges water for a user to wash their hands. The discharged water is discharged from the drain groove of sink 20.

[0012] Camera 3 is installed above sink 20, for example at a position facing the space formed by the recessed structure of sink 20, facing downwards with the drain side as the shooting direction. Camera 3 shoots an area (target area) including the user's hands. Camera 3 shoots the user's hands from above. Camera 3 shoots a two-dimensional image (shot image). Camera 3 outputs color information (RGB (Red Green Blue) values) for each dot. Camera 3 may also have a light that illuminates the user's hands.

[0013] The input / output device 5 is installed in a position visible to the user. In the example of Fig. 1, the input / output device 5 is installed in front of the user who is washing his / her hands at the sink 20.

[0014] The input / output device 5 is an interface that receives input of instructions from an operator and displays various information to the operator. The input / output device 5 has an operation unit that receives input of instructions and a display unit that displays information.

[0015] The input / output device 5 transmits a signal indicating an operation received from an operator to a control device 10 (described later) as an operation of the operation unit. For example, the operation unit is configured using a touch panel. The operation unit may also have a keyboard or a numeric keypad.

[0016] The input / output device 5 operates as a display unit to display an image from the control device 10. For example, the display unit is configured using a liquid crystal monitor. The display unit may be formed integrally with a touch panel serving as an operation unit.

[0017] Note that the input / output device 5 may have an output unit that outputs voice or notification sound from the control device 10, instead of or in addition to the display unit. This output unit is configured using, for example, a speaker.

[0018] The reader 6 is an acquisition device that acquires identification information for identifying a user. For example, the reader 6 acquires identification information (such as an ID) for identifying a user from a terminal (such as an IC card or an RFID (radio frequency identification) tag) held by the user. Here, it is assumed that the user connects his / her own terminal to the reader 6. The reader 6 acquires the identification information from the terminal by connecting to the terminal via a wired or wireless connection. The reader 6 may also read codes such as barcodes or two-dimensional codes. The identification information may be information for identifying an individual user, or may indicate the user's position (such as a member of the cooking department or a plating worker).

[0019] Here, another example of the movement determining device 1 will be described.

[0020] Fig. 2 is a diagram illustrating another configuration example (motion determination device 1') of the motion determination device 1 according to the embodiment. The configuration example (motion determination device 1') of Fig. 2 includes a camera 3, an input / output device 5, a reader 6, and a sink 20, similar to the configuration example (motion determination device 1) of Fig. 1.

[0021] In the configuration example of FIG. 2, the camera 3 is installed beside the sink 20 (on the left side in FIG. 2). The camera 3 captures an image of the user's hand from the side (the side). The input / output device 5 is installed beside the sink 20. The other configuration is similar to the configuration example of FIG. 1 (motion determination device 1), and therefore description thereof will be omitted.

[0022] In this embodiment, when there is no need to distinguish between the configuration example of FIG. 1 (motion determining device 1) and the configuration example of FIG. 2 (motion determining device 1′), they will be simply referred to as the motion determining device 1.

[0023] Next, a control system of the movement determining device 1 will be described.

[0024] Fig. 3 is a block diagram showing an example of a hardware configuration of the movement determining device 1 according to the embodiment. As shown in Fig. 3, the movement determining device 1 further includes a control device 10. The control device 10 is communicatively connected to the camera 3, the input / output device 5, and the reader 6.

[0025] The control device 10 (information processing device) has a processor 11, a ROM 12, a RAM 13, an NVM (Non-volatile memory) 14, a communication I / F (interface) 15, a camera I / F 16, an input / output I / F 18, and a reader I / F 19.

[0026] The processor 11 is communicatively connected to the ROM 12, the RAM 13, the NVM 14, the communication I / F 15, the camera I / F 16, the input / output I / F 18, and the reader I / F 19. The camera 3 is communicatively connected to the camera I / F 16. The input / output I / F 18 is communicatively connected to the input / output device 5. The reader 6 is communicatively connected to the reader I / F 19.

[0027] The processor 11 controls the overall operation of the control device 10. The processor 11 may have an internal cache and various interfaces, etc. The processor 11 executes programs stored in advance in the internal cache, the ROM 12, or the NVM 14 to realize various processes.

[0028] Some of the various functions realized by the processor 11 executing the programs may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0029] With regard to the various functions realized by processor 11 executing a program, one function may be realized by cooperation of two or more processors, or two or more functions may be realized by a common processor.

[0030] The ROM 12 is a non-volatile memory in which control programs, control data, etc. are stored in advance. The control programs and control data stored in the ROM 12 are pre-installed in accordance with the specifications of the control device 10. The ROM 12 stores, for example, a program for controlling a circuit board of the control device 10.

[0031] The RAM 13 is a volatile memory. The RAM 13 temporarily stores data being processed by the processor 11. The RAM 13 stores various application programs based on instructions from the processor 11. The RAM 13 may also store data necessary for executing the application programs and execution results of the application programs.

[0032] The NVM 14 is a non-volatile memory in which data can be written and rewritten. The NVM 14 is, for example, a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The NVM 14 stores control programs, applications, and various data according to the operational purpose of the control device 10.

[0033] The NVM 14 also stores in advance a pattern setting table 22 (see FIG. 5) showing a plurality of hand washing patterns, a feature model table 23 (see FIG. 6) showing a feature model for determining a hand action class, an action class table 24 (see FIG. 7) showing an action class, and an identification information table 25 (see FIG. 8) showing a hand washing pattern for each user. Each of the pattern setting table 22, the feature model table 23, the action class table 24, and the identification information table 25 may be updated as appropriate according to an operation by an operator. Each of the pattern setting table 22, the feature model table 23, the action class table 24, and the identification information table 25 is not limited to the configuration examples shown in FIGS. 5 to 8 described later. Each table may have a specific configuration added, deleted, or changed as necessary.

[0034] Furthermore, the NVM 14 stores in advance a feature model (classifier) ​​for determining hand motion classes (motion classes "1" to "6" described below). Motion classification using this feature model can be realized, for example, by performing image classification using machine learning such as deep learning. The feature model specifies, for example, parameters of a network (model) obtained by machine learning or the like. For example, the NVM 14 stores the configuration and parameters of a machine learning model as the feature model.

[0035] As an example, the feature model is based on a machine learning model such as a neural network whose parameters are determined by deep learning. As the model, for example, a convolutional neural network (CNN) can be used, but other networks may also be used.

[0036] As an example, the feature model is a machine learning model whose parameters are determined (trained) to output a classification result that classifies a user's action contained in a captured image into one of multiple action classes in response to an input of the captured image.

[0037] The communication I / F 15 is an interface for transmitting and receiving data to and from an external device. The communication I / F 15 connects to the external device via a network, etc. For example, the communication I / F 15 supports a wired or wireless LAN (Local Area Network) connection.

[0038] The camera I / F 16 (image interface) is an interface for transmitting and receiving data to and from the camera 3. For example, the camera I / F 16 transmits a signal to instruct the camera 3 to capture an image based on the control of the processor 11. The camera I / F 16 also acquires a captured image obtained by capturing an image of a target area from the camera 3. For example, the camera I / F 16 may be one that supports a USB (Universal Serial Bus) connection, or one that supports a connection via Camera Link.

[0039] The input / output I / F 18 (notification interface) is an interface for transmitting and receiving data to and from the input / output device 5. For example, the input / output I / F 18 receives a signal indicating an operation accepted from an operator from the input / output device 5. The input / output I / F 18 transmits the received signal to the processor 11. Furthermore, the input / output I / F 18 transmits information indicating a screen to be displayed to the operator to the input / output device 5 under the control of the processor 11. For example, the input / output I / F 18 may be one that supports a USB connection or one that supports a connection via a parallel interface.

[0040] The reader I / F 19 (identification information interface) is an interface for transmitting and receiving data to and from the reader 6. For example, the reader I / F 19 transmits a signal to start the reader 6 under the control of the processor 11. The reader I / F 19 also transmits identification information from the reader 6 to the processor 11. For example, the reader I / F 19 may support a USB connection.

[0041] The control device 10 is realized by using a computer such as a desktop PC (Personal Computer), a notebook PC, a tablet PC, or a smartphone.

[0042] The control device 10 may further have other configurations as necessary in addition to the configuration example of Fig. 3, or a specific configuration may be excluded from the configuration example of Fig. 3. In addition, in the control device 10, at least two elements among the communication I / F 15, the camera I / F 16, the input / output I / F 18, and the reader I / F 19 may be integrally configured.

[0043] In the motion determining device 1, at least two of the elements of the camera 3, the input / output device 5, the reader 6, and the control device 10 may be integrally configured.

[0044] Next, the hand washing pattern will be described.

[0045] The hand-washing patterns are related to the user's hand-washing. The hand-washing patterns are predefined patterns used by the action determination device 1 to determine whether the hand-washing procedure is appropriate. Fig. 4 is a diagram showing examples of hand-washing patterns according to the embodiment. Fig. 4 illustrates hand-washing patterns "A", "B" and "C".

[0046] Hand-washing pattern "A" is used to determine the hand-washing procedure in factory "A". Hand-washing pattern "A" indicates that the hand-washing procedure continues for 5 seconds each in the order of action classes "3", "4" and "5". Hand-washing pattern "A" is also used when camera 3 captures the user's hands from above (as in FIG. 1).

[0047] Hand-washing pattern "B", like hand-washing pattern "A", is used to determine the hand-washing procedure in factory "A". Hand-washing pattern "B", like hand-washing pattern "A", indicates that the hand-washing procedure continues for 5 seconds each in the order of action classes "3", "4" and "5". Hand-washing pattern "B" is also used when camera 3 captures the user's hands from the side (left in the example of FIG. 4) (as in FIG. 2).

[0048] Hand-washing pattern "C" is used to determine the hand-washing procedure in factory "I". Hand-washing pattern "C" indicates that the hand-washing procedure is to continue the action classes "3", "4", "5" and "6" in any order for a total of 20 seconds. Hand-washing pattern "C" is also used when camera 3 captures the user's hands from above (as in FIG. 1).

[0049] Next, the pattern setting table 22 will be described.

[0050] The pattern setting table 22 shows a plurality of hand-washing patterns (task patterns), each of which indicates the actions and sequence of hand-washing (a predetermined series of tasks). Fig. 5 is a diagram showing an example of the pattern setting table 22 according to the embodiment. As shown in Fig. 5, the pattern setting table 22 stores a "pattern ID", "action sequence ID", "action class ID", "action duration", "feature model ID" and "total hand-washing time" in association with each other.

[0051] "Pattern ID" is an ID that identifies the hand-washing pattern (default pattern). "Default" indicates the hand-washing pattern when no hand-washing pattern is specified. "A", "B", and "C" indicate hand-washing patterns "A", "B", and "C", respectively.

[0052] The "action order ID" indicates the order of the action class. For example, in the hand-washing pattern "A," the pattern setting table 22 indicates "1" as the "action order ID" corresponding to the action class "3" as the "action class ID." That is, in the hand-washing pattern "A," the pattern setting table 22 indicates "1" as the order of the action class "3."

[0053] Furthermore, in the hand-washing pattern "C," the pattern setting table 22 indicates "0" as the "action order ID" corresponding to each "action class ID." In other words, the pattern setting table 22 indicates that in the hand-washing pattern "C," the action classes "3" to "6" are in no particular order.

[0054] "Action class ID" indicates the action class executed in the hand-washing pattern. Here, "1" to "6" indicate action classes "1" to "6", respectively.

[0055] The "action duration" indicates the duration of the corresponding action class. Here, the "action duration" is an example of a common pass threshold for the corresponding action class. For example, the pattern setting table 22 indicates "5s" as the "action duration" corresponding to the action class "3" as the "action class ID" for the hand-washing pattern "A". That is, the pattern setting table 22 indicates that the action class "3" continues for 5 seconds for the hand-washing pattern "A". For example, the pattern setting table 22 indicates "0" as the "action duration" corresponding to each "action class ID" for the hand-washing pattern "C". The pattern setting table 22 also indicates "20s" as the "action duration" at the end. That is, the pattern setting table 22 indicates that the action classes "3" to "6" continue for a total of 20 seconds for the hand-washing pattern "C".

[0056] The "feature model ID" is an ID that identifies the feature model used to determine the action class. The "feature model ID" corresponds to the direction in which the camera 3 captures the image of the hand. In this case, when the camera 3 captures the image of the hand from above (as in FIG. 1), the "feature model ID" indicates "H1." Also, when the camera 3 captures the image of the hand from the side (as in FIG. 2), the "feature model ID" indicates "H2."

[0057] "Total hand washing time" indicates the upper limit of the time for a user to wash their hands. For example, the pattern setting table indicates "30s" as the "total hand washing time" for hand washing pattern "A." That is, the pattern setting table 22 indicates that in hand washing pattern "A," the upper limit of the time for a user to wash their hands is 30 seconds. In addition, the "total hand washing time" does not have to be set as in hand washing pattern "C."

[0058] Next, the feature quantity model table 23 will be described.

[0059] 6 is a diagram showing an example of a feature amount model table 23 according to the embodiment. As shown in FIG 6, the feature amount model table 23 stores a "feature amount model ID", a "corresponding action class ID", and a "shooting environment" in association with each other.

[0060] The "feature model ID" is an ID for identifying a feature model.

[0061] The "Corresponding Action Class ID" indicates the action class that the feature model can determine. Here, "H1" and "H2" indicate that the feature model can determine the action classes "1" to "8" indicated by "1" to "8."

[0062] "Shooting environment" indicates the shooting environment that the feature model corresponds to. Here, the feature model table 23 indicates that "H1" corresponds to a shooting image shot from above. Also, the feature model table 23 indicates that "H2" corresponds to a shooting image shot from the side (left and right).

[0063] Next, the action class table 24 will be described.

[0064] Fig. 7 is a diagram showing an example of the action class table 24 according to the embodiment. As shown in Fig. 7, the action class table 24 stores an "action class ID", an "action class display name", and an "action class display image" in association with each other.

[0065] The "action class ID" is an ID for identifying an action class.

[0066] The "action class display name" is a character string that indicates an action class. For example, the "action class display name" indicates "whisk well" as the "action class display name" of the action class "3".

[0067] An "action class display image" is an image that indicates an action class. For example, an "action class display image" is composed of an illustration of an action class, an "action class display name", and the like.

[0068] Next, the identification information table 25 will be described.

[0069] The identification information table 25 indicates changes made to the set hand-washing pattern for each user. Fig. 8 is a diagram showing an example of the identification information table 25 according to the embodiment. As shown in Fig. 8, the identification information table 25 stores "identification information", "action class ID", "action duration (common pass threshold)", "correction parameters", "corrected pass threshold", etc. in association with each other.

[0070] The "identification information" is identification information that identifies a user. Here, the "identification information" may indicate a specific user (such as Mr. A) or may indicate a job title (such as a member of the cooking department or a person involved in plating).

[0071] "Action class ID" indicates the action class.

[0072] "Action duration" indicates the common pass threshold for each action included in hand washing (a series of actions), i.e., the duration of the corresponding action class. Here, "action duration" is "S3", "S4", and "S5" when "action class ID" is "3", "4", and "5", respectively.

[0073] The "correction parameter" is a personal parameter indicating the correction rate of the "action duration" for each action class for the user indicated by the "identification information." Here, the "correction parameter" is "r3," "r4," and "r5" when the "action class ID" is "3," "4," and "5," respectively.

[0074] In this example, the correction rate of the "motion duration" is used as the "correction parameter" (individual parameter), but is not limited to this. The correction amount of the "motion duration" may be stored as the "correction parameter". In other words, this individual parameter may be a correction value for calculating the corrected pass threshold, which is the threshold for the duration of each motion, using the common pass threshold.

[0075] The "corrected pass threshold" indicates the corrected duration of the corresponding action class. In other words, the "corrected pass threshold" indicates the corrected pass threshold for each action class for the user indicated by the "identification information." Here, the "corrected pass threshold" based on personal parameters is a threshold for the duration of each action calculated by multiplying the "action duration" and the "correction parameter."

[0076] In addition, in the identification information table 25, the "motion duration" is not a required configuration and may not be stored. In addition, in the identification information table 25, either the "correction parameter" or the "corrected pass threshold" may not be stored. In other words, in the present disclosure, the personal parameter is at least one of the "correction parameter" and the "corrected pass threshold". In other words, this personal parameter may be the corrected pass threshold, which is a threshold for the duration of each motion.

[0077] Next, functions realized by the movement determining device 1 will be described.

[0078] 9 is a diagram showing an example of a functional configuration of the movement determining device 1 according to the embodiment. In the movement determining device 1, a control device 10 (information processing device) realizes functions as an acquisition unit 101, a threshold setting unit 102, a movement determining unit 103, and an output unit 104 by a processor 11 executing a program stored in an internal memory, a ROM 12, an NVM 14, or the like.

[0079] The acquisition unit 101 acquires an execution instruction for an operation threshold setting mode or an operation determination mode input to the input / output device 5 via the input / output I / F 18. For example, an operator inputs or selects a mode to be executed to the input / output device 5. The acquisition unit 101 acquires the operation of the execution instruction via the input / output I / F 18.

[0080] Furthermore, the acquisition unit 101 acquires a hand-washing pattern input to the input / output device 5 via the input / output I / F 18. For example, an operator inputs a hand-washing pattern to the input / output device 5 based on the installation location of the sink 20 and the angle of the camera 3, etc. The acquisition unit 101 acquires the operation via the input / output I / F 18.

[0081] Furthermore, the acquisition unit 101 acquires a feature model based on the hand-washing pattern. When the hand-washing pattern is set, the acquisition unit 101 refers to the pattern setting table 22 and acquires from the NVM 14 a feature model corresponding to the ID of the set hand-washing pattern.

[0082] Furthermore, when the feature quantity model is acquired, the acquisition unit 101 refers to the pattern setting table 22 and acquires the action order ID, the action class ID, and the action duration of the set hand-washing pattern.

[0083] Furthermore, the acquisition unit 101 acquires the identification information from the reader 6 through the reader I / F 19. After the instruction screens 501, 502 (see Figs. 10 and 11) are displayed, the acquisition unit 101 waits until the reader 6 acquires the identification information. The acquisition unit 101 acquires the user's identification information from the reader 6. When the acquisition unit 101 determines whether the acquired identification information is included in the identification information table 25, it acquires from the identification information table 25 the "identification information", "action class ID", "action duration (common pass threshold)", "correction parameters", "corrected pass threshold", etc. corresponding to the acquired identification information.

[0084] Furthermore, the acquisition unit 101 acquires a captured image from the camera 3 via the camera I / F 16 .

[0085] Furthermore, when the determination logic is constructed, the acquiring unit 101 refers to the action class table 24 and acquires an action class display image corresponding to the acquired action class ID.

[0086] The threshold setting unit 102 sets correction parameters in the operational threshold setting mode (see FIGS. 12, 13 and 15).

[0087] As an example, in the action threshold setting mode, the threshold setting unit 102 determines the duration and action time of each hand-washing action of the user based on the captured image from the camera 3.

[0088] Here, the duration of each hand-washing action of the user is, for example, a count value of the number of captured images (number of frames) classified into a target action class, but may be a time measured by a timer. Also, the motion time of each hand-washing action of the user is, for example, a time measured by a timer, but may be the number of captured images (number of frames) acquired. Note that the time measured by the timer may be a continuous time width or a cumulative time width. Similarly, the number of frames may be a continuous number or a cumulative number. That is, in the present disclosure, the duration of each hand-washing action (series of tasks) may be the cumulative number of frames of the captured images for each hand-washing action identified based on the captured images, the consecutive number of frames of the captured images, the cumulative time, or the duration. Note that in the motion determination mode, the duration is equal to the motion time.

[0089] As an example, in the action threshold setting mode, the threshold setting unit 102 stores a parameter based on the ratio between the duration of each determined action and the common pass threshold (action duration) as a personal parameter for the user identified by the identification information in association with the identification information in the NVM 14, i.e., in the identification information table 25. For example, the personal parameter (correction rate) may be a value obtained by dividing the duration of each determined action by the common pass threshold (action duration), or a value obtained by multiplying or adding a predetermined coefficient to that value.

[0090] As an example, in the action threshold setting mode, the threshold setting unit 102 sets a personal parameter when the duration of each determined action reaches a predetermined lower limit. In other words, the threshold setting unit 102 does not set a personal parameter when the duration of each determined action does not reach a predetermined lower limit. The lower limit of the duration may be set for each action class or for each hand-washing pattern. These lower limits are, for example, predetermined and stored in the NVM 14. These lower limits may be stored in the pattern setting table 22 or the identification information table 25.

[0091] The action determination unit 103 sets a hand-washing pattern (default pattern). For example, the action determination unit 103 determines whether the input hand-washing pattern exists in the pattern setting table 22. That is, the action determination unit 103 determines whether the ID of the input hand-washing pattern exists in the pattern setting table 22. If the action determination unit 103 determines that the input hand-washing pattern exists in the pattern setting table 22, it sets the input hand-washing pattern. On the other hand, if the input hand-washing pattern does not exist in the pattern setting table 22, the action determination unit 103 sets a default hand-washing pattern.

[0092] Furthermore, the action determination unit 103 constructs a determination logic for determining whether the hand-washing procedure is appropriate. For example, when the action determination unit 103 acquires the action order ID, the action class ID, and the action duration, the action determination unit 103 constructs a determination logic based on the action order ID, the action class ID, the action duration, etc.

[0093] Furthermore, the action determination unit 103 changes (sets) the hand-washing pattern based on the identification information. When the action determination unit 103 acquires the identification information, it determines whether the acquired identification information is included in the identification information table 25. When the action determination unit 103 acquires the "identification information," "action class ID," "action duration (common pass threshold)," "correction parameters," "corrected pass threshold," and the like, it changes the hand-washing pattern based on the acquired information. When the action determination unit 103 changes the hand-washing pattern, it updates the judgment logic based on the changed hand-washing pattern. That is, the action determination unit 103 acquires personal parameters related to the user based on the identification information and sets them as the corrected pass threshold.

[0094] The action determination unit 103 may identify the user's job title from the acquired identification information. For example, the NVM 14 may store in advance a table in which identification information indicating individual users is associated with job titles. The action determination unit 103 may refer to the table and identify the user's job title (cooking department member, plating worker, etc.) from the acquired identification information. The action determination unit 103 may change the hand-washing pattern based on the identification information table 25 and the identified job title.

[0095] Furthermore, the motion determination unit 103 determines whether the user's hand-washing is in accordance with a hand-washing pattern based on the determination logic. For example, the motion determination unit 103 identifies the user's motion by classifying the user's motion using a feature quantity model (classifier).

[0096] As an example, the movement determination unit 103 identifies a user's movement using a feature model based on a machine learning model whose parameters are determined (learned) so as to output a classification result that classifies the user's movement contained in a captured image into one of a plurality of movement classes in response to an input of the captured image.

[0097] As an example, the motion judging unit 103 classifies the user's motion using a feature quantity model in each of a motion threshold setting mode (see Figs. 12, 13, and 15) and a motion judging mode (see Figs. 12 to 14), thereby identifying the user's motion. Specifically, the motion judging unit 103 executes a motion threshold setting mode in which a common pass threshold is used to set personal parameters for each user. Also, the motion judging unit 103 executes a motion judging mode in which the personal parameters are used to judge whether hand washing (series of tasks) has been performed.

[0098] As an example, the motion determination unit 103 performs motion determination to identify a motion of a user performing hand washing (series of tasks) based on a captured image and determine whether the identified motion of the user is consistent with a hand washing pattern (task pattern). For example, when a captured image is acquired, the motion determination unit 103 waits until the user's hand is detected from the captured image. When the motion determination unit 103 detects the user's hand, it determines a motion class of the hand based on a feature model. When the motion determination unit 103 determines the motion class, it determines whether the motion class set in the first order and the determined motion class are consistent. When it determines that the two are consistent, the motion determination unit 103 measures the time (duration) during which the user's hand is determined to be the motion class. The motion determination unit 103 waits until the measured motion time exceeds the motion duration corresponding to the motion class. That is, the motion determination unit 103 determines the duration of each motion of the user based on the captured image. Furthermore, the motion determination unit 103 determines whether hand washing has been performed appropriately based on the duration of each determined motion and a correction pass threshold based on personal parameters. For example, when the duration of each determined action is equal to or greater than the correction pass threshold, the action determination unit 103 determines that the hand-washing action was performed properly. Note that the action determination unit 103 may determine that the hand-washing procedure was not performed properly when the measured action time does not exceed the action duration even after a predetermined time has elapsed since the detection of the hand (or since the hand action class changed).

[0099] Furthermore, when the instruction screens 501, 502 are updated, the action determination unit 103 determines whether or not there is an action class set in the next order after the action class. If the action determination unit 103 determines that there is an action class set in the next order, it operates as described above for the action class. If there is no action class set in the next order, the action determination unit 103 determines that the user has properly washed their hands.

[0100] Note that the action determination unit 103 measures the time that has elapsed since the hand was detected (or the time that has elapsed since the identification information was acquired). If the time that has elapsed since the hand was detected exceeds the total hand-washing time, the action determination unit 103 determines that the hand-washing procedure has not been performed properly. For example, the action determination unit 103 displays information indicating that the hand-washing procedure has not been performed properly on the instruction screens 501 and 502.

[0101] In the action threshold setting mode (see Figs. 12, 13, and 15), the output unit 104 displays an instruction screen 501 (see Fig. 10) showing the steps of hand-washing on the input / output device 5. In the action determination mode (see Figs. 12 to 14), the output unit 104 displays an instruction screen 502 (see Fig. 11) showing the steps of hand-washing on the input / output device 5. For example, when an action class display image is acquired, the output unit 104 generates instruction screens 501 and 502 in which the action class display images are arranged according to the acquired action order ID. When the output unit 104 generates instruction screens 501 and 502, it outputs display information for displaying the generated instruction screens 501 and 502 to the input / output device 5 via the input / output I / F 18, thereby causing the input / output device 5 to display them.

[0102] Fig. 10 is a diagram showing an example of an instruction screen 501 in the operation threshold setting mode according to the embodiment. Fig. 11 is a diagram showing an example of an instruction screen 502 in the operation determination mode according to the embodiment. The instruction screens 501 and 502 include displays of icons 51, 52, and 54, as shown in Fig. 10 and Fig. 11. Furthermore, the instruction screen 501 includes a display of an icon 55, as shown in Fig. 10.

[0103] Icon 51 displays an action class display image. Icon 51 also displays information indicating whether the action class corresponding to the action class display image has been completed. Here, icon 51 displays icon 53, which is either an "o" indicating that the action class has been completed or an "x" indicating that the action class has not been completed. Icon 51 may also display an order corresponding to the action classes. Instruction screen 50 displays the same number of icons 51 as the number of action classes indicated by the hand-washing pattern.

[0104] The icon 52 indicates the progress of the action classes. Here, the icon 52 displays the number of action classes indicated by the hand-washing pattern and the number of action classes that have been completed.

[0105] The icon 54 is an operation icon for instructing a transition to a top screen from which a hand-washing pattern input screen, an identification information acquisition screen, an instruction screen 50, or the like can be transitioned.

[0106] Icons 55 display the motion duration (common pass threshold) for each motion class.

[0107] In this way, the output unit 104 notifies the user of the target action time for hand washing actions of each action class by displaying an icon 55 indicating the action duration of each action class together with an icon 51 displaying an action class display image on the instruction screen 501 for the action threshold setting mode.

[0108] The output unit 104 may notify the user by, for example, displaying another text message on the instruction screens 501 and 502, or by outputting a notification sound or voice from a speaker provided in the movement determining device 1.

[0109] Note that the instruction screens 501 and 502 may further have configurations as necessary in addition to the configurations exemplified in Figures 10 and 11, and specific configurations may be excluded from the configurations exemplified in Figures 10 and 11. The configurations of the instruction screens 501 and 502 are not limited to specific configurations.

[0110] Furthermore, when the hand-washing pattern is changed, the output unit 104 updates the instruction screens 501 and 502 based on the changed hand-washing pattern.

[0111] Furthermore, when the measured action time elapses the action duration, the output unit 104 updates the instruction screens 501 and 502. For example, the output unit 104 displays information indicating that the action class has been completed on the icon 51 on the instruction screens 501 and 502. That is, the output unit 104 displays a circle icon 53 on the icon 51 corresponding to the action class.

[0112] In addition, when the output unit 104 determines on the instruction screen 502 of the action determination mode that the hand-washing procedure has not been performed properly, it displays an "X" icon 53 on the corresponding icon 51, indicating that the hand-washing procedure has not been performed properly.

[0113] In this way, the output unit 104 outputs display information for displaying, on the instruction screen 501 of the action threshold setting mode, information indicating the result of the action judgment, for example, an icon 55 indicating "○" indicating that the action class has been completed or an action duration (common pass threshold) indicating that the action class has not been completed, on the icon 51 of the corresponding action class.

[0114] Similarly, the output unit 104 outputs display information for displaying, on the instruction screen 502 of the action judgment mode, an icon 53 indicating the result of the action judgment, for example, an "O" indicating that the action class has been completed or an "X" indicating that the action class has not been completed, on the icon 51 of the corresponding action class, as information indicating the result of the action judgment.

[0115] Next, an example of the operation of the movement determining device 1 will be described.

[0116] Fig. 12 is a flowchart showing an example of the flow of processing executed in the movement determining device 1 according to the embodiment. Note that the flow of processing in Fig. 12 is merely an example, and steps can be added, deleted, and the order can be changed as desired.

[0117] First, the processor 11 of the motion determining device 1 determines whether to set a motion threshold based on an execution instruction acquired through the input / output I / F 18 (S1).

[0118] When it is determined that the operation threshold is not to be set (S1: No), the processor 11 determines whether to perform an operation determination based on an execution instruction acquired through the input / output I / F 18 (S2).

[0119] When it is determined that motion judgment is to be performed (S2: Yes), the processor 11 executes the motion judgment mode using the judgment model file 701 and the correction parameters 702 of the common pass threshold for each individual, and sets the correction parameters 702 of the common pass threshold for each individual (S3).

[0120] When it is determined that the movement threshold is to be set (S1: Yes), the processor 11 executes the movement threshold setting mode using the judgment model file 701, and sets the correction parameter 702 of the common pass threshold for each individual (S4). After the movement threshold setting mode, the processor 11 proceeds to the process of S2.

[0121] If it is determined that the motion determination is not to be performed (S2: No), or after the motion determination mode, the processor 11 ends the operation.

[0122] Here, the process in the motion determination mode (the process in S3 in FIG. 12) will be described.

[0123] Figures 13 and 14 are flow charts showing an example of the process flow of the motion determination mode in Figure 12. Note that the process flows in Figures 13 and 14 are merely examples, and steps can be added, deleted, and the order can be changed as desired.

[0124] First, processor 11 of movement determination device 1 inputs a hand-washing pattern through input / output I / F 18 (S11). When the hand-washing pattern is input, processor 11 determines whether the input hand-washing pattern is included in pattern setting table 22 (S12).

[0125] If it is determined that the input hand washing pattern is included in the pattern setting table 22 (S12: Yes), the processor 11 refers to the feature model table 23 and acquires a feature model corresponding to the hand washing pattern (S13).

[0126] When the feature quantity model is acquired, the processor 11 constructs a decision logic (S14). After constructing the decision logic, the processor 11 displays instruction screens 501 and 502 on the input / output device 5 via the input / output I / F 18 (S15).

[0127] When it is determined that the input hand washing pattern is not included in the pattern setting table 22 (S12: No), the processor 11 refers to the feature model table 23 and acquires a default feature model (S16). When the default feature model is acquired, the processor 11 constructs a default determination logic (S17). When the default determination logic is constructed, the processor 11 displays a default instruction screen 502 on the input / output device 5 through the input / output I / F 18 (S18).

[0128] When the instruction screen 502 is displayed on the input / output device 5 (S15) or when the default instruction screen 502 is displayed on the input / output device 5 (S18), the processor 11 determines whether or not identification information has been acquired through the reader 6 (S19).

[0129] If it is determined that the identification information has not been acquired (S19: No), the processor 11 returns to the process of S19.

[0130] When it is determined that the identification information has been acquired (S19: Yes), the processor 11 determines whether to change the hand-washing pattern (S20). That is, the processor 11 determines whether the identification information is included in the identification information table 25.

[0131] When it is determined that the hand-washing pattern is to be changed (S20: Yes), the processor 11 refers to the identification information table 25, changes the hand-washing pattern, and updates the determination logic and the instruction screen 502 based on the changed hand-washing pattern (S21).

[0132] When it is determined that the hand washing pattern is not to be changed (S20: No), or when the instruction screen 502 is updated (S21), the processor 11 acquires the pass threshold corresponding to the first unprocessed action (S22).

[0133] For example, when personal parameters are not set based on the identification information, the processor 11 acquires the action duration corresponding to the first unprocessed action sequence ID as the pass threshold. For example, when personal parameters are set based on the identification information, the processor 11 acquires the corrected pass value as the pass threshold. Note that when correction parameters such as a correction rate or a correction value are set as personal parameters, the processor 11 acquires the corrected pass threshold by calculating it based on the action duration as the common pass threshold.

[0134] After that, the processor 11 judges whether the total hand-washing time has elapsed (S23). If it is judged that the total hand-washing time has not elapsed (S23: No), the processor 11 judges the user's hand motion class based on the captured image and the feature model (S24). After the motion class is judged, the processor 11 judges whether the judged motion class is consistent with a motion class for hand-washing position judgment or a motion class corresponding to the motion sequence ID (S25).

[0135] If it is determined that the two action classes are inconsistent (S25: No), the processor 11 returns to the process of S23.

[0136] If it is determined that both action classes match (S25: Yes), the processor 11 adds up the action times (S26). Here, the action time is equal to the duration. After adding up the action times, the processor 11 determines whether the action time exceeds the action duration (predetermined time) (S27). Here, the action time is, for example, a count value of the number of captured images (number of frames) classified into the target action class, but may also be a time measured by a timer.

[0137] If it is determined that the action time has not exceeded the action duration time (S27: No), the processor 11 returns to the process of S23.

[0138] If it is determined that the action time has exceeded the action duration (S27: Yes), the processor 11 updates the instruction screen 502 (S28). After updating the instruction screen 502, the processor 11 determines whether or not the next action sequence ID exists (S29).

[0139] If it is determined that a next operation sequence ID exists (S29: Yes), the processor 11 updates the first unprocessed operation sequence ID (S30). After updating the first unprocessed operation sequence ID, the processor 11 returns to the process of S22.

[0140] When it is determined that the total hand-washing time has elapsed (S23: Yes) or when it is determined that there is no next action sequence ID (S29: No), the processor 11 displays the determination result as to whether the hand-washing procedure was performed appropriately (S31). For example, when it is determined that the total hand-washing time has elapsed (S23: Yes), the processor 11 displays information indicating that the hand-washing position was not appropriate or information indicating that the hand-washing procedure was not performed appropriately on the instruction screen 502. When it is determined that there is no next action sequence ID (S29: No), the processor 11 displays information indicating that the hand-washing position was appropriate or information indicating that the hand-washing procedure was performed appropriately on the instruction screen 502.

[0141] After displaying the determination result, the processor 11 determines whether or not to end the operation (S32). For example, the processor 11 determines whether or not an operation to end the operation has been input through the input / output device 5.

[0142] When it is determined not to end the operation (S32: No), the processor 11 returns to S19. On the other hand, when it is determined to end the operation (S32: Yes), the processor 11 ends the operation.

[0143] Here, the process of the operational threshold setting mode (the process of S4 in FIG. 12) will be described.

[0144] Figures 13 and 15 are flow charts showing an example of the process flow of the operational threshold setting mode in Figure 12. Note that the process flows in Figures 13 and 15 are merely examples, and steps can be added, deleted, and the order can be changed as desired.

[0145] It should be noted that the processes in S11 to S21 in FIG. 13 are similar in each mode, and therefore the description thereof will be omitted here.

[0146] If it is determined that the hand washing pattern should not be changed (S20: No), or if the instruction screen 501 is updated (S21), the processor 11 obtains the common pass threshold for the action class corresponding to the first unprocessed action (S41).

[0147] Thereafter, processor 11 repeats the processes of S24 to S27 in the same manner as described above. Note that, here, the motion time is equal to or longer than the duration.

[0148] When it is determined that the action time has exceeded the predetermined time (action duration) (S27: Yes), processor 11 determines whether the duration exceeds a lower limit (S42). That is, processor 11 determines whether the duration of the hand-washing action determined to be an action of the action class, when the user has performed the hand-washing action for the action duration, exceeds a lower limit.

[0149] If it is determined that the duration exceeds the lower limit (S42: Yes), the processor 11 calculates a correction parameter based on the ratio of the duration and the movement duration (common pass threshold), and stores at least one of the correction parameter and a corrected pass value based on the correction parameter in the NVM 14 as a personal parameter (S43).

[0150] If it is determined that the duration does not exceed the lower limit (S42: No), or after the personal parameters are saved, the processor 11 updates the instruction screen 501 (S28).

[0151] Thereafter, the processor 11 performs the processes of S29 to S32 in the same manner as described above.

[0152] When the movement classes are in no particular order, the processor 11 may measure the movement time of each movement class regardless of which movement class the hand belongs to.

[0153] In addition, if the processor 11 does not detect the user's hands even after a predetermined time has elapsed since acquiring the identification information, the processor 11 may display information on the instruction screens 501 and 502 indicating that the hand washing procedure has not been performed properly.

[0154] Furthermore, the processor 11 may transmit the determination result of each determination block to an external device via the communication I / F 15.

[0155] Furthermore, the motion determination device 1 may include a distance sensor that measures a distance in a target area including the hand. The processor 11 may determine the motion class of the hand based on the measurement result of the distance sensor. The processor 11 may determine the motion class of the hand based on the measurement result of the distance sensor and the captured image.

[0156] In general, in a system that uses images to identify a person's actions, there are cases where the classification of actions varies depending on the individual differences in the actions performed during a series of tasks, such as washing hands. For example, even if the same action is performed, due to misclassification of actions caused by differences in actions between multiple users, one user may be classified as having performed the action for 4 seconds (duration length) during the execution of the action for 5 seconds (action duration), while another user may only be classified as performing the action for 3 seconds (duration length). Or, due to differences in the sense of time between multiple users, a user may think that he or she performed an action for 5 seconds (action duration), but in reality, it may have been performed for 4 seconds (duration length) or 3 seconds (duration length). In such a case, if the common pass threshold for determining that an action has been performed is 4 seconds, one user will be considered to have washed his or her hands, while another user will be considered not to have washed his or her hands. In such a situation, since both individuals who performed the action believe that they performed the action for 5 seconds, one user may be dissatisfied with the judgment result.

[0157] In this situation, the movement judgment device 1 according to the embodiment is configured to execute a movement threshold setting mode in which personal parameters for each user are set using a common pass threshold, and a movement judgment mode in which the personal parameters are used to judge whether hand washing (series of tasks) has been performed.

[0158] As an example, in the movement threshold setting mode, the movement judgment device 1 is configured to inform the user of a common pass threshold for each movement, judge the duration of each of the user's movements based on the captured image, and store a parameter based on the ratio between the determined duration of each movement and the common pass threshold in NVM 14 in association with the identification information as a personal parameter for the user.

[0159] According to this configuration, since the human motions for a predetermined series of tasks can be standardized and made into a form that is easy to manage, the series of tasks can be evaluated taking into account interpersonal variability. Therefore, according to the above configuration, dissatisfaction with the assessment results caused by interpersonal variability can be eliminated.

[0160] As an example, in the movement judgment mode, the movement judgment device 1 is configured to acquire personal parameters related to the user set as described above based on the identification information, judge the duration of each of the user's movements based on the captured image, judge whether a series of tasks has been performed based on the determined duration of each movement and a corrected pass threshold based on the personal parameters, and output information indicating the judgment result.

[0161] According to this configuration, a series of predetermined tasks can be evaluated taking into account individual variability, and therefore, the above configuration can eliminate dissatisfaction with the assessment results caused by individual variability.

[0162] (Modification) In the above embodiment, the personal parameters are set for the common pass threshold, but the present invention is not limited to this. The personal parameters may be reset depending on the user's proficiency in hand-washing actions, for example.

[0163] Fig. 16 is a diagram showing another example of the identification information table 25 according to the embodiment. Fig. 17 is a diagram for explaining another example of the operational threshold setting mode according to the embodiment.

[0164] The correction parameter may be obtained by multiplying the correction parameter according to the above embodiment by a coefficient k relating to the skill level, as shown in FIG.

[0165] For example, in the action threshold setting mode according to the above embodiment, when a personal parameter is set for the common pass threshold, the value of the coefficient k is set to "1." Here, the corrected pass value corresponds to the duration of the action actually classified as a hand-washing action in the action threshold setting mode.

[0166] Thereafter, as the user becomes proficient at the hand-washing action, the duration (action time) of the action classified as an actual hand-washing action in the action determination mode becomes longer, and the pass rate increases. In this way, when the pass rate reaches a threshold value that is determined in advance and stored in NVM 14 or the like, processor 11 of action determination device 1 in action threshold setting mode sets the value of coefficient k to a value "m" that is greater than the value of coefficient k (=1) that was set at that time. Thereafter, in a similar manner, processor 11 of action determination device 1 increases the value of coefficient k as the pass rate increases.

[0167] According to this configuration, the corrected pass value can be corrected in accordance with the user's proficiency in the action threshold setting mode. Therefore, the user can gradually improve his / her hand washing so as to increase the pass rate each time he / she washes his / her hands based on the judgment result in the action judgment mode. Therefore, according to the above configuration, the user's hand washing can be brought closer to the hand washing defined by the hand washing pattern while suppressing the user's dissatisfaction with the judgment result. In other words, the efficiency and accuracy of the series of tasks can be improved.

[0168] In the above embodiment, the technology according to the present disclosure is applied to hand washing, but the present disclosure is not limited to this. The technology according to the present disclosure can be applied to a predetermined series of tasks. Here, the predetermined series of tasks is, for example, a task that includes actions that are provided with a manual and that are required to be performed continuously for a certain period of time. In addition, the predetermined series of tasks may be a sequence of actions, that is, actions for each step of the tasks. Examples of the series of tasks include hand washing, yoga, calisthenics, manufacturing processes, cooking, and the like.

[0169] According to at least one of the embodiments described above, a predetermined series of tasks can be evaluated while taking into account inter-personal variability.

[0170] In the above-mentioned embodiment, "determining whether it is A" may mean "determining that it is A", or "determining that it is not A", or "determining whether it is A or not".

[0171] Each control program executed by the movement determining device 1 of this embodiment is provided in a state that it is pre-installed in a recording medium such as a ROM.

[0172] Each control program executed by the operation determination device 1 of this embodiment may be configured to be provided by being recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

[0173] Furthermore, each control program executed by the movement determining device 1 of the present embodiment may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, each control program executed by the movement determining device 1 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

[0174] For example, the control program executed by the control device 10 in the motion determination device 1 of this embodiment has a modular configuration including execution units for the above-mentioned functions (e.g., the acquisition unit 101, the threshold setting unit 102, the motion determination unit 103, and the output unit 104). The processor 11 of the control device 10 reads the control program from the recording medium and loads each of the above-mentioned units onto a main storage device such as the RAM 13 of the control device 10. As a result, each of the above-mentioned units is generated on the main storage device.

[0175] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims.

[0176] (Additional Note) (1) an identification information interface for acquiring identification information for identifying a user; A memory for storing a work pattern indicating a predetermined series of work actions, a common pass threshold value for each of the operations included in the series of work actions, and an order of the operations; an image interface for acquiring a captured image of a target area; a notification interface for notifying the user; At least one processor that executes an action threshold setting mode that sets personal parameters for each user using the common pass threshold, and an action determination mode that determines whether the series of tasks has been performed using the personal parameters; An information processing device comprising: (2) The at least one processor, in the operational threshold setting mode, notifying the user of the common pass threshold for each of the actions; determining a duration of each of the user's actions based on the captured image; a parameter based on a ratio between the determined duration length of each of the actions and the common pass threshold is stored in the memory in association with the identification information as a personal parameter related to the user identified by the identification information; The information processing device according to (1) above. (3) In the motion determination mode, the at least one processor obtaining the personal parameters for the user based on the identification information; determining a duration of each of the user's actions based on the captured image; determining whether the series of tasks has been performed based on the determined duration of each of the actions and a corrected pass threshold based on the personal parameters; outputting information indicating the determination result; The information processing device according to (1) or (2) above. (4) The corrected pass threshold based on the personal parameters is a threshold of a duration of each of the movements calculated using the parameters based on the ratio. The information processing device according to (3) above. (5) The individual parameter is a correction value for calculating the corrected pass threshold, which is a threshold of the duration of each of the movements, using the common pass threshold. The information processing device according to (3) above. (6) The at least one processor, in the action threshold setting mode, does not set the personal parameter when the determined duration of each of the actions does not reach a predetermined lower limit value. An information processing device according to any one of (1) to (5) above. (7) The duration of each movement is a cumulative number of frames of the captured images, a consecutive number of frames of the captured images, a cumulative time, or a duration time of each movement included in the series of tasks identified based on the captured images. An information processing device according to any one of (1) to (6) above. (8) an identification information interface for acquiring identification information for identifying a user; A memory for storing a work pattern indicating a predetermined series of work actions, a common pass threshold value for each of the operations included in the series of work actions, and an order of the operations; an image interface for acquiring a captured image of a target area; a notification interface for notifying a user; In an information processing apparatus including at least one processor, a motion threshold setting mode for setting personal parameters for each user using the common pass threshold, and a motion determination mode for determining whether the series of tasks has been performed using the personal parameters; Information processing methods. (9) A motion threshold setting mode is executed in which a personal parameter for each user is set using a common pass threshold for each motion of a predetermined series of tasks, and a motion determination mode is executed in which it is determined whether the series of tasks has been performed using the personal parameter. A program that makes a computer do something. (10) A recording medium (Computer Program Product) having recorded thereon the program described in (9) above, which is executed by a computer. [Explanation of symbols]

[0177] 1. Motion determination device (system) 3. Camera 5 Input / Output Devices (Display Terminals) 6. Leader 10 Control device (information processing device) 11 Processors 12 ROM 13 RAM 14 NVM 15 Communication I / F 16 Camera I / F (Image Interface) 18 Input / Output I / F (notification interface) 19 Reader I / F (Identification Information Interface) 20. Sink 21 Faucet 22 Pattern setting table 23 Feature Model Table 24 Action Class Table 25 Identification Table 101 Acquisition Department 102 Threshold setting unit 103 Operation judgment section 104 Output section [Prior art documents] [Patent documents]

[0178] [Patent Document 1] JP 2020-140678 A

Claims

1. an identification information interface for acquiring identification information for identifying a user; A memory for storing a work pattern indicating a predetermined series of work actions, a common pass threshold value for each of the operations included in the series of work actions, and an order of the operations; an image interface for acquiring a captured image of a target area; a notification interface for notifying the user; at least one processor that executes an action threshold setting mode that sets personal parameters for each user using the common pass threshold, and an action determination mode that determines whether the series of tasks has been performed using the personal parameters; The at least one processor, in the operational threshold setting mode, notifying the user of the common pass threshold for each of the actions; determining a duration of each of the user's actions based on the captured image; a parameter based on a ratio between the determined duration length of each of the actions and the common pass threshold is stored in the memory in association with the identification information as a personal parameter related to the user identified by the identification information; Information processing device.

2. In the motion determination mode, the at least one processor obtaining the personal parameters for the user based on the identification information; determining a duration of each of the user's actions based on the captured image; determining whether the series of tasks has been performed based on the duration of each of the determined actions and a corrected pass threshold based on the personal parameters; outputting information indicating the determination result; The information processing device according to claim 1 .

3. The corrected pass threshold based on the personal parameters is a threshold of a duration of each of the movements calculated using the parameters based on the ratio. The information processing device according to claim 2 .

4. The individual parameter is a correction value for calculating the corrected pass threshold, which is a threshold of the duration of each of the movements, using the common pass threshold. The information processing device according to claim 2 .

5. The at least one processor, in the action threshold setting mode, does not set the personal parameter when the determined duration of each of the actions does not reach a predetermined lower limit value. The information processing device according to claim 1 .

6. A motion threshold setting mode is executed in which a personal parameter for each user is set using a common pass threshold for each motion of a predetermined series of tasks, and a motion determination mode is executed in which it is determined whether the series of tasks has been performed using the personal parameter; In the operation threshold setting mode, notifying the user of the common pass threshold for each of the actions; determining a duration of each of the user's actions based on a captured image of a target area; a parameter based on a ratio between the determined duration of each of the actions and the common pass threshold is stored in a memory as a personal parameter related to the user in association with identification information for identifying the user; A program that makes a computer do something.

Images