Information processing device and program

Abstract

To provide an information processing device which evaluates a series of predetermined operations while considering variation between individuals.SOLUTION: An information processing device comprises: an identification information interface (I / F); a memory: an image I / F; a reporting I / F; and at least one processor. The at least one processor executes an operation threshold setting mode for setting an individual parameter for each user using a common acceptance threshold for each operation included in a series of predetermined operation, and an operation determination mode for determining whether the series of work has been executed using the individual parameters. The at least one processor reports the common acceptance threshold for each operation to a user in the operation threshold setting mode, determines a continuation length for each operation of the user on the basis of an image in which an object region is imaged, and associates a parameter based on a ratio of the continuation length for each operation and the common acceptance threshold with identification information identifying the user as the individual parameter related to the user and stores them in the memory.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus and a program.

Background Art

[0002] A system for determining whether handwashing has been performed in a predetermined procedure is provided. Such a system identifies a user's handwashing operation and determines whether the identified handwashing operation follows a specific handwashing pattern.

[0003] Among such systems for evaluating a predetermined series of operations such as handwashing operations using images, for example, even when multiple users perform the same handwashing operation, due to individual variations, the results of operation classification may differ. Here, individual variations include variations in the sense of time among multiple users and misclassifications in operation classification due to differences in operations among multiple users. Thus, there was a risk that users would be dissatisfied with the determination results if the determination results differed among users or were not correctly determined even though a predetermined handwashing operation had been performed.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to evaluate a predetermined series of operations taking into account individual variations.

Means for Solving the Problems

[0005] The information processing device according to the embodiment includes an identification information interface, a memory, an image interface, a notification interface, and at least one processor. The identification information interface acquires identification information to identify a user. The memory stores a work pattern indicating the operation of a predetermined series of tasks, a common pass threshold for each operation included in the series of tasks, and the order of those operations. The image interface acquires a captured image of a target area. The notification interface provides notification to the user. The at least one processor executes an operation threshold setting mode in which it sets personal parameters for each user using the common pass threshold, and an operation determination mode in which it determines whether the series of tasks has been performed using the personal parameters. In the operation threshold setting mode, the at least one processor notifies the user of the common pass threshold for each operation, determines the duration of each operation for the user based on the captured image, and stores a parameter based on the ratio of the determined duration of each operation to the common pass threshold in the memory as a personal parameter relating to the user identified by the identification information. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 is a schematic diagram showing an example of the configuration of an operation determination device according to an embodiment. [Figure 2] Figure 2 is a schematic diagram showing another example of the configuration of the operation determination device according to the embodiment. [Figure 3] Figure 3 is a block diagram showing an example of the hardware configuration of the operation determination device according to the embodiment. [Figure 4] Figure 4 shows an example of a handwashing pattern according to the embodiment. [Figure 5] Figure 5 shows an example of a pattern setting table according to the embodiment. [Figure 6] Figure 6 shows an example of a feature model table according to the embodiment. [Figure 7]Figure 7 shows an example of an operation class table according to the embodiment. [Figure 8] Figure 8 shows an example of an identification information table according to the embodiment. [Figure 9] Figure 9 shows an example of the functional configuration of the operation determination device according to this embodiment. [Figure 10] Figure 10 shows an example of the instruction screen for the operation threshold setting mode according to the embodiment. [Figure 11] Figure 11 shows an example of the instruction screen for the operation determination mode according to the embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the processing flow performed in the operation determination device according to the embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the processing flow for each mode in Figure 12. [Figure 14] Figure 14 is a flowchart showing an example of the processing flow for the operation determination mode shown in Figure 12. [Figure 15] Figure 15 is a flowchart showing an example of the processing flow in the operating threshold setting mode shown in Figure 12. [Figure 16] Figure 16 shows another example of an identification information table according to the embodiment. [Figure 17] Figure 17 illustrates another example of the operating threshold setting mode according to the embodiment. [Modes for carrying out the invention]

[0007] The following description will refer to the drawings and explain the information processing device, display terminal, system, information processing method, program, and recording medium according to the embodiment.

[0008] The embodiments described below are one example of an information processing device, display terminal, system, information processing method, program, and recording medium, and do not limit their configuration or specifications. The embodiments described below are examples of applications to a system that evaluates the hand-washing action of a person using images.

[0009] The action determination device (system) according to this embodiment is an example of a system for determining whether a user's handwashing was performed according to a predetermined procedure. The action determination device photographs the user's hands in a sink or the like. The action determination device determines the action class of the captured image of the user's hands. For example, the action determination device determines the action class by classifying hand movements based on the area being washed (back of the hand, between fingers, thumb, etc.). Based on the order of the action classes and the duration of the movements, the action determination device determines whether the user's handwashing was performed according to a predetermined procedure.

[0010] Figure 1 is a schematic diagram showing an example of the configuration of the operation determination device 1 according to the embodiment. As shown in Figure 1, the operation determination device 1 includes a camera 3, an input / output device 5 (display terminal), a reader 6, and a sink 20.

[0011] Sink 20 is a basin in which a user washes their hands. Sink 20 has a recessed structure. A drain is formed at the bottom of sink 20. A faucet 21 is installed in sink 20, for example, opposite the drain, so as to be able to discharge water toward the space formed by the recessed structure of sink 20. The faucet 21 discharges water for the user to wash their hands. The discharged water is drained out of the drain of sink 20.

[0012] Camera 3 is installed above the sink 20, for example, facing the space formed by the recessed structure of the sink 20, with the drain side facing downwards as the shooting direction. Camera 3 photographs the area including the user's hand (target area). Camera 3 photographs the user's hand from above. Camera 3 captures a two-dimensional image (captured image). Camera 3 outputs color information (RGB (Red Green Blue) values) for each dot. Camera 3 may also have a light to illuminate the user's hand.

[0013] The input / output device 5 is installed in a position visible to the user. In the example shown in Figure 1, the input / output device 5 is installed directly in front of the user washing their hands at the sink 20.

[0014] The input / output device 5 is an interface that receives instructions input from an operator and displays various information to the operator. The input / output device 5 has an operation unit that receives the input of instructions and a display unit that displays information.

[0015] As an operation of the operation unit, the input / output device 5 transmits a signal indicating the operation received from the operator to a control device 10 described later. For example, the operation unit is configured using a touch panel. Note that the operation unit may have a keyboard or a numeric keypad.

[0016] As an operation of the display unit, the input / output device 5 displays an image from the control device 10. For example, the display unit is configured using a liquid crystal monitor. The display unit may be integrally formed with a touch panel as the operation unit.

[0017] Note that the input / output device 5 may have an output unit that outputs voice or a notification sound from the control device 10 instead of or in addition to the display unit. This output unit is configured using, for example, a speaker.

[0018] The reader 6 is an acquisition device that acquires identification information for identifying a user. For example, the reader 6 acquires identification information (such as an ID) for identifying a user from a terminal (such as an IC card, an RFID (radio frequency identification) tag, etc.) held by the user. Here, it is assumed that the user connects their own terminal to the reader 6. The reader 6 connects to the terminal by wire or wirelessly and acquires the identification information from the terminal. Also, the reader 6 may be one that reads a code such as a barcode or a two-dimensional code. The identification information may be information for identifying individual users or may indicate the user's position (such as a member of the cooking department, a person engaged in serving work, etc.).

[0019] Here, another example of the operation determination device 1 will be described.

[0020] Figure 2 is a schematic diagram showing another configuration example (operation determination device 1') of the operation determination device 1 according to the embodiment. The configuration example in Figure 2 (operation determination device 1') includes a camera 3, input / output device 5, reader 6, and sink 20, similar to the configuration example in Figure 1 (operation determination device 1).

[0021] In the configuration example shown in Figure 2, camera 3 is installed to the side of the sink 20 (to the left in Figure 2). Camera 3 photographs the user's hand from the side. The input / output device 5 is installed to the side of the sink 20. The other configurations are the same as in the configuration example shown in Figure 1 (motion detection device 1), so their explanation is omitted.

[0022] In this embodiment, if there is no distinction between the configuration example in Figure 1 (operation determination device 1) and the configuration example in Figure 2 (operation determination device 1'), it will simply be described as operation determination device 1.

[0023] Next, we will explain the control system of the operation determination device 1.

[0024] Figure 3 is a block diagram showing an example of the hardware configuration of the operation determination device 1 according to the embodiment. As shown in Figure 3, the operation determination device 1 further includes a control device 10. The control device 10, camera 3, input / output device 5, and reader 6 are communicated together.

[0025] The control device 10 (information processing device) includes a processor 11, ROM 12, RAM 13, NVM (Non-volatile memory) 14, communication I / F (interface) 15, camera I / F 16, input / output I / F 18, and reader I / F 19.

[0026] The processor 11 is connected to the ROM 12, RAM 13, NVM 14, communication interface 15, camera interface 16, input / output interface 18, and reader interface 19 in a communication manner. Camera 3 is connected to camera interface 16 in a communication manner. Input / output device 5 is connected to input / output interface 18 in a communication manner. Reader 6 is connected to reader interface 19 in a communication manner.

[0027] The processor 11 controls the operation of the entire control unit 10. The processor 11 may have an internal cache and various interfaces. The processor 11 performs various processes by executing programs pre-stored in the internal cache, ROM 12, or NVM 14.

[0028] Furthermore, some of the various functions realized by the execution of a program by the processor 11 may be realized by hardware circuits. In this case, the processor 11 controls the functions executed by the hardware circuits.

[0029] Furthermore, regarding the various functions realized by the execution of a program by the processor 11, one function may be realized through the cooperation of two or more processors, or two or more functions may be realized by a common processor.

[0030] ROM12 is a non-volatile memory in which control programs and control data are pre-stored. The control programs and control data stored in ROM12 are pre-loaded according to the specifications of the control device 10. For example, ROM12 stores programs that control the circuit board of the control device 10.

[0031] RAM13 is volatile memory. RAM13 temporarily stores data being processed by processor 11. RAM13 stores various application programs based on instructions from processor 11. RAM13 may also store data necessary for the execution of application programs and the execution results of application programs.

[0032] NVM14 is a non-volatile memory that allows data to be written to and rewritten. NVM14 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or flash memory. NVM14 stores control programs, applications, and various data according to the operational purpose of the control device 10.

[0033] Furthermore, the NVM14 pre-stores a pattern setting table 22 (see Figure 5) showing multiple handwashing patterns, a feature model table 23 (see Figure 6) showing a feature model for determining hand movement classes, a movement class table 24 (see Figure 7) showing movement classes, and an identification information table 25 (see Figure 8) showing handwashing patterns for each user. Note that the pattern setting table 22, feature model table 23, movement class table 24, and identification information table 25 may be updated as needed according to operator operations. Also, the pattern setting table 22, feature model table 23, movement class table 24, and identification information table 25 are not limited to the configuration examples shown in Figures 5 to 8 described later. Each table can be modified by adding, deleting, or changing specific configurations as needed.

[0034] Furthermore, NVM14 pre-stores a feature model (classifier) ​​for determining hand movement classes (movement classes "1" to "6" described later). Movement classification using this feature model can be achieved, for example, by performing image classification using machine learning such as deep learning. The feature model defines the parameters of the network (model) obtained by machine learning, for example. For example, NVM14 stores the configuration and parameters of a machine learning model as a feature model.

[0035] For example, a feature model is based on a machine learning model, such as a neural network, whose parameters are determined by deep learning. While a Convolutional Neural Network (CNN) is a suitable model, other networks may also be used.

[0036] As an example, a feature model is a machine learning model whose parameters are determined (learned) to output a classification result that categorizes the user's actions contained in a captured image into one of several action classes, based on the input image.

[0037] Communication I / F15 is an interface for sending and receiving data with external devices. Communication I / F15 connects to external devices via a network or other means. For example, Communication I / F15 supports wired or wireless LAN (Local Area Network) connections.

[0038] The camera interface (I / F16) is an interface for sending and receiving data to and from the camera 3. For example, the camera interface (I / F16) sends a signal to the camera 3 to take a picture based on the control of the processor 11. The camera interface (I / F16) also acquires the captured image obtained from the camera 3 during the capture of the target area. For example, the camera interface (I / F16) may support a USB (Universal Serial Bus) connection or a Camera Link connection.

[0039] The input / output interface 18 (notification interface) is an interface for sending and receiving data with the input / output device 5. For example, the input / output interface 18 receives signals from the input / output device 5 indicating operations received from the operator. The input / output interface 18 transmits the received signals to the processor 11. In addition, based on the control of the processor 11, the input / output interface 18 transmits information to the input / output device 5 indicating the screen to be displayed to the operator. For example, the input / output interface 18 may support USB connection or parallel interface connection.

[0040] The Reader I / F 19 (Identification Information Interface) is an interface for sending and receiving data with the Reader 6. For example, the Reader I / F 19 sends a signal to activate the Reader 6 based on the control of the Processor 11. The Reader I / F 19 also sends identification information from the Reader 6 to the Processor 11. For example, the Reader I / F 19 may support a USB connection.

[0041] The control device 10 is implemented using a computer such as a desktop PC (Personal Computer), notebook PC, tablet PC, or smartphone.

[0042] Furthermore, the control device 10 may have additional configurations as needed in addition to the configuration example shown in Figure 3, or certain configurations may be excluded from the configuration example shown in Figure 3. Also, in the control device 10, at least two of the following elements may be integrated: the communication I / F 15, camera I / F 16, input / output I / F 18, and reader I / F 19.

[0043] In addition, at least two of the elements among the camera 3, input / output device 5, reader 6, and control device 10 may be integrated into the operation determination device 1.

[0044] Next, I will explain the handwashing patterns.

[0045] The handwashing pattern relates to the user's handwashing. The handwashing pattern is a default pattern used by the operation determination device 1 to determine whether the handwashing procedure is appropriate. Figure 4 shows examples of handwashing patterns according to the embodiment. Figure 4 illustrates handwashing patterns "A", "B", and "C".

[0046] Handwashing pattern "A" is used to determine the handwashing procedure in factory "A". Handwashing pattern "A" indicates that the handwashing procedure consists of continuing for 5 seconds each in the order of operation classes "3", "4", and "5". Handwashing pattern "A" is also used when camera 3 photographs the user's hands from above (as in Figure 1).

[0047] Handwashing pattern "B" is used to determine the handwashing procedure in factory "A," similar to handwashing pattern "A." Handwashing pattern "B" indicates that the handwashing procedure consists of continuing for 5 seconds each in the order of action classes "3," "4," and "5," similar to handwashing pattern "A." Handwashing pattern "B" is also used when camera 3 photographs the user's hands from the side (left in the example in Figure 4) (as in Figure 2).

[0048] Handwashing pattern "C" is used to determine the handwashing procedure in factory "I". Handwashing pattern "C" indicates that the handwashing procedure involves performing action classes "3", "4", "5", and "6" in any order for a total of 20 seconds. Handwashing pattern "C" is also used when camera 3 photographs the user's hands from above (as in Figure 1).

[0049] Next, we will explain the pattern setting table 22.

[0050] The pattern setting table 22 shows multiple handwashing patterns (work patterns), each representing the actions and sequence of handwashing (a predetermined series of tasks). Figure 5 shows an example of the pattern setting table 22 according to the embodiment. As shown in Figure 5, the pattern setting table 22 stores the "pattern ID," "operation sequence ID," "operation class ID," "operation duration," "feature model ID," and "total handwashing time" in association with each other.

[0051] "Pattern ID" is an ID that identifies the handwashing pattern (default pattern). "Default" indicates the handwashing pattern used when no handwashing pattern is specified. "A", "B", and "C" refer to handwashing patterns "A", "B", and "C", respectively.

[0052] The "Operation Sequence ID" indicates the order of the operation classes. For example, in the pattern setting table 22, for handwashing pattern "A", the "Operation Sequence ID" corresponding to operation class "3" is "1". In other words, in the handwashing pattern "A", the pattern setting table 22 indicates "1" as the order of operation class "3".

[0053] Furthermore, in the handwashing pattern "C", the pattern setting table 22 indicates "0" as the "operation sequence ID" corresponding to each "operation class ID". In other words, the pattern setting table 22 indicates that in the handwashing pattern "C", each operation class "3" to "6" is in no particular order.

[0054] The "Action Class ID" indicates the action class performed in the handwashing pattern. Here, "1" to "6" represent action classes "1" to "6," respectively.

[0055] "Operation duration" indicates the duration of the corresponding operation class. Here, "Operation duration" is an example of a common pass threshold for the corresponding operation class. For example, in pattern setting table 22, for handwashing pattern "A", the "Operation duration" corresponding to operation class "3" as "Operation class ID" is "5s". That is, pattern setting table 22 indicates that in handwashing pattern "A", operation class "3" is continued for 5 seconds. Also, for example, in handwashing pattern "C", pattern setting table 22 indicates "0" as the "Operation duration" corresponding to each "Operation class ID". Also, pattern setting table 22 indicates "20s" as the final "Operation duration". That is, pattern setting table 22 indicates that in handwashing pattern "C", each operation class "3" to "6" is continued for a total of 20 seconds.

[0056] The "Feature Model ID" is an ID that identifies the feature model used to determine the behavior class. The "Feature Model ID" corresponds to the direction in which camera 3 photographs the hand. Here, when camera 3 photographs the hand from above (as in Figure 1), the "Feature Model ID" is "H1". When camera 3 photographs the hand from the side (as in Figure 2), the "Feature Model ID" is "H2".

[0057] "Total handwashing time" indicates the maximum time a user can spend washing their hands. For example, the pattern setting table shows "30s" as the "Total handwashing time" for handwashing pattern "A". That is, pattern setting table 22 indicates that for handwashing pattern "A", the maximum time a user can spend washing their hands is 30 seconds. Also, "Total handwashing time" does not have to be set as in handwashing pattern "C".

[0058] Next, we will explain the feature model table 23.

[0059] Figure 6 shows an example of a feature model table 23 according to the embodiment. As shown in Figure 6, the feature model table 23 stores the "feature model ID," "corresponding operation class ID," and "shooting environment" in association with each other.

[0060] The "feature model ID" is an ID that identifies the feature model.

[0061] The "Corresponding Behavior Class ID" indicates the behavior class that the feature model can determine. Here, "H1" and "H2" indicate that they can determine behavior classes "1" through "8" as indicated by "1" through "8".

[0062] "Shooting environment" indicates the shooting environment that the feature model corresponds to. Here, Feature Model Table 23 shows that "H1" corresponds to images taken from above. Also, Feature Model Table 23 shows that "H2" corresponds to images taken from the side (left or right).

[0063] Next, we will explain the operation class table 24.

[0064] Figure 7 shows an example of an operation class table 24 according to the embodiment. As shown in Figure 7, the operation class table 24 stores the "operation class ID", "operation class display name", and "operation class display image" in association with each other.

[0065] The "Operation Class ID" is an ID that identifies the operation class.

[0066] The "Action Class Display Name" is a string that indicates the action class. For example, the "Action Class Display Name" for action class "3" indicates "Whip well".

[0067] The "operation class display image" is an image that represents an operation class. For example, the "operation class display image" consists of an illustration of the operation class and the "operation class display name," etc.

[0068] Next, we will explain the identification information table 25.

[0069] The identification information table 25 shows user-specific changes to the configured handwashing pattern. Figure 8 shows an example of the identification information table 25 according to the embodiment. As shown in Figure 8, the identification information table 25 stores the following in association: "identification information", "operation class ID", "operation duration (common pass threshold)", "correction parameter", and "correction pass threshold".

[0070] "Identification information" refers to information that identifies a user. Here, "identification information" may indicate a specific user (e.g., Mr. A) or a job title (e.g., member of the cooking department, person in charge of food preparation).

[0071] The "Operation Class ID" indicates the operation class.

[0072] "Action duration" refers to the common pass threshold for each action included in handwashing (a series of operations), i.e., the duration of the corresponding action class. Here, "Action duration" is "S3," "S4," and "S5" when "Action Class ID" is "3," "4," and "5," respectively.

[0073] The "correction parameter" is a personal parameter that indicates the correction rate for the "operation duration" for each operation class for the user indicated by the "identification information". Here, the "correction parameter" is "r3", "r4", and "r5" when the "operation class ID" is "3", "4", and "5", respectively.

[0074] Here, the correction rate for "operation duration" is used as an example of a "correction parameter" (individual parameter), but this is not the only example. The correction amount for "operation duration" may also be stored as the "correction parameter." In other words, this individual parameter may be a correction value used to calculate the correction pass threshold, which is the threshold for the duration of each operation, using a common pass threshold.

[0075] The "corrected pass threshold" indicates the corrected duration of the corresponding operation class. In other words, the "corrected pass threshold" indicates the corrected pass threshold for each operation class for the user indicated by the "identification information." Here, the "corrected pass threshold" based on individual parameters is the threshold for the duration of each operation, calculated by multiplying the "operation duration" and the "correction parameter."

[0076] Furthermore, in the identification information table 25, "operation duration" is not a required component and may not be stored. Also, in the identification information table 25, either "correction parameter" or "correction pass threshold" may not be stored. In other words, in this disclosure, a personal parameter is at least one of "correction parameter" and "correction pass threshold". In other words, this personal parameter may be a correction pass threshold, which is a threshold for the duration length of each operation.

[0077] Next, we will explain the functions that the operation determination device 1 implements.

[0078] Figure 9 shows an example of the functional configuration of the operation determination device 1 according to the embodiment. In the operation determination device 1, the control device 10 (information processing device) realizes the functions of an acquisition unit 101, a threshold setting unit 102, an operation determination unit 103, and an output unit 104 by having the processor 11 execute a program stored in the internal memory, ROM 12, or NVM 14, etc.

[0079] The acquisition unit 101 acquires execution instructions for operation threshold setting modes and operation determination modes input to the input / output device 5 via the input / output interface 18. For example, an operator inputs or selects a mode to be executed to the input / output device 5. The acquisition unit 101 acquires the operation of said execution instruction via the input / output interface 18.

[0080] Furthermore, the acquisition unit 101 acquires the handwashing pattern input to the input / output device 5 via the input / output interface 18. For example, the operator inputs the handwashing pattern to the input / output device 5 based on the location of the sink 20 and the angle of the camera 3. The acquisition unit 101 acquires this operation via the input / output interface 18.

[0081] Furthermore, the acquisition unit 101 acquires a feature model based on the handwashing pattern. When a handwashing pattern is set, the acquisition unit 101 refers to the pattern setting table 22 and acquires the feature model corresponding to the ID of the set handwashing pattern from the NVM 14.

[0082] Furthermore, once the feature model is acquired, the acquisition unit 101 refers to the pattern setting table 22 to obtain the operation sequence ID, operation class ID, and operation duration of the set handwashing pattern.

[0083] Furthermore, the acquisition unit 101 acquires identification information from the reader 6 via the reader I / F 19. After the instruction screens 501 and 502 (see Figures 10 and 11) are displayed, the acquisition unit 101 waits until the reader 6 acquires the identification information. The acquisition unit 101 acquires the user's identification information from the reader 6. When the acquisition unit 101 determines whether the acquired identification information is included in the identification information table 25, it acquires the "identification information," "operation class ID," "operation duration (common pass threshold)," "correction parameters," and "correction pass threshold" corresponding to the acquired identification information from the identification information table 25.

[0084] Furthermore, the acquisition unit 101 acquires captured images from the camera 3 via the camera I / F 16.

[0085] Furthermore, once the determination logic is established, the acquisition unit 101 refers to the operation class table 24 and acquires an operation class display image corresponding to the acquired operation class ID.

[0086] The threshold setting unit 102 sets the correction parameters in the operating threshold setting mode (see Figures 12, 13, and 15).

[0087] As an example, in the operation threshold setting mode, the threshold setting unit 102 determines the duration and duration of each hand-washing action of the user based on the image captured from the camera 3.

[0088] Here, the duration of each handwashing action by the user is, for example, the count value of the number of captured images (frames) classified into the target action class, but it may also be the time measured by a timer. Similarly, the action time for each handwashing action by the user is, for example, the time measured by a timer, but it may also be the number of captured images (frames) acquired. The time measured by the timer may be a continuous time interval or a cumulative time interval. Likewise, the number of frames may be a continuous number or a cumulative number. In other words, in this disclosure, the duration of each handwashing action (series of operations) may be the cumulative number of captured images for each handwashing action identified based on the captured images, the continuous number of frames of said captured images, the cumulative time, or the duration. In the action determination mode, the duration is equal to the action time.

[0089] As an example, in the operation threshold setting mode, the threshold setting unit 102 stores a parameter based on the ratio of the duration of each determined operation to the common pass threshold (operation duration) as a personal parameter for the user identified by the identification information, in association with the identification information, in the NVM 14, i.e., in the identification information table 25. For example, the personal parameter (correction rate) may be the value obtained by dividing the duration of each determined operation by the common pass threshold (operation duration), or a value obtained by multiplying or adding a predetermined coefficient to that value.

[0090] For example, in the operation threshold setting mode, the threshold setting unit 102 sets personal parameters when the duration of each determined operation reaches a predetermined lower limit. In other words, the threshold setting unit 102 does not set personal parameters if the duration of each determined operation does not reach a predetermined lower limit. The lower limit of the duration may be set for each operation class or for each handwashing pattern. These lower limits may be predetermined and stored in, for example, the NVM 14. These lower limits may also be stored in the pattern setting table 22 or the identification information table 25.

[0091] The operation determination unit 103 sets a handwashing pattern (default pattern). For example, the operation determination unit 103 determines whether the entered handwashing pattern exists in the pattern setting table 22. That is, the operation determination unit 103 determines whether the ID of the entered handwashing pattern exists in the pattern setting table 22. If the operation determination unit 103 determines that the entered handwashing pattern exists in the pattern setting table 22, it sets the entered handwashing pattern. On the other hand, if the operation determination unit 103 does not exist in the pattern setting table 22, it sets a default handwashing pattern.

[0092] Furthermore, the operation determination unit 103 constructs a determination logic for determining whether the handwashing procedure is appropriate. For example, when the operation sequence ID, operation class ID, and operation duration are obtained, the operation determination unit 103 constructs a determination logic based on the operation sequence ID, operation class ID, and operation duration, etc.

[0093] Furthermore, the operation determination unit 103 changes (sets) the handwashing pattern based on the identification information. When the operation determination unit 103 acquires identification information, it determines whether the acquired identification information is included in the identification information table 25. When the operation determination unit 103 acquires "identification information," "operation class ID," "operation duration (common pass threshold)," "correction parameters," and "correction pass threshold," it changes the handwashing pattern based on the acquired information. Also, when the operation determination unit 103 changes the handwashing pattern, it updates the judgment logic based on the changed handwashing pattern. In other words, the operation determination unit 103 acquires personal parameters related to the user based on the identification information and sets them as the correction pass threshold.

[0094] The operation determination unit 103 may also identify the user's job title from the acquired identification information. For example, the NVM 14 may pre-store a table that associates identification information representing individual users with their job titles. The operation determination unit 103 may refer to this table to identify the user's job title (e.g., cooking department member, food preparation worker) from the acquired identification information. The operation determination unit 103 may change the handwashing pattern based on the identification information table 25 and the identified job title.

[0095] Furthermore, the motion determination unit 103 determines whether the user's handwashing follows a handwashing pattern based on the determination logic. For example, the motion determination unit 103 identifies the user's actions by classifying them using a feature model (classifier).

[0096] As an example, the motion determination unit 103 identifies the user's actions using a feature model based on a machine learning model whose parameters have been determined (learned) to output a classification result that classifies the user's actions contained in the captured image into one of several action classes in response to the input image.

[0097] As an example, the action determination unit 103 identifies user actions by classifying them using a feature model in both the action threshold setting mode (see Figures 12, 13, and 15) and the action determination mode (see Figures 12 to 14). Specifically, the action determination unit 103 executes an action threshold setting mode in which it sets individual parameters for each user using a common pass threshold. The action determination unit 103 also executes an action determination mode in which it determines whether handwashing (a series of tasks) has been performed using the individual parameters.

[0098] As an example, the motion determination unit 103 identifies the user's actions (a series of operations) based on the captured image and performs motion determination to determine whether the identified user actions are consistent with a handwashing pattern (work pattern). For example, once an image is acquired, the motion determination unit 103 waits until it detects the user's hand from the image. Once the motion determination unit 103 detects the user's hand, it determines the hand's action class based on a feature model. After determining the action class, the motion determination unit 103 determines whether the determined action class is consistent with the action class initially set in the sequence. If the motion determination unit 103 determines that the two are consistent, it measures the time (duration) that the user's hand is determined to be in that action class. The motion determination unit 103 waits until the measured action time exceeds the action duration corresponding to that action class. In other words, the motion determination unit 103 determines the duration of each user action based on the captured image. Furthermore, the motion determination unit 103 determines whether handwashing was performed appropriately based on the determined duration of each action and a corrected pass threshold based on personal parameters. For example, the motion determination unit 103 determines that the handwashing action was performed correctly if the duration of each determined action is equal to or greater than the correction pass threshold. The motion determination unit 103 may also determine that the handwashing procedure was not performed correctly if the measured action time does not exceed the action duration even after a predetermined time has elapsed since the hand was detected (or since the hand action class changed).

[0099] Furthermore, when instruction screens 501 and 502 are updated, the operation determination unit 103 determines whether there is an operation class set in the next order after the operation class in question. If the operation determination unit 103 determines that there is an operation class set in the next order, it operates for that operation class as described above. If there is no operation class set in the next order, the operation determination unit 103 determines that the user's handwashing was performed properly.

[0100] The operation determination unit 103 measures the time elapsed since detecting a hand (or the time elapsed since acquiring identification information). If the time elapsed since detecting a hand exceeds the total handwashing time, the operation determination unit 103 determines that the handwashing procedure was not performed properly. For example, the operation determination unit 103 displays information indicating that the handwashing procedure was not performed properly on the instruction screens 501 and 502.

[0101] In the operation threshold setting mode (see Figures 12, 13, and 15), the output unit 104 displays an instruction screen 501 (see Figure 10) showing the handwashing procedure on the input / output device 5. In the operation determination mode (see Figures 12 to 14), the output unit 104 displays an instruction screen 502 (see Figure 11) showing the handwashing procedure on the input / output device 5. For example, when the operation class display image is acquired, the output unit 104 generates instruction screens 501 and 502 by arranging the operation class display images according to the acquired operation sequence ID. After generating the instruction screens 501 and 502, the output unit 104 outputs display information for displaying the generated instruction screens 501 and 502 to the input / output device 5 via the input / output I / F 18, thereby displaying them on the input / output device 5.

[0102] Figure 10 shows an example of the instruction screen 501 for the operation threshold setting mode according to the embodiment. Figure 11 shows an example of the instruction screen 502 for the operation determination mode according to the embodiment. Instruction screens 501 and 502 include the display of icons 51, 52, and 54, as shown in Figures 10 and 11. In addition, instruction screen 501 includes the display of icon 55, as shown in Figure 10.

[0103] Icon 51 displays an action class display image. Icon 51 also displays information indicating whether the action class corresponding to the action class display image has been completed. Here, icon 51 displays either a "○" indicating that the action class has been completed or an "×" indicating that the action class has not been completed. Icon 51 may also display the order corresponding to the action classes. The instruction screen 50 displays the same number of icons 51 as the number of action classes indicated by the handwashing pattern.

[0104] Icon 52 indicates the progress of the action classes. Here, icon 52 displays the number of action classes represented by the handwashing pattern and the number of completed action classes.

[0105] Icon 54 is an icon used to instruct the user to transition to a top screen, such as the handwashing pattern input screen, the identification information acquisition screen, or the instruction screen 50.

[0106] Icon 55 displays the operating duration (common pass threshold) for each operation class.

[0107] In this way, the output unit 104 informs the user of the target operating time for each handwashing operation for each operation class by displaying an icon 55 indicating the operating duration for each operation class, along with an icon 51 that displays an image of the operation class, on the instruction screen 501 of the operation threshold setting mode.

[0108] The output unit 104 may also provide notifications to the user, for example, by displaying other text messages on the instruction screens 501 and 502, or by outputting notification sounds or voices through a speaker provided in the operation determination device 1.

[0109] Note that instruction screens 501 and 502 may have additional configurations as needed, in addition to the configurations illustrated in Figures 10 and 11, or certain configurations may be excluded from the configurations illustrated in Figures 10 and 11. The configurations of instruction screens 501 and 502 are not limited to any specific configuration.

[0110] Furthermore, when the handwashing pattern is changed, the output unit 104 updates the instruction screens 501 and 502 based on the changed handwashing pattern.

[0111] Furthermore, the output unit 104 updates the instruction screens 501 and 502 when the measured operation time exceeds the operation duration. For example, the output unit 104 displays information indicating that the operation class has been completed on the icon 51 of the instruction screens 501 and 502. That is, the output unit 104 draws a "○" icon 53 on the icon 51 corresponding to the operation class.

[0112] Furthermore, if the output unit 104 determines that the handwashing procedure was not performed correctly on the instruction screen 502 of the operation determination mode, it displays an "×" icon 53 on the corresponding icon 51 to indicate that the handwashing procedure was not performed correctly.

[0113] In this manner, the output unit 104 outputs display information for displaying, for example, an icon 55 that displays either "○" indicating that the operation class has been completed or the operation duration (common pass threshold) indicating that the operation class has not been completed, on the corresponding operation class icon 51 in the instruction screen 501 of the operation threshold setting mode, as information indicating the result of the operation judgment.

[0114] Similarly, the output unit 104 outputs display information for displaying, for example, an icon 53 indicating that the operation class has been completed or an icon 51 indicating that the operation class has not been completed, on the corresponding operation class icon 51 in the instruction screen 502 of the operation determination mode, as information indicating the result of the operation determination.

[0115] Next, we will explain an example of the operation of the operation determination device 1.

[0116] Figure 12 is a flowchart showing an example of the processing flow performed in the operation determination device 1 according to the embodiment. Note that the processing flow in Figure 12 is merely an example, and steps can be added, deleted, and their order changed as desired.

[0117] First, the processor 11 of the operation determination device 1 determines whether to set an operation threshold based on the execution instruction obtained through the input / output I / F 18 (S1).

[0118] If it is determined that no operating threshold is set (S1: No), the processor 11 determines whether to perform an operation based on the execution instruction obtained through the input / output interface 18 (S2).

[0119] If it is determined that an operation judgment should be performed (S2:Yes), the processor 11 executes the operation judgment mode using the judgment model file 701 and the correction parameter 702 for the common pass threshold for each individual, and sets the correction parameter 702 for the common pass threshold for each individual (S3).

[0120] If it is determined that an operating threshold should be set (S1: Yes), the processor 11 executes the operating threshold setting mode using the determination model file 701 and sets the correction parameter 702 for the common pass threshold for each individual (S4). After the operating threshold setting mode, the processor 11 proceeds to the process in S2.

[0121] If it is determined that no operation check is performed (S2: No), or after the operation check mode, the processor 11 terminates its operation.

[0122] Here, we will explain the processing in the operation determination mode (processing S3 in Figure 12).

[0123] Figures 13 and 14 are flowcharts illustrating an example of the processing flow for the operation determination mode shown in Figure 12. Note that the processing flow in Figures 13 and 14 is merely an example, and steps can be added, deleted, and their order changed as desired.

[0124] First, the processor 11 of the operation determination device 1 receives a handwashing pattern via the input / output interface 18 (S11). Upon receiving the handwashing pattern, the processor 11 determines whether the input handwashing pattern is included in the pattern setting table 22 (S12).

[0125] If the processor determines that the input handwashing pattern is included in the pattern setting table 22 (S12: Yes), the processor 11 refers to the feature model table 23 and obtains the feature model corresponding to the handwashing pattern (S13).

[0126] Once the feature model is obtained, the processor 11 constructs the decision logic (S14). After constructing the decision logic, the processor 11 displays instruction screens 501 and 502 on the input / output device 5 via the input / output interface 18 (S15).

[0127] If the processor determines that the input handwashing pattern is not included in the pattern setting table 22 (S12: No), the processor 11 refers to the feature model table 23 and obtains a default feature model (S16). Once the default feature model is obtained, the processor 11 constructs a default decision logic (S17). Once the default decision logic is constructed, the processor 11 displays the default instruction screen 502 to the input / output device 5 via the input / output interface 18 (S18).

[0128] When the instruction screen 502 is displayed on the input / output device 5 (S15) or when the default instruction screen 502 is displayed on the input / output device 5 (S18), the processor 11 determines whether identification information has been obtained through the reader 6 (S19).

[0129] If it is determined that identification information has not been obtained (S19: No), the processor 11 returns to the process of S19.

[0130] If it determines that identification information has been obtained (S19: Yes), the processor 11 determines whether to change the handwashing pattern (S20). That is, the processor 11 determines whether the identification information is included in the identification information table 25.

[0131] If it is determined that the handwashing pattern should be changed (S20: Yes), the processor 11 refers to the identification information table 25 to change the handwashing pattern and updates the determination logic and instruction screen 502 based on the changed handwashing pattern (S21).

[0132] If it is determined that the handwashing pattern will not be changed (S20: No), or if the instruction screen 502 is updated (S21), the processor 11 obtains the pass threshold corresponding to the first unprocessed operation (S22).

[0133] For example, if personal parameters are not set based on identification information, the processor 11 obtains the operation duration corresponding to the first unprocessed operation sequence ID as the pass threshold. For example, if personal parameters are set based on identification information, the processor 11 obtains the corrected pass value as the pass threshold. If correction parameters such as correction rate or correction value are set as personal parameters, the processor 11 obtains the corrected pass threshold by calculating it based on the operation duration as the common pass threshold.

[0134] Subsequently, the processor 11 determines whether the total handwashing time has elapsed (S23). If it determines that the total handwashing time has not elapsed (S23: No), the processor 11 determines the user's hand movement class based on the captured image and feature model (S24). After determining the movement class, the processor 11 determines whether the determined movement class matches the movement class for determining the handwashing position or the movement class corresponding to the movement sequence ID (S25).

[0135] If it is determined that the two operating classes are incompatible (S25: No), the processor 11 returns to the process in S23.

[0136] If the processor determines that both operation classes are compatible (S25: Yes), it adds the operation time (S26). Here, the operation time is equal to the duration. After adding the operation time, the processor determines whether the operation time has exceeded the operation duration (predetermined time) (S27). Here, the operation time is, for example, the count value of the number of captured images (number of frames) classified into the target operation class, but it may also be the time measured by a timer.

[0137] If it is determined that the operating time does not exceed the operating duration (S27: No), the processor 11 returns to the process in S23.

[0138] If it is determined that the operating time has exceeded the operating duration (S27: Yes), the processor 11 updates the instruction screen 502 (S28). After updating the instruction screen 502, the processor 11 determines whether the next operation sequence ID exists (S29).

[0139] If it determines that the next operation sequence ID exists (S29: Yes), the processor 11 updates the first unprocessed operation sequence ID (S30). After updating the first unprocessed operation sequence ID, the processor 11 returns to the process in S22.

[0140] If it is determined that the total handwashing time has elapsed (S23: Yes), or if it is determined that there is no next operation sequence ID (S29: No), the processor 11 displays the result of the determination of whether the handwashing procedure was performed properly (S31). For example, if it is determined that the total handwashing time has elapsed (S23: Yes), the processor 11 displays on the instruction screen 502 information indicating that the handwashing position was not appropriate or that the handwashing procedure was not performed properly. Also, if it is determined that there is no next operation sequence ID (S29: No), the processor 11 displays on the instruction screen 502 information indicating that the handwashing position was appropriate or that the handwashing procedure was performed properly.

[0141] When the determination result is displayed, the processor 11 determines whether to terminate the operation (S32). For example, the processor 11 determines whether an operation to terminate the operation has been input through the input / output device 5.

[0142] If it is determined that the operation should not be terminated (S32: No), the processor 11 returns to S19. On the other hand, if it is determined that the operation should be terminated (S32: Yes), the processor 11 terminates the operation.

[0143] Here, we will explain the processing in the operating threshold setting mode (processing S4 in Figure 12).

[0144] Figures 13 and 15 are flowcharts illustrating an example of the processing flow for the operating threshold setting mode shown in Figure 12. Note that the processing flow in Figures 13 and 15 is merely an example, and steps can be added, deleted, and reordered as desired.

[0145] Note that the processes S11 to S21 in Figure 13 are the same in each mode, so their explanation is omitted here.

[0146] If it is determined that the handwashing pattern will not be changed (S20: No), or if the instruction screen 501 is updated (S21), the processor 11 obtains a common pass threshold for the operation class corresponding to the first unprocessed operation (S41).

[0147] Subsequently, processor 11 repeats the processing of S24 to S27 in the same manner as described above. Note that the operation time here is greater than or equal to the duration.

[0148] If it is determined that the operation time has exceeded a predetermined time (operation duration) (S27: Yes), the processor 11 determines whether the duration exceeds a lower limit (S42). In other words, the processor 11 determines whether the duration of the hand-washing operation, which was determined to be an operation of that operation class, exceeds a lower limit, assuming that the user has performed the hand-washing operation for the operation duration.

[0149] If the processor determines that the duration exceeds the lower limit (S42: Yes), the processor 11 calculates a correction parameter based on the ratio of the duration and the operating duration (common pass threshold), and stores at least one of the correction parameter and the corrected pass value based on the correction parameter as an individual parameter in the NVM 14 (S43).

[0150] If it is determined that the duration does not exceed the lower limit (S42: No), or after the personal parameters have been saved, the processor 11 updates the instruction screen 501 (S28).

[0151] Subsequently, processor 11 performs the processing in S29-S32 in the same manner as described above.

[0152] Furthermore, if the operation classes are not in any particular order, the processor 11 may measure the operation time of each operation class regardless of which operation class the hand is currently using.

[0153] Furthermore, if the processor 11 does not detect the user's hands even after a predetermined time has elapsed since acquiring identification information, it may display information on the instruction screens 501 and 502 indicating that the handwashing procedure was not performed properly.

[0154] Furthermore, the processor 11 may transmit the determination result of each determination block to an external device via the communication interface 15.

[0155] Furthermore, the motion determination device 1 may include a distance sensor that measures the distance in the target area including the hand. The processor 11 may determine the hand motion class based on the measurement result of the distance sensor. Alternatively, the processor 11 may determine the hand motion class based on the measurement result of the distance sensor and the captured image.

[0156] In general, systems that use images to identify human actions sometimes exhibit inconsistencies in action classification due to individual differences in how people perform a series of actions, such as handwashing. For example, even when performing similar actions, misclassification due to differences in actions among multiple users could result in one user classifying an action as having lasted 4 seconds (duration) out of a total of 5 seconds (duration), while another user's action might only be classified as having lasted 3 seconds (duration). Alternatively, variations in time perception among multiple users could lead to a situation where one user believes they performed an action for 5 seconds (duration), but in reality, it lasted 4 seconds (duration) or even 3 seconds (duration). In such cases, if the common passing threshold for considering an action as performed is 4 seconds, one user might be deemed to have performed handwashing, while another user might be deemed not to have performed handwashing. In this situation, since both individuals believe they performed the action for 5 seconds, one user might be dissatisfied with the judgment result.

[0157] In this context, the operation determination device 1 according to the embodiment is configured to perform an operation threshold setting mode in which individual parameters for each user are set using a common pass threshold, and an operation determination mode in which hand washing (a series of operations) is determined using the individual parameters.

[0158] As an example, in the operation threshold setting mode, the operation determination device 1 is configured to notify the user of a common pass threshold for each operation, determine the duration of each operation based on the captured image, and store parameters based on the ratio of the determined duration of each operation to the common pass threshold in the NVM 14 as personal parameters for that user, associated with identification information.

[0159] This configuration allows for the standardization and management of human actions for a predetermined series of tasks, enabling evaluation of those tasks while taking into account individual variability. Therefore, this configuration eliminates dissatisfaction with judgment results caused by individual variability.

[0160] As an example, in the operation determination mode, the operation determination device 1 acquires personal parameters about the user set as described above based on the identification information, determines the duration of each user operation based on the captured image, determines whether a series of operations have been performed based on the determined duration of each operation and the corrected pass threshold based on the personal parameters, and outputs information indicating the determination result.

[0161] This configuration allows for the evaluation of a predetermined series of tasks while taking into account individual variability. Therefore, this configuration can eliminate dissatisfaction with judgment results that arise due to individual variability.

[0162] (modified version) In the above embodiment, an example was given where individual parameters are set relative to a common pass threshold, but this is not limited to this. Individual parameters may be reset, for example, according to the user's proficiency in handwashing.

[0163] Figure 16 shows another example of the identification information table 25 according to the embodiment. Figure 17 is a diagram illustrating another example of the operation threshold setting mode according to the embodiment.

[0164] The correction parameter may be obtained by multiplying the correction parameter according to the above embodiment by a coefficient k related to proficiency, as shown in Figure 16.

[0165] For example, in the operation threshold setting mode according to the above embodiment, when individual parameters are set for a common pass threshold, the value of coefficient k is set to "1". Here, the corrected pass value corresponds to the duration for which the operation is classified as a hand-washing operation in the operation threshold setting mode.

[0166] Subsequently, as the user becomes proficient in the handwashing motion, the duration (operation time) for which an action is classified as an actual handwashing motion in the operation judgment mode increases, and the pass rate rises. When the pass rate reaches a predetermined threshold stored in the NVM14 or the like, the processor 11 of the operation judgment device 1 sets the value of coefficient k to a value "m" that is greater than the value of coefficient k (=1) that was set at that time, in the operation threshold setting mode. Thereafter, the processor 11 of the operation judgment device 1 increases the value of coefficient k in the same manner as the pass rate rises.

[0167] This configuration allows the system to adjust the pass value in the operation threshold setting mode according to the user's proficiency level. Therefore, the user can gradually improve their handwashing technique based on the judgment results in the operation judgment mode, thereby increasing their pass rate each time they wash their hands. Consequently, this configuration helps to reduce user dissatisfaction with the judgment results while bringing the user's handwashing closer to the handwashing pattern defined by the system. In other words, it improves the efficiency and accuracy of the entire process.

[0168] In the embodiments described above, the application of the technology relating to this disclosure to hand washing is used as an example, but the invention is not limited to this. The technology relating to this disclosure can be applied to a predetermined series of tasks. Here, a predetermined series of tasks refers to tasks that include actions that are required to be performed continuously for a certain period of time, for example, by providing a manual. Furthermore, the predetermined series of tasks may also have a defined procedure for the actions, i.e., the actions for each step of the task. Examples of such a series of tasks include, in addition to hand washing, yoga, gymnastics, manufacturing processes, cooking, etc.

[0169] According to at least one embodiment described above, a predetermined series of tasks can be evaluated while taking into account individual variability.

[0170] In the embodiments described above, "determining whether it is A" may mean "determining that it is A," "determining that it is not A," or "determining whether or not it is A."

[0171] The control programs executed by the operation determination device 1 of this embodiment are provided pre-loaded onto a recording medium such as ROM.

[0172] Each control program executed by the operation determination device 1 of this embodiment may be configured to be provided as a file in an installable or executable format, recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0173] Furthermore, the control programs executed by the operation determination device 1 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the control programs executed by the operation determination device 1 of this embodiment may be provided or distributed via a network such as the Internet.

[0174] For example, in the operation determination device 1 of this embodiment, the control program executed by the control device 10 has a modular configuration that includes execution units for each of the functions described above (e.g., acquisition unit 101, threshold setting unit 102, operation determination unit 103, and output unit 104). The processor 11 of the control device 10 reads the control program from the recording medium and loads each of the above units onto the main memory of the control device 10, such as the RAM 13. As a result, each of the above units is generated on the main memory.

[0175] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0176] (Note) (1) An identification information interface for obtaining user identification information, A memory that stores a work pattern indicating a predetermined series of operations, a common pass threshold for each operation included in the series of operations, and the sequence of operations, An image interface for acquiring captured images of the target area, A notification interface for notifying the user, An at least one processor that performs an operation threshold setting mode for setting individual parameters for each user using the common pass threshold, and an operation determination mode for determining whether the series of operations have been performed using the individual parameters, An information processing device equipped with the following features. (2) The at least one processor, in the operating threshold setting mode, The common pass threshold for each operation is notified to the user. Based on the captured image, the duration of each of the user's actions is determined. A parameter based on the ratio of the duration of each determined operation to the common pass threshold is stored in the memory as a personal parameter relating to the user identified by the identification information, in association with the identification information. The information processing device described in (1) above. (3) The at least one processor, in the operation determination mode, Based on the identification information, the personal parameters relating to the user are obtained, Based on the captured image, the duration of each of the user's actions is determined. Based on the determined duration of each operation and the corrected pass threshold based on the personal parameters, it is determined whether the series of operations has been performed. Output information showing the judgment result. The information processing device described in (1) or (2) above. (4) The corrected pass threshold based on the aforementioned personal parameters is the threshold for the duration of each operation calculated using the parameters based on the aforementioned ratio. The information processing device described in (3) above. (5) The aforementioned individual parameters are correction values ​​for calculating the corrected pass threshold, which is the threshold for the duration of each operation, using the common pass threshold. The information processing device described in (3) above. (6) In the operation threshold setting mode, if the duration of each operation determined does not reach a predetermined lower limit, the at least one processor does not set the personal parameters. An information processing device as described in any one of the above items (1) to (5). (7) The duration of each operation is the cumulative number of frames of the captured images for each operation included in the series of operations identified based on the captured images, the number of consecutive frames of the captured images, the cumulative time, or the duration. An information processing device as described in any one of the above items (1) to (6). (8) An identification information interface for obtaining user identification information, A memory that stores a work pattern indicating a predetermined series of operations, a common pass threshold for each operation included in the series of operations, and the sequence of operations, An image interface for acquiring captured images of the target area, A notification interface for informing users, An information processing apparatus comprising at least one processor, wherein the at least one processor comprises: The system performs an operation threshold setting mode in which it sets individual parameters for each user using the common pass threshold, and an operation determination mode in which it determines whether the series of operations have been performed using the individual parameters. Information processing methods. (9) The system executes an operation threshold setting mode that sets individual parameters for each user using a common pass threshold for each operation in a predetermined series of tasks, and an operation determination mode that determines whether the series of tasks has been performed using the individual parameters. A program that causes a computer to perform a task. (10) A computer program that is executed by a computer, and on which the program described in (9) above is recorded (Computer Program Product). [Explanation of symbols]

[0177] 1. Operation detection device (system) 3 cameras 5. Input / Output Device (Display Terminal) 6 Leader 10 Control device (information processing device) 11 processors 12 ROM 13 RAM 14 NVM 15 Communication I / F 16. Camera I / F (Image Interface) 18 Input / Output Interface (Notification Interface) 19. Reader I / F (Identification Information Interface) 20 sinks 21 Faucets 22 Pattern Setting Table 23 Feature Model Table 24 Operation Class Table 25 Identification Information Table 101 Acquisition Department 102 Threshold setting section 103 Operation judgment section 104 Output section [Prior art documents] [Patent Documents]

[0178] [Patent Document 1] Japanese Patent Publication No. 2020-140678

Claims

1. An identification information interface for obtaining user identification information, A memory that stores a work pattern indicating a predetermined series of operations, a common pass threshold for each operation included in the series of operations, and the sequence of operations, An image interface for acquiring captured images of the target area, A notification interface for notifying the user, The system includes at least one processor that performs an operation threshold setting mode for setting individual parameters for each user using the common pass threshold, and an operation determination mode for determining whether the series of operations have been performed using the individual parameters. The at least one processor, in the operating threshold setting mode, The common pass threshold for each operation is notified to the user. Based on the captured image, the duration of each of the user's actions is determined. A parameter based on the ratio of the duration of each determined operation to the common pass threshold is stored in the memory as a personal parameter relating to the user identified by the identification information, in association with the identification information. Information processing device.

2. The at least one processor, in the operation determination mode, Based on the identification information, the personal parameters relating to the user are obtained, Based on the captured image, the duration of each of the user's actions is determined. Based on the determined duration of each operation and the corrected pass threshold based on the personal parameters, it is determined whether the series of operations has been performed. Output information showing the judgment result. The information processing apparatus according to claim 1.

3. The corrected pass threshold based on the aforementioned personal parameters is the threshold for the duration of each operation calculated using the parameters based on the aforementioned ratio. The information processing apparatus according to claim 2.

4. The aforementioned individual parameters are correction values ​​for calculating the corrected pass threshold, which is the threshold for the duration of each operation, using the common pass threshold. The information processing apparatus according to claim 2.

5. In the operation threshold setting mode, if the duration of each operation determined does not reach a predetermined lower limit, the at least one processor does not set the personal parameters. An information processing apparatus according to any one of claims 1 to 4.

6. The system executes an operation threshold setting mode in which individual parameters for each user are set using a common pass threshold for each operation in a predetermined series of tasks, and an operation determination mode in which it is determined whether the series of tasks has been performed using the individual parameters. In the aforementioned operating threshold setting mode, The common pass threshold for each operation is notified to the user. Based on the captured image of the target area, the duration of each of the user's actions is determined. A parameter based on the ratio between the duration of each determined operation and the common pass threshold is stored in memory as a personal parameter for the user, associated with identification information that identifies the user. A program that causes a computer to perform a task.

Images