Management device, management method, and program

Abstract

This management device (10) has an operation detection means (11), a correspondence relationship identification means (12), an appropriateness degree calculation means (13), and an output means (14). The operation detection means (11) detects each of a first operation performed by a first operator and a second operation performed by a second operator who is different from the first operator, the first and second operators being included in an image in which a plurality of operators are imaged in a location where prescribed work is carried out. The correspondence relationship identification means (12) identifies a correspondence relationship that includes the time and / or the positions of the first and second operations. The appropriateness degree calculation means (13) calculates the appropriateness of work on the basis of the correspondence relationship. The output means (14) outputs appropriateness degree information that includes the result of calculation.

Description

[Technical Field]

[0001] The present disclosure relates to a management device, a management method, and a computer-readable medium. [Background technology]

[0002] When a plurality of workers work together in a predetermined space such as a construction site, there is a need for a technology for monitoring whether the work is being done appropriately.

[0003] For example, Patent Document 1 discloses a technology that does not accept input from the operation input unit if the detected movement is from the driver's seat side of the vehicle toward the operation input unit, and accepts input if the detected movement is from a seat other than the driver's seat toward the operation input unit.

[0004] Patent document 2 discloses a technology that compares motion information with reference motion information, extracts motion information that meets predetermined conditions, and uses video to display a scene in which a worker is performing the motion indicated by the extracted motion information.

[0005] Patent Document 3 discloses a technique in which it is determined whether a motion command on the user A's side corresponds to a motion command on the user X's side, and an operation corresponding to the motion command is executed. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2020-079011 [Patent Document 2] Japanese Patent Application Publication No. 2019-125023 [Patent Document 3] Japanese Patent Application Laid-Open No. 2006-039917 Summary of the Invention [Problem to be solved by the invention]

[0007] However, because workers on-site perform a variety of different actions, it is difficult to comprehensively manage work aptitude by aggregating various perspectives.In addition, there is a demand for simpler technology to ensure safety in work performed by multiple workers in cooperation with each other.

[0008] In view of the above-mentioned problems, an object of the present disclosure is to provide a management system etc. that can efficiently and easily manage the appropriateness of work performed by multiple workers in collaboration. [Means for solving the problem]

[0009] A management device according to one aspect of the present disclosure includes a motion detection means, a correspondence identification means, an appropriateness calculation means, and an output means. The motion detection means detects a first motion performed by a first worker and a second motion performed by a second worker different from the first worker, which are included in an image of multiple workers photographed at a location where a predetermined task is being performed. The correspondence identification means identifies a correspondence between the first motion and the second motion, which includes at least either the time or the position. The appropriateness calculation means calculates the appropriateness of the task based on the correspondence. The output means outputs appropriateness information including the calculation result.

[0010] In a management method according to one aspect of the present disclosure, a computer executes the following method. The computer detects a first action performed by a first worker and a second action performed by a second worker different from the first worker, both of which are included in an image of multiple workers photographed at a location where a predetermined task is being performed. The computer identifies a correspondence between the first action and the second action, the correspondence including at least one of time and location. The computer calculates the appropriateness of the task based on the correspondence. The computer outputs appropriateness information including the calculation result.

[0011] A computer-readable medium according to one aspect of the present disclosure stores a program that causes a computer to execute the following management method. The computer detects a first action performed by a first worker and a second action performed by a second worker different from the first worker, both of which are included in an image of multiple workers photographed at a location where a predetermined task is being performed. The computer identifies a correspondence between the first action and the second action, the correspondence including at least one of time and location. The computer calculates the appropriateness of the task based on the correspondence. The computer outputs appropriateness information including the calculation result. [Effects of the Invention]

[0012] The present disclosure makes it possible to provide a management system or the like that can efficiently and simply manage the appropriateness of work performed by multiple workers in collaboration. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 2 is a block diagram of a management device according to the first embodiment. [Figure 2] 1 is a flowchart showing a management method according to the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating the overall configuration of a management system according to a second embodiment. [Figure 4] FIG. 10 is a diagram showing skeletal data extracted from image data. [Figure 5] FIG. 10 is a diagram for explaining a registered action database according to the second embodiment. [Figure 6] FIG. 10 is a diagram for explaining a first example of a registration operation according to the second embodiment. [Figure 7] FIG. 10 is a diagram for explaining a second example of the registration operation according to the second embodiment. [Figure 8] FIG. 10 is a diagram for explaining a correspondence database according to the second embodiment. [Figure 9] FIG. 2 is a diagram showing a first example of an image captured by a camera. [Figure 10] FIG. 10 is a first diagram showing skeletal data extracted by the management device. [Figure 11]FIG. 10 is a second diagram showing skeletal data extracted by the management device. [Figure 12] This is a diagram showing a second example of an image captured by a camera with skeletal data superimposed on it. [Figure 13] 10A and 10B are diagrams illustrating examples of rules for positional relationships in correspondence data. [Figure 14] FIG. 10 is a diagram illustrating a second example of the correspondence database. [Figure 15] FIG. 10 is a diagram showing an image according to a second example of the correspondence database. [Figure 16] FIG. 10 is a diagram showing the overall configuration of a management system according to a third embodiment. [Figure 17] FIG. 11 is a diagram illustrating an example of a correspondence relationship database according to the third embodiment. [Figure 18] FIG. 10 is a diagram showing the overall configuration of a management system according to a fourth embodiment. [Figure 19] FIG. 10 is a block diagram of an authentication device according to a fourth embodiment. [Figure 20] 10 is a flowchart showing a management method according to a fourth embodiment. [Figure 21] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0014] The present disclosure will be described below through embodiments, but the disclosure according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations are omitted as necessary.

[0015] <Embodiment 1> First, a first embodiment of the present disclosure will be described. Fig. 1 is a block diagram of a management device 10 according to the first embodiment. The management device 10 shown in Fig. 1 analyzes the posture and movement of a person included in an image captured by a camera installed at a predetermined work site, for example, and calculates the appropriateness of the work, etc., performed by the person, thereby managing the work of the worker.

[0016] The management device 10 mainly comprises a movement detection unit 11, a correspondence relationship identification unit 12, an appropriateness calculation unit 13, and an output unit 14. In this disclosure, "posture" refers to the shape of at least a part of the body, and "movement" refers to the state of taking a predetermined posture over time. "Movement" is not limited to cases where posture changes, but also includes cases where a constant posture is maintained. Therefore, simply referring to "movement" may also include posture.

[0017] The motion detection unit 11 detects motions of workers included in image data of an image captured by a camera of multiple workers at a location where a predetermined task is being performed. In this case, if the image includes multiple workers, the motion detection unit 11 detects the motions of each worker. That is, for example, if the image includes a first worker and a second worker, the motion detection unit 11 detects a first motion performed by the first worker and a second motion performed by the second worker, who is different from the first worker. The image data is image data of multiple consecutive frames capturing a series of motions of workers. The image data is, for example, image data conforming to a predetermined format such as H.264 or H.265. That is, the image data may be a still image or a video.

[0018] The predetermined movement detected by the movement detection unit 11 is estimated from, for example, an image of the body of a person who is a worker extracted from image data. The movement detection unit 11 detects from the image of the person's body that the person is performing a predetermined task. The predetermined task is, for example, a predetermined pattern of work that is likely to be performed at the work site.

[0019] The correspondence identifying unit 12 identifies the correspondence between the first action and the second action described above. The correspondence relates to at least one of time and position. The correspondence identifying unit 12 identifies this correspondence from the image data. That is, for example, the correspondence identifying unit 12 obtains information about the time of the frames of image data related to the first action and the second action, and associates the information about the time with the time of each action.

[0020] The time correspondence between the first and second actions may indicate, for example, that either the first or second action starts or ends first. The time correspondence between the first and second actions may indicate, for example, that the first and second actions start or end simultaneously. Alternatively, the time correspondence between the first and second actions may indicate, for example, the progress of the first and second actions in terms of time. That is, the time correspondence may indicate, for example, the difference between the start times, the difference between the end times, or the difference in the time elapsed from the start time to the end time. In this case, time may be indicated by frames in the image data. Furthermore, the "start" and "end" of the actions described above may refer to the start or end of the action itself, or may indicate that the action detection unit 11 has started or ended detection.

[0021] The correspondence relationship between the positions of the first action and the second action is the positional relationship between the first worker involved in the first action and the second worker involved in the second action detected in the image data. The correspondence relationship identification unit 12 may calculate or refer to the positional relationship by analyzing the angle of view, angle, etc. of the image from a predetermined object or scenery included in the image captured by the camera.

[0022] Note that the positional relationship in the present disclosure may correspond to an actual three-dimensional space in the captured image. The positional relationship may be calculated by estimating a pseudo three-dimensional space in the captured image. The positional relationship may be a positional relationship on a plane in the captured image. The appropriateness calculation unit 13 may calculate or refer to the above-mentioned positional relationship by previously setting the angle of view, angle, etc. of the image captured by the camera.

[0023] The positional relationship may be, for example, the distance between people involved in the detected movements, or may be, for example, a positional relationship between predetermined positions on the body of people involved in the detected movements.

[0024] The appropriateness calculation unit 13 calculates the appropriateness of an activity performed by a person included in an image captured by a camera. When making this calculation, the appropriateness calculation unit 13 refers to the actions detected by the action detection unit 11. When making this calculation, the appropriateness calculation unit 13 also refers to the detected first and second actions and the correspondence relationship. In this way, the appropriateness calculation unit 13 calculates the appropriateness of the activity. The appropriateness may be indicated as a plurality of levels using indices such as a predetermined range of values, a score, or a symbol. The appropriateness may be indicated, for example, by two values, "appropriate" and "inappropriate."

[0025] The output unit 14 outputs appropriateness information including the result of the calculation performed by the appropriateness calculation unit 13. In this case, the appropriateness information may indicate, as a result of the calculation, a score for evaluating the work performed by the first worker and the second worker whose actions were detected. The appropriateness information may indicate whether the work performed by the first worker and the second worker is appropriate or inappropriate. The output unit 14 may output the appropriateness information to, for example, a display device (not shown) included in the management device 10. The output unit 14 may output the appropriateness information to an external device communicatively connected to the management device 10.

[0026] Next, the processing executed by the management device 10 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the management method according to the first embodiment. The flowchart shown in Fig. 2 starts, for example, when the management device 10 acquires image data.

[0027] First, the action detection unit 11 detects a first action performed by a first worker and a second action performed by a second worker different from the first worker, which are included in image data of an image capturing a plurality of workers at a location where a predetermined task is being performed (step S11). When the action detection unit 11 detects a predetermined action performed by a person, it supplies information about the detected action to the correspondence identification unit 12 and the appropriateness calculation unit 13.

[0028] Next, the correspondence specifying unit 12 specifies a correspondence between the first action and the second action, the correspondence including at least one of the time and the position (step S12). The correspondence specifying unit 12 supplies information on the specified correspondence to the appropriateness calculation unit 13.

[0029] Next, the appropriateness calculation unit 13 calculates the appropriateness of the task by referring to the detected first and second actions and the correspondence relationship (step S13). After generating appropriateness information including the calculation result, the appropriateness calculation unit 13 supplies the generated appropriateness information to the output unit 14.

[0030] Next, the output unit 14 outputs the appropriateness information received from the appropriateness calculation unit 13 to a predetermined output destination (step S14). When the output unit 14 outputs the appropriateness information, the management device 10 ends the series of processes.

[0031] In the above process, step S11 and step S12 may be performed in the reverse order, may be performed simultaneously, or may be performed in parallel.

[0032] Although the first embodiment has been described above, the configuration of the management device 10 is not limited to the above. For example, the management device 10 may have a processor and a storage device, which are not shown in the figure. The storage device may include a storage device including a nonvolatile memory such as a flash memory or an SSD (Solid State Drive). In this case, the storage device of the management device 10 stores a computer program (hereinafter simply referred to as a program) for executing the above-described management method. The processor also loads the computer program from the storage device into a buffer memory such as a DRAM (Dynamic Random Access Memory) and executes the program.

[0033] Each component of the management device 10 may be implemented using dedicated hardware. Furthermore, some or all of the components may be implemented using general-purpose or dedicated circuits, processors, or a combination thereof. These may be implemented using a single chip, or multiple chips connected via a bus. Some or all of the components of each device may be implemented using a combination of the above-described circuits and programs. Furthermore, processors such as CPUs (Central Processing Units), GPUs (Graphics Processing Units), and FPGAs (Field-Programmable Gate Arrays) may be used. The description of the configurations described herein may also be applied to other devices or systems described below in this disclosure.

[0034] Furthermore, when some or all of the components of the management device 10 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in which they are connected via a communication network. Furthermore, the functions of the management device 10 may be provided in a SaaS (Software as a Service) format. Furthermore, the above-described method may be stored on a computer-readable medium to cause a computer to execute the above-described method.

[0035] As described above, according to this embodiment, it is possible to provide a management system or the like that can efficiently and simply manage the appropriateness of work performed by a plurality of workers in cooperation with each other.

[0036] <Embodiment 2> Next, a second embodiment of the present disclosure will be described. Fig. 3 is a diagram showing the overall configuration of a management system 2 according to the second embodiment. The management system 2 includes a management device 20 and a camera 100. The management device 20 and the camera 100 are connected to each other via a network N1 so as to be able to communicate with each other.

[0037] The camera 100 may also be referred to as an imaging device. The camera 100 includes an objective lens and an image sensor, and captures images of the work site where it is installed at predetermined intervals. For example, a first worker P11 and a second worker P12 are present at the work site photographed by the camera 100. By photographing the work site, the camera 100 captures at least a portion of the bodies of the first worker P11 and the second worker P12. In the following description, multiple workers may be simply referred to as workers. Furthermore, the first worker P11 and the second worker P12 may be simply referred to as workers.

[0038] Camera 100 generates image data for each captured image and sequentially supplies the data to management device 20 via network N1. The predetermined period may be, for example, but is not limited to, 1 / 15th of a second, 1 / 30th of a second, or 1 / 60th of a second. Camera 100 may have functions such as panning, tilting, and zooming.

[0039] The management device 20 is a computer device with a communication function, such as a personal computer, a tablet PC, a smartphone, etc. The management device 20 includes an image data acquisition unit 201, a display unit 202, an operation reception unit 203, and a storage unit 210 in addition to the configuration described in the first embodiment.

[0040] In this embodiment, the motion detection unit 11 extracts skeletal data of the first worker P11 and the second worker P12 from the image data. More specifically, the motion detection unit 11 detects an image area (body area) of the worker's body from a frame image included in the image data and extracts (e.g., cuts out) it as a body image. The motion detection unit 11 then uses a machine learning-based skeletal estimation technique to extract skeletal data of at least a portion of the person's body based on features such as the person's joints recognized in the body image. The skeletal data is information including "key points," which are characteristic points such as joints, and "bone links," which indicate links between the key points. The motion detection unit 11 may use a skeletal estimation technique such as OpenPose. Note that in this disclosure, the above-mentioned bone links may also be simply referred to as "bones." Bones refer to pseudo-skeleton structures.

[0041] Furthermore, the movement detection unit 11 detects a predetermined posture or movement from the extracted skeletal data of the first worker P11 and the second worker P12. When detecting a posture or movement, the movement detection unit 11 searches for registered movements registered in a registered movement database stored in the storage unit 210 and compares the skeletal data associated with the searched registered movements with the worker's skeletal data. If the worker's skeletal data and the skeletal data associated with the registered movements are similar, the movement detection unit 11 recognizes the skeletal data as a predetermined posture or movement. In other words, if the movement detection unit 11 detects a registered movement similar to the person's skeletal data, it associates the movement associated with the skeletal data with the registered movement and recognizes it as a predetermined posture or movement. In other words, the movement detection unit 11 associates the worker's skeletal data with the registered movement to recognize the type of the worker's movement.

[0042] In the following description, the action performed by the first worker P11 will be referred to as a first action, and the action performed by the second worker P12 will be referred to as a second action. In addition, the first action and the second action may be collectively referred to simply as an action.

[0043] In the similarity determination, the action detection unit 11 detects the first action and the second action by comparing the skeletal data relating to the action of the worker with the skeletal data as the registered action, with respect to the shapes of the elements constituting the skeletal data. That is, the action detection unit 11 detects the first action and the second action by calculating the similarity of the shapes of the elements constituting the skeletal data.

[0044] Skeletal data has pseudo joints or skeletal structures set as its components to indicate the posture of the body. The form of the elements that make up skeletal data can be defined as the relative geometric relationship, such as the position, distance, and angle of other key points or bones when a certain key point or bone is used as the reference. Alternatively, the form of the elements that make up skeletal data can be defined as a single integrated form formed by multiple key points or bones.

[0045] The movement detection unit 11 analyzes whether the relative shapes of the components between the two pieces of skeletal data being compared are similar. At this time, the movement detection unit 11 calculates the similarity between the two pieces of skeletal data. When calculating the similarity, the movement detection unit 11 may calculate the similarity using, for example, feature amounts calculated from the components of the skeletal data.

[0046] In addition, instead of the above similarity, the object of calculation by the movement detection unit 11 may be the similarity between a portion of the extracted skeletal data and skeletal data related to the registered movement, or the similarity between the extracted skeletal data and a portion of the skeletal data related to the registered movement, or the similarity between a portion of the extracted skeletal data and a portion of the skeletal data related to the registered movement.

[0047] The motion detection unit 11 may calculate the above-mentioned similarity directly or indirectly using the skeletal data. For example, the motion detection unit 11 may convert at least a part of the skeletal data into another format and calculate the above-mentioned similarity using the converted data. In this case, the similarity may be the similarity between the converted data itself, or may be a value calculated using the similarity between the converted data.

[0048] The conversion method may be normalization of the image size of the skeletal data, or conversion into a feature using the angle of the skeletal structure (i.e., the degree of bending of the joints), or the conversion method may be a 3D posture converted by a pre-trained machine learning model.

[0049] With the above-described configuration, the motion detection unit 11 in this embodiment detects a first motion and a second motion that are similar to a predetermined registered motion. Alternatively, the motion detection unit 11 detects a predetermined registered motion that is most similar to the detected first motion. Similarly, the motion detection unit 11 detects a predetermined registered motion that is most similar to the detected second motion.

[0050] Alternatively, the motion detection unit 11 calculates the similarity between a motion performed by a worker and a predetermined registered motion. The predetermined registered motion is, for example, information about typical work motions performed by a person at a work site. If the detected motion is similar to the predetermined registered motion, the motion detection unit 11 supplies a signal indicating that this motion is similar to the registered motion to the appropriateness calculation unit 13. In this case, the motion detection unit 11 may supply information about the similar registered motion to the appropriateness calculation unit 13, or may supply the similarity for the similar registered motion.

[0051] As described above, the movement detection unit 11 in this embodiment detects movements from skeletal data relating to the body structure of a person extracted from image data relating to an image including a person. That is, the movement detection unit 11 extracts images of the bodies of the first worker P11 and the second worker P12 from the image data and estimates pseudo-skeleton structures relating to the body structures of the extracted workers. Furthermore, in this case, the movement detection unit 11 detects movements by comparing the skeletal data relating to the movements with skeletal data as registered movements based on the shapes of the elements constituting the skeletal data.

[0052] The movement detection unit 11 may detect posture or movement from skeletal data extracted from a single image data. The movement detection unit 11 may also detect movement from posture changes extracted in time series from multiple image data captured at multiple different times. That is, the movement detection unit 11 detects posture changes of the first worker P11 and the second worker P12 from multiple frames. With this configuration, the management device 20 can flexibly analyze movements according to the state of changes in posture or movement to be detected. In this case, the movement detection unit 11 may also use the registered movement database.

[0053] The correspondence identifying unit 12 in this embodiment may use the skeletal data extracted by the motion detecting unit 11. For example, the correspondence identifying unit 12 may identify the correspondence in terms of position by comparing predetermined positions of the skeletal data of the first worker P11 and the skeletal data of the second worker P12. The correspondence identifying unit 12 may also identify the correspondence in terms of time between the first worker P11 and the second worker P12 from changes in posture of these skeletal data.

[0054] The appropriateness calculation unit 13 in this embodiment calculates the appropriateness by referring to predetermined correspondence data. The appropriateness calculation unit 13 reads a correspondence database stored in the storage unit 210. The correspondence database includes multiple pieces of correspondence data. The correspondence data is data used to calculate the appropriateness of an operation performed by a worker, and includes data on the worker's actions and data on the correspondence. In other words, the appropriateness calculation unit 13 can also be said to calculate the appropriateness based on the types of first and second actions and the correspondence detected by the correspondence identification unit 12. In other words, the appropriateness calculation unit 13 refers to correspondence data corresponding to the types of actions detected by the action detection unit 11 and combinations of actions by multiple workers. The appropriateness calculation unit 13 generates appropriateness information for the operation performed by the worker by referring to the correspondence database. The output unit 14 in this embodiment outputs the appropriateness information generated by the appropriateness calculation unit 13 to the display unit 202.

[0055] In calculating the appropriateness, the appropriateness calculation unit 13 evaluates, for example, the timing and positional relationship of the worker's movements. When evaluating the timing of movements, the appropriateness calculation unit 13 may, for example, compare the difference in timing between predetermined movements with a pre-stored reference value. When evaluating the positional relationship of movements, the appropriateness calculation unit 13 may, for example, store the positional relationship of predetermined skeletal data as a rule and perform evaluation according to the stored rule. Alternatively, the appropriateness calculation unit 13 may have a query including the positional relationships of multiple skeletal data and compare the similarity with the query. The appropriateness calculation unit 13 may also use the above-mentioned methods appropriately depending on the situation.

[0056] The image data acquisition unit 201 is an interface that acquires image data supplied from the camera 100. The image data acquired by the image data acquisition unit 201 includes images captured by the camera 100 at predetermined intervals. The image data acquisition unit 201 supplies the acquired image data to, for example, the motion detection unit 11.

[0057] The display unit 202 is a display including a liquid crystal panel or organic electroluminescence. The display unit 202 displays the appropriateness information output by the output unit 14, and presents the appropriateness of the work performed by the worker to the user of the management device 20.

[0058] The operation reception unit 203 includes information input means such as a keyboard or a touchpad, and receives operations from a user who operates the management device 20. The operation reception unit 203 may be a touch panel that is superimposed on the display unit 202 and configured to operate in conjunction with the display unit 202.

[0059] The storage unit 210 is a storage means including a nonvolatile memory such as a flash memory. The storage unit 210 stores at least a registered action database and a correspondence database. The registered action database includes skeletal data as registered actions. The correspondence database includes multiple pieces of correspondence data. That is, the storage unit 210 stores at least correspondence data relating to the correspondence between a first action performed by the first worker P11 and a second action performed by the second worker P12. The correspondence data includes information indicating combinations of first and second actions that can be detected, and, for such combinations, at least one of a correspondence relationship relating to the time of each action and a correspondence relationship relating to the position of each action. That is, the correspondence data may include different correspondence relationships for each content of the action pattern (combination of first and second actions).

[0060] Next, an example of detecting a person's posture will be described with reference to Fig. 4. Fig. 4 is a diagram showing skeletal data extracted from image data. The image shown in Fig. 4 is a body image F10 in which the body of a first worker P11 is extracted from an image captured by camera 100. In management device 20, movement detection unit 11 cuts out body image F10 from the image captured by camera 100, and further sets a skeletal structure.

[0061] The motion detection unit 11, for example, extracts feature points that can be key points of the first worker P11 from the image. The motion detection unit 11 then detects key points from the extracted feature points. When detecting key points, the motion detection unit 11 refers to, for example, information learned by machine learning about the image of the key points.

[0062] In the example shown in Figure 4, the movement detection unit 11 detects the following key points of the first worker P11: head A1, neck A2, right shoulder A31, left shoulder A32, right elbow A41, left elbow A42, right hand A51, left hand A52, right hip A61, left hip A62, right knee A71, left knee A72, right foot A81, and left foot A82.

[0063] Furthermore, the motion detection unit 11 sets bones connecting these key points as the pseudo skeletal structure of the first worker P11, as shown below: Bone B1 connects the head A1 and neck A2. Bone B21 connects the neck A2 and right shoulder A31, and bone B22 connects the neck A2 and left shoulder A32. Bone B31 connects the right shoulder A31 and right elbow A41, and bone B32 connects the left shoulder A32 and left elbow A42. Bone B41 connects the right elbow A41 and right hand A51, and bone B42 connects the left elbow A42 and left hand A52. Bone B51 connects the neck A2 and right hip A61, and bone B52 connects the neck A2 and left hip A62. Bone B61 connects the right hip A61 to the right knee A71, and bone B62 connects the left hip A62 to the left knee A72. Bone B71 connects the right knee A71 to the right foot A81, and bone B72 connects the left knee A72 to the left foot A82. After generating skeletal data related to the above-mentioned skeletal structure, the movement detection unit 11 uses the generated skeletal data to compare with registered movements.

[0064] Next, an example of the registered action database will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining the registered action database according to the second embodiment. In the table shown in Fig. 5, registered action IDs (identification, identifier) are associated with a plurality of action patterns. For ease of understanding, the action content is shown next to the action pattern. The action pattern for the action with the registered action ID (or action ID) "R01" is "work M11." The action content of the work "M11" is "load lifting action."

[0065] Similarly, the action pattern for the registered action ID "R02" is "Task M12" and the action content is "Work performed on a stepladder." The action pattern for the registered action ID "R03" is "Task M13" and the action content is "Posture supporting a stepladder." Note that the registered action database may contain action patterns of actions that are inappropriate for a specific task.

[0066] As described above, the data on registered actions contained in the registered action database is stored with each action linked to an action ID and an action pattern. Each action pattern is linked to one or more pieces of skeleton data. For example, a registered action with an action ID of "R01" includes skeleton data indicating the action of lifting a specified load.

[0067] Skeleton data related to a registered motion will be described with reference to FIG. 6. FIG. 6 is a diagram for explaining a first example of a registered motion according to the second embodiment. FIG. 6 shows skeletal data related to a motion with a motion ID of "R01" among the registered motions included in the registered motion database. FIG. 6 shows a plurality of skeletal data including skeletal data F11 and skeletal data F12 arranged in the left-right direction. Skeletal data F11 is located to the left of skeletal data F12. Skeletal data F11 is a posture capturing a scene of a person performing a series of luggage lifting motions. Skeletal data F12 is a scene of a person performing a series of luggage lifting motions, and is a different posture from skeletal data F11.

[0068] 6 indicates that in the registered action with action ID "R01", the person takes the posture corresponding to the skeleton data F11 and then the posture corresponding to the skeleton data F12. Note that although two skeleton data have been described here, the registered action with action ID "R01" may include skeleton data other than the above-mentioned skeleton data.

[0069] Fig. 7 is a diagram for explaining a second example of a registered motion according to the second embodiment. Fig. 7 shows skeleton data F21 relating to the motion with motion ID "R02" shown in Fig. 5. For the registered motion with motion ID "R02", only one piece of skeleton data F21 indicating a person working on a stepladder at a work site is registered.

[0070] As described above, the registered motions included in the registered motion database may include only one piece of skeletal data, or may include two or more pieces of skeletal data. The motion detection unit 11 compares the registered motions including the skeletal data with the skeletal data estimated from the image received from the image data acquisition unit 201, and determines whether there are any similar registered motions.

[0071] Next, the correspondence database will be described with reference to FIG. 8. FIG. 8 is a diagram for explaining the correspondence database according to the second embodiment. The table shown in FIG. 8 shows the correspondence database, in which "action patterns" and "correspondences" are arranged in the left-right direction. The "action patterns" include "first actions" and "second actions." The "correspondences" include "time" and "position." The symbols in the table shown in FIG. 8 correspond to those in FIG. 5.

[0072] In the first row of this table, the first action is labeled "Task M11," and the second action is also labeled "Task M11." The same row also displays the time-related correspondence as "Mismatch Time 0±1 max (s)." This indicates that the maximum time mismatch between the first and second actions is zero plus or minus one second. In other words, it is preferable that the first and second actions are essentially synchronized. The same row also displays the position-related correspondence for the distance D10 as "1.5 (m) < distance D10 < 2.5 (m)." This indicates that the preferred correspondence is for the first worker P11 and the second worker P12, who are working together to lift a load, to work at a distance greater than the first threshold of 1.5 meters and less than the second threshold of 2.5 meters.

[0073] That is, the first line indicates that the first worker P11 and the second worker P12 are both performing task M11, that their actions are basically synchronized, and that the distance D10 between the first worker P11 and the second worker P12 is 1.5 to 2.5 meters. The appropriateness calculation unit 13 selects the above-mentioned correspondence data from the actions detected by the first worker P11 and the second worker P12 in the image captured by the camera, and compares it with the correspondence included in the selected correspondence data. The appropriateness calculation unit 13 then calculates the appropriateness from the degree of match or mismatch detected as a result of the comparison.

[0074] 8, the first action is "Task M12," the second action is "Task M13," the time correspondence is "The first action comes after the second action," and the position correspondence is "The first worker is higher than the second worker." This indicates that when the first worker P11 performs "work on a stepladder" as action M12, the second worker P12 takes a "posture supporting a stepladder" as action M13.

[0075] In this case, it is also shown that the first task performed by the first worker P11 is detected later than the second task. Furthermore, it is shown that the position of the first worker P11 working on the stepladder is higher than the position of the second worker P12 supporting the stepladder. The appropriateness calculation unit 13 compares the actions detected from the first worker P11 and the second worker P12 in the image captured by the camera with the above-mentioned correspondence data to calculate the appropriateness.

[0076] Next, the above-mentioned correspondence relationship will be explained with reference to specific image examples. Fig. 9 is a diagram showing a first example of an image captured by a camera. Image F30 shown in Fig. 9 is an image captured by camera 100, and includes a first worker P11, a second worker P12, and a package G11. Image F30 shows a scene in which the first worker P11 and the second worker P12 cooperate to lift the package G11. Here, the suitability of the work performed by the first worker P11 and the second worker P12 is set to be that they should lift the package G11 in coordinated movements so as not to tilt it.

[0077] Fig. 10 is a first diagram showing skeletal data extracted by the management device. Image F30 shown in Fig. 10 is extracted by the motion detection unit 11 from image F30 shown in Fig. 9, and includes a first image F31 and a second image F32. The first image F31 includes a body image and skeletal data of a first worker P11. The second image F32 includes a body image and skeletal data of a second worker P12.

[0078] 10, the correspondence identification unit 12 detects a correspondence between the positions of a first worker P11 in the first image F31 and a second worker P12 in the second image F32. Here, the correspondence identification unit 12 further identifies a first point M11 at the bottom center of the first image F31 and a second point M12 at the bottom center of the second image F32, and then measures a distance D10 between the first point M11 and the second point M12.

[0079] For image F30 in FIG. 10, the correspondence identification unit 12 records the relationship between the actions and time in the first image F31. Similarly, the correspondence identification unit 12 records the relationship between the actions and time in the second image F32. Image F30 shown in FIG. 10 has time T30 indicated in the upper left. The correspondence identification unit 12 records the state of each action at this time T30. Alternatively, the correspondence identification unit 12 may detect a difference between these actions by comparing the state of each action at time T30.

[0080] FIG. 11 is a second diagram showing skeletal data extracted by the management device. Image F40 shown in FIG. 11 includes a first image F41 and a second image F42 extracted from an image taken at time T40, which is after time T30 of image F30 shown in FIG. 10. First image F41 includes a body image and skeletal data of first worker P11. Second image F42 includes a body image and skeletal data of second worker P12. The image shown in FIG. 11 shows that at time T40, which is after time T30, first worker P11 and second worker P12 simultaneously lift both ends of package G11.

[0081] 9 to 11, the action detection unit 11 detects the actions of the first worker P11 and the second worker P12. Then, the correspondence identification unit 12 detects or measures the correspondence between the first action of the first worker P11 and the second action of the second worker P12. Then, the appropriateness calculation unit 13 compares the information detected or measured from FIGS. 10 and 11 with the correspondence database shown in FIG. 8, and calculates the appropriateness from the degree of match or mismatch detected as a result of the comparison.

[0082] Next, a further example of the function realized by the management device 20 will be described with reference to Fig. 12. Fig. 12 is a diagram in which skeletal data is superimposed on a second example of an image captured by a camera. FImage F50 is an image captured by camera 100, showing a first worker P11 performing a predetermined task on a stepladder G12 while a second worker P12 supports the stepladder G12. Image F50 is an image captured by camera 100 with a first image F51 and a second image F52 superimposed on it. The first image F51 includes a body image and skeletal data of the first worker P11. In the first image F51, the first worker P11 is performing task M12 (task performed on a stepladder) shown in FIG. 5. The second image F52 includes a body image and skeletal data of the second worker P12. In the second image F52, the second worker P12 is performing task M13 (posture supporting a stepladder) shown in FIG. 5.

[0083] In the above situation, the correspondence identification unit 12 compares image F50 with the correspondence database shown in FIG. 8. Image F50 corresponds to the example in the second row of the correspondence database shown in FIG. 8. That is, the action detection unit 11 detects task M12 in the first image F51 as the first action, and detects task M13 in the second image F52 as the second action. The correspondence identification unit 12 determines whether or not the second action of the second worker P12 was detected at a time before the first action of the first worker P11 was detected, from an image taken at a time before image F50 shown in FIG. 12.

[0084] The correspondence relationship identifying unit 12 also measures the positional relationship between the first worker P11 performing the first action and the second worker P12 performing the second action. For example, the correspondence relationship identifying unit 12 can measure the difference in height between the first worker P11 and the second worker P12 by comparing a first point M21 shown in the lower center of the first image F51 with a second point M22 shown in the lower center of the second image F52. The correspondence relationship identifying unit 12 may also set a predetermined plane M20 in the image F50 and measure the positions and heights of the workers based on the set plane M20. The first point M21 and the second point M22 described above are merely examples, and the method for comparing the positional relationships of the workers is not limited to the above. Instead of the lower center of the worker's body image, the correspondence relationship identifying unit 12 may set, for example, the upper or central part of the body image, or a position corresponding to a predetermined part of the body, such as the head or waist.

[0085] Next, a further example of the correspondence database will be described with reference to FIG. 13. FIG. 13 is a diagram for explaining an example of rules for positional relationships in correspondence data. FIG. 13 shows the arrangement of elements corresponding to a predetermined pseudo three-dimensional space. FIG. 13 includes a reference plane M30, first skeletal data B11, and second skeletal data B12. The reference plane M30 is a pseudo-set floor surface that can correspond to the plane M20 in FIG. 12. The first skeletal data B11 is skeletal data located above and spaced apart from the reference plane M30. The first skeletal data B11 assumes a posture corresponding to the task M12 (task performed on a stepladder) shown in FIG. 5. The second skeletal data B12 is skeletal data that is on the ground above the reference plane M30. The second skeletal data B12 assumes a posture corresponding to the task M13 (posture supporting a stepladder) shown in FIG. 5.

[0086] FIG. 13 shows one example of correspondence data relating to positions in the correspondence data, and shows a correspondence relating to positions among the correspondences shown in the second row of the correspondence database shown in FIG. 8. That is, the correspondence identification unit 12 compares the data in FIG. 13 with skeletal data identified from an image captured by a camera (e.g., image F50 in FIG. 12). In this way, the rules for correspondence in the correspondence database are not limited to the text rules shown in FIG. 8, but may be set by the arrangement of elements indicating actions as shown in FIG. 13. Furthermore, while FIG. 13 is set for a fixed positional relationship, the positional relationship shown in FIG. 13 may also include postures that change over time.

[0087] Next, a further example of the correspondence database will be described with reference to FIG. 14. FIG. 14 is a diagram illustrating a second example of the correspondence database. The table shown in FIG. 14 differs from the correspondence database shown in FIG. 8 in that the correspondence includes "direction." The correspondence identification unit 12 recognizes the body orientation of the worker from the skeletal data generated by the motion detection unit 11. That is, the "direction" shown in FIG. 14 indicates a correspondence related to the body orientation of the person. For example, when the first worker P11 and the second worker P12 are facing the same direction, the angle is 0 degrees, and when they are facing each other, the angle is 180 degrees. Note that the above-mentioned 0 degrees may be substantially 0 degrees, and may include a tolerance range of, for example, about ±10 degrees.

[0088] 15 is a diagram showing an image according to a second example of the correspondence database. 1 Worker P1 1 The second worker P12 is standing directly in front of the first worker P11. Therefore, the correspondence relationship in terms of the directions between the first worker P11 and the second worker P12 in the image F60 is 180 degrees.

[0089] On the other hand, in the correspondence database shown in Fig. 14, the positional relationship in terms of direction is "90 degrees" according to the correspondence data shown in the second row. That is, the correspondence database sets that when the first worker P11 is performing the task M12 on the stepladder G12, it is appropriate for the second worker P12 to support the stepladder G12 from the side of the first worker P11. Therefore, the appropriateness calculation unit 13 calculates the appropriateness of the task related to the image F60 to be lower than the appropriateness of the task related to the image F50 shown in Fig. 12. Alternatively, the appropriateness calculation unit 13 generates appropriateness information that determines that the task related to the image F60 is inappropriate.

[0090] Although the second embodiment has been described above, the configuration of the management device 20 according to the second embodiment is not limited to the above. The contents of the correspondence database shown in Fig. 8 and Fig. 14 are merely examples, and the items related to the correspondence may include items that a person skilled in the art can conceive of in addition to time, position, and direction. There may be more than one worker, and the number of workers may be three or more.

[0091] The number of cameras 100 included in the management system 2 is not limited to one, and may be multiple. Some of the functions of the motion detection unit 11 may be included in the camera 100. In this case, for example, the camera 100 may extract a body image of a person by processing the captured image. Alternatively, the camera 100 may further extract skeletal data of at least a part of the person's body from the body image based on features of the person's joints, etc., recognized in the body image.

[0092] The management device 20 and the camera 100 may be able to communicate directly without going through the network N1. The management device 20 may include the camera 100. In other words, the management system 2 may be synonymous with the management device 20.

[0093] The second embodiment has been described above. The appropriateness calculation unit 13 may employ various methods for calculating the appropriateness. For example, the appropriateness calculation unit 13 may detect a predetermined first action but not detect a second action corresponding to the first action. In such a case, the appropriateness calculation unit 13 may calculate the appropriateness in the above case to be lower than the appropriateness in the case where both the first action and the second action are detected.

[0094] Furthermore, there are cases where the appropriateness calculation unit 13 detects both the first action and the second action and detects that the positional relationship between the first action and the second action does not satisfy a predetermined condition. In such cases, the appropriateness calculation unit 13 may calculate the above-mentioned appropriateness to be lower than the appropriateness when both the first action and the second action are detected and the positional relationship between the first action and the second action satisfies the predetermined condition.

[0095] The appropriateness calculation unit 13 may detect both the first action and the second action and detect that the time-series relationship between the first action and the second action does not satisfy a predetermined condition. In such a case, the appropriateness calculation unit 13 may calculate the appropriateness to be lower than the appropriateness when both the first action and the second action are detected and the time-series relationship between the first action and the second action satisfies the predetermined condition.

[0096] Furthermore, the number of workers included in an image captured by camera 100 is not limited to two, and may be three or more. In this case, management device 20 may calculate the appropriateness of the correspondence between at least two of the workers included in the image. As described above, according to the second embodiment, a management system or the like can be provided that can efficiently and easily manage the appropriateness of work performed by multiple workers in cooperation with each other.

[0097] <Embodiment 3> Next, a third embodiment will be described. Fig. 16 is a diagram showing the overall configuration of a management system 3 according to the third embodiment. The management system 3 according to the third embodiment includes a management device 30 and a camera 100. The management device 30 differs from the management device 20 according to the second embodiment in that it includes a related image identification unit 15.

[0098] The related image identification unit 15 identifies, from the image data, related images that indicate predetermined objects or areas related to the work. The predetermined related images are preset and may include, for example, luggage carried by the worker or tools used by the worker. The predetermined related images may also be images related to facilities, passageways, and predetermined areas used by the worker.

[0099] The related image specifying unit 15 may specify the related image by recognizing the above-described image from an image captured by a camera. The related image specifying unit 15 may detect the related image by performing a predetermined convolution process on image data including a predetermined object, using a known method such as HOG (Histogram of oriented gradients) or machine learning. The related image specifying unit 15 may also specify a predefined area superimposed on the image captured by the camera.

[0100] In this case, the correspondence specifying unit 12 specifies the positional relationship between the first action, the second action, and the related image. The correspondence specifying unit 12 can also specify the positional relationship between the worker involved in the first action and the second action and the related image over time. This allows the appropriateness calculation unit 13 to calculate the appropriateness taking the related image into consideration.

[0101] Fig. 17 is a diagram showing an example of a correspondence database according to the third embodiment. The table of the correspondence database shown in Fig. 17 differs from the correspondence database shown in Fig. 8 in that an item related to an object is added. In the table of Fig. 17, the object in the first row is shown as "luggage G10". In addition, in the table of Fig. 17, the object in the second row is shown as "stepladder G11".

[0102] 9, the related image specification unit 15 detects the package G11. As a result, the management device 30 identifies the worker who carries the package G11 and the package G1 1 12 and 15, the related image specification unit 15 may detect a stepladder. This allows the management device 30 to recognize the positional relationship between the stepladder and the worker in more detail. This allows the management device 30 to more appropriately calculate the appropriateness of the work.

[0103] With the configuration described above, according to the third embodiment, it is possible to provide a management system or the like that can efficiently and simply manage the appropriateness of work performed by a plurality of workers in cooperation with each other.

[0104] <Embodiment 4> Next, a fourth embodiment will be described with reference to Fig. 18. Fig. 18 is a diagram showing the overall configuration of a management system 4 according to the fourth embodiment. The management system shown in Fig. 18 4 The management system 4 of this embodiment includes a management device 40, a camera 100, an authentication device 300, and a management terminal 400. These components are connected to each other so as to be able to communicate with each other via a network N1. That is, the management system 4 of this embodiment differs from the second embodiment in that it includes a management device 40 instead of the management device 20, and in that it includes an authentication device 300 and a management terminal 400.

[0105] The management device 40 identifies a specific person in cooperation with the authentication device 300, calculates the appropriateness of the work performed by the identified person, and outputs the determination result to the management terminal 400. The management device 40 differs from the management device 20 according to the second embodiment in that it has a person identification unit 16. Also, the management device 40 differs from the management device 20 according to the second embodiment in that the storage unit 210 of the management device 40 stores a person attribute database related to the person to be identified.

[0106] The person identification unit 16 identifies a person included in the image data. The person identification unit 16 identifies a person included in the image captured by the camera 100 by linking the authentication data of the person authenticated by the authentication device 300 with the attribute data stored in the person attribute database.

[0107] In this case, the output unit 14 outputs the appropriateness of the work performed by the identified person to the management terminal 400. If the work performed by the identified person is inappropriate, a warning signal corresponding to the identified person is output to the management terminal 400. That is, the output unit 14 in this embodiment outputs a predetermined warning signal when it is determined that the work performed by the worker is inappropriate.

[0108] The appropriateness calculation unit 13 may have multiple appropriateness levels for determining whether an operation is appropriate. In this case, the output unit 14 outputs a warning signal according to the level. With this configuration, the management device 40 can manage operations more flexibly.

[0109] The person attribute database stored in the storage unit 210 includes attribute data of the identified person. The attribute data includes the person's name, unique identifier, etc. The attribute data may also include data related to the person's work. That is, the attribute data may include, for example, the group to which the person belongs or the type of work the person performs. The attribute data may also include, for example, the person's age or gender as data related to the appropriateness of the work.

[0110] In this embodiment, the action detection unit 11, the related image identification unit 15, and the appropriateness calculation unit 13 may make a determination based on the attribute data of the person. That is, for example, the action detection unit 11 may check registered actions corresponding to the identified person. The related image identification unit 15 may recognize related images corresponding to the identified person. Furthermore, the appropriateness calculation unit 13 may make a determination by referring to correspondence data corresponding to the identified person. With this configuration, the management device 40 can manage tasks customized for the identified person.

[0111] The authentication device 300 is a computer or server device including one or more arithmetic devices. The authentication device 300 authenticates people present at the work site from images captured by the camera 100 and supplies the authentication results to the management device 30. If the person authentication is successful, the authentication device 300 supplies the management device 30 with authentication data linked to the person attribute data stored in the management device 30.

[0112] The management terminal 400 is a dedicated terminal device having a tablet terminal, a smartphone, or a display device, and can receive the suitability information generated by the management device 30 and present the received suitability information to the manager P20. By recognizing the suitability information presented on the management terminal 400 at the work site, the manager P20 can know the status of the work of the workers, the first worker P11 and the second worker P12.

[0113] Next, the configuration of authentication device 300 will be described in detail with reference to Fig. 19. Fig. 19 is a block diagram of authentication device 300. Authentication device 300 authenticates a person by extracting a predetermined feature image from an image captured by camera 100. The feature image is, for example, a facial image. Authentication device 300 has an authentication storage unit 310, a feature image extraction unit 320, a feature point extraction unit 330, a registration unit 340, and an authentication unit 350.

[0114] The authentication storage unit 310 stores a person ID and feature data of the person in association with each other. The feature image extraction unit 320 detects feature areas included in the image acquired by the camera 100 and outputs the detected feature areas to the feature point extraction unit 330. The feature point extraction unit 330 extracts feature points from the feature areas detected by the feature image extraction unit 320 and outputs data related to the feature points to the registration unit 340. The data related to the feature points is a collection of the extracted feature points.

[0115] The registration unit 340 issues a new person ID when registering the feature data. The registration unit 340 associates the issued person ID with the feature data extracted from the registered image and registers them in the authentication storage unit 310. The authentication unit 350 compares the feature data extracted from the feature image with the feature data in the authentication storage unit 310. If the feature data match, the authentication unit 350 determines that authentication is successful, and if the feature data do not match, the authentication unit 350 determines that authentication is unsuccessful. The authentication unit 350 notifies the management device 30 of the success or failure of authentication. Furthermore, if the authentication is successful, the authentication unit 350 identifies the person ID associated with the successful feature data and notifies the management device 30 of the authentication result including the identified person ID.

[0116] The authentication device 300 may authenticate a person using a means other than the camera 100. The authentication may be biometric authentication, or authentication using a mobile terminal, an IC card, or the like.

[0117] The processing performed by the management device 30 in this embodiment will be described with reference to Fig. 20. Fig. 20 is a flowchart showing a management method according to embodiment 4. The flowchart shown in Fig. 20 differs from the flowchart shown in Fig. 2 in the processing after step S13.

[0118] After step S13, the person identification unit 16 identifies the person associated with the appropriateness information from the image data and authentication data (step S21). Next, the output unit 14 outputs the appropriateness information for the identified person to the management terminal 400 (step S22). After outputting the appropriateness information to the management terminal 400, the management device 30 ends the series of processes.

[0119] The method executed by the management device 30 is not limited to the method shown in Fig. 20. The management device 30 may execute step S21 before step S13. Furthermore, the processes from step S11 to step S13 may be performed according to the person identified as described above.

[0120] With the above-described configuration, according to the fourth embodiment, it is possible to provide a management device or the like that can efficiently and simply manage the appropriateness of work performed by a plurality of workers in cooperation with one another.

[0121] <Example of hardware configuration> Hereinafter, a case will be described in which each functional configuration of the determination device according to the present disclosure is realized by a combination of hardware and software.

[0122] FIG. 21 is a block diagram illustrating an example of a hardware configuration of a computer. The management device of the present disclosure can realize the above-described functions by a computer 500 including the hardware configuration shown in the figure. The computer 500 may be a portable computer such as a smartphone or tablet terminal, or a stationary computer such as a PC. The computer 500 may be a dedicated computer designed to realize each device, or may be a general-purpose computer. The computer 500 can realize desired functions by installing a predetermined program.

[0123] The computer 500 has a bus 502, a processor 504, a memory 506, a storage device 508, an input / output interface 510 (an interface is also called an I / F (Interface)), and a network interface 512. The bus 502 is a data transmission path through which the processor 504, the memory 506, the storage device 508, the input / output interface 510, and the network interface 512 transmit and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to a bus connection.

[0124] The processor 504 is one of various processors such as a CPU, a GPU, an FPGA, etc. The memory 506 is a main storage device realized using a RAM (Random Access Memory) or the like.

[0125] The storage device 508 is an auxiliary storage device realized using a hard disk, an SSD, a memory card, a ROM (Read Only Memory), or the like. The storage device 508 stores programs for realizing desired functions. The processor 504 reads the programs into the memory 506 and executes them to realize the respective functional components of each device.

[0126] The input / output interface 510 is an interface for connecting the computer 500 with input / output devices. For example, the input / output interface 510 is connected to an input device such as a keyboard and an output device such as a display device.

[0127] The network interface 512 is an interface for connecting the computer 500 to a network.

[0128] Although an example of a hardware configuration in the present disclosure has been described above, the above-described embodiment is not limited to this. Any processing in the present disclosure can also be realized by causing a processor to execute a computer program.

[0129] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0130] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the invention.

[0131] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) An action detection means for detecting a first action performed by a first worker and a second action performed by a second worker different from the first worker, the first action being included in an image of a plurality of workers photographed at a location where a predetermined task is being performed; a correspondence specifying means for specifying a correspondence between the first action and the second action, the correspondence including at least one of a time and a position; an appropriateness calculation means for calculating an appropriateness of the task based on the correspondence; an output means for outputting appropriateness information regarding the appropriateness; A management device comprising: (Appendix 2) the motion detection means detects the first motion and the second motion that are similar to a predetermined registered motion; 2. The management device of claim 1. (Appendix 3) the action detection means detects the first action and the second action from skeletal data relating to the body structure of the worker extracted from an image including the worker. 3. The management device of claim 2. (Appendix 4) the motion detection means detects the first motion and the second motion by comparing the skeletal data relating to the motion of the worker with the skeletal data as the registered motion based on the shapes of the elements constituting the skeletal data. 4. The management device of claim 3. (Appendix 5) the motion detection means detects a type of motion performed by the worker based on the registered motion; the appropriateness calculation means calculates the appropriateness based on the types of the first action and the second action and the correspondence relationship; The management device according to any one of Supplementary notes 2 to 4. (Appendix 6) the movement detection means detects the first movement and the second movement from posture changes extracted in time series from each of a plurality of images taken at a plurality of different times; 6. The management device according to any one of Supplementary notes 1 to 5. (Appendix 7) further comprising a storage means for storing correspondence data relating to the correspondence between the first action and the second action; the appropriateness calculation means calculates the appropriateness by referring to the correspondence data; 7. The management device according to any one of Supplementary notes 1 to 6. (Appendix 8) the appropriateness calculation means calculates the appropriateness when a predetermined first action is detected and the second action corresponding to the first action is not detected to be lower than the appropriateness when both the first action and the second action are detected. 8. The management device according to any one of Supplementary notes 1 to 7. (Appendix 9) The appropriateness calculation means detects both the first action and the second action, and calculates the appropriateness when a positional relationship between the first action and the second action does not satisfy a predetermined condition, by: detecting both the first motion and the second motion, and calculating the appropriateness to be lower than the appropriateness when a positional relationship between the first motion and the second motion satisfies a predetermined condition; 8. The management device according to any one of Supplementary notes 1 to 7. (Appendix 10) The appropriateness calculation means detects both the first action and the second action, and calculates the appropriateness when a time-series relationship between the first action and the second action does not satisfy a predetermined condition, by: detecting both the first action and the second action, and calculating the appropriateness to be lower than the appropriateness when a time-series relationship between the first action and the second action satisfies a predetermined condition; 8. The management device according to any one of Supplementary notes 1 to 7. (Appendix 11) a related image specifying unit for specifying a related image showing a predetermined object or area related to the work; the correspondence relationship specifying means specifies a positional relationship between the first action, the second action, and the related image; The management device according to any one of Supplementary Notes 1 to 10. (Appendix 12) The output means outputs a predetermined warning signal when the appropriateness is lower than a predetermined threshold. 12. The management device according to any one of Supplementary Notes 1 to 11. (Appendix 13) the output means has a plurality of warning signals corresponding to the appropriateness, and outputs the warning signals corresponding to the appropriateness. 13. The management device of claim 12. (Appendix 14) The image processing system further includes a person identification unit for identifying a person who is the worker and is included in the image. When the appropriateness is lower than a predetermined threshold, the output means outputs the warning signal corresponding to the worker whose appropriateness is low. 14. The management device of claim 12 or 13. (Appendix 15) The computer Detecting a first action performed by a first worker and a second action performed by a second worker different from the first worker, the first action and the second action being included in an image of a plurality of workers photographed at a location where a predetermined task is being performed; identifying a correspondence between the first action and the second action, the correspondence including at least one of a time and a position; calculating the appropriateness of the task based on the correspondence; outputting appropriateness information regarding the calculated appropriateness; Management method. (Appendix 16) Detecting a first action performed by a first worker and a second action performed by a second worker different from the first worker, the first action and the second action being included in an image of a plurality of workers photographed at a location where a predetermined task is being performed; identifying a correspondence between the first action and the second action, the correspondence including at least one of a time and a position; calculating the appropriateness of the task based on the correspondence; outputting appropriateness information regarding the calculated appropriateness; A non-transitory computer-readable medium storing a program that causes a computer to execute a management method. [Explanation of symbols]

[0132] 2 Management System 3 Management System 10 Management device 11 Motion detection unit 12 Correspondence Identification Unit 13 Appropriateness calculation section 14 Output section 15 Related image identification section 16 Person identification section 20 Management device 30 Management device 100 cameras 201 Image data acquisition unit 202 Display section 203 Operation reception section 210 Storage section 300 Authentication Device 310 Authentication storage unit 320 Feature Image Extraction Unit 330 Feature Point Extraction Unit 340 Registration Department 350 Authentication Department 400 Management terminal 500 computers 504 processor 506 memory 508 Storage Devices 510 Input / Output Interface 512 network interface N1 Network P11 1st worker P12 2nd worker

Claims

1. a motion detection means for detecting a first motion performed by a first worker included in an image of a plurality of workers at a location where a predetermined task is performed and a second motion performed by a second worker different from the first worker, the first motion and the second motion being similar to a predetermined registered motion; a correspondence relationship specifying means for specifying a correspondence relationship between the first action and the second action, the correspondence relationship including at least one of a time and a position; an appropriateness calculation means for calculating an appropriateness of the task based on the correspondence; an output means for outputting appropriateness information regarding the appropriateness; A management device comprising:

2. the action detection means detects the first action and the second action from skeletal data relating to a body structure of the worker extracted from an image including the worker. The management device according to claim 1 .

3. the motion detection means detects the first motion and the second motion by comparing the skeletal data relating to the motion of the worker with the skeletal data as the registered motion based on the shapes of the elements constituting the skeletal data. The management device according to claim 2 .

4. the motion detection means detects a type of motion performed by the worker based on the registered motion; the appropriateness calculation means calculates the appropriateness based on the types of the first action and the second action and the correspondence relationship; The management device according to any one of claims 1 to 3.

5. the movement detection means detects the first movement and the second movement from posture changes extracted in time series from each of a plurality of images captured at a plurality of different times; The management device according to any one of claims 1 to 4.

6. further comprising a storage means for storing correspondence data relating to the correspondence between the first action and the second action; the appropriateness calculation means calculates the appropriateness by referring to the correspondence data; The management device according to any one of claims 1 to 5.

7. the appropriateness calculation means calculates the appropriateness when a predetermined first action is detected and the second action corresponding to the first action is not detected to be lower than the appropriateness when both the first action and the second action are detected. The management device according to any one of claims 1 to 6.

8. The appropriateness calculation means detects both the first motion and the second motion, and calculates the appropriateness when the positional relationship between the first motion and the second motion does not satisfy a predetermined condition, by: detecting both the first motion and the second motion, and calculating the appropriateness to be lower than the appropriateness when a positional relationship between the first motion and the second motion satisfies a predetermined condition; The management device according to any one of claims 1 to 7.

9. The computer detecting a first action performed by a first worker and a second action performed by a second worker different from the first worker, the first action and the second action being similar to a predetermined registered action, the first action and the second action being included in an image of a plurality of workers photographed at a location where a predetermined task is performed; identifying a correspondence between the first motion and the second motion, the correspondence including at least one of a time and a position; calculating the appropriateness of the task based on the correspondence; outputting appropriateness information regarding the calculated appropriateness; Management method.

10. Detecting a first action performed by a first worker and a second action performed by a second worker different from the first worker, the first action and the second action being similar to a predetermined registered action, which are included in an image of a plurality of workers at a location where a predetermined task is being performed; identifying a correspondence between the first motion and the second motion, the correspondence including at least one of a time and a position; calculating the appropriateness of the task based on the correspondence; outputting appropriateness information regarding the calculated appropriateness; A program that causes a computer to execute a management method.

Images