Management device, management method, and program

Abstract

A management device (102) is provided with a master storage unit (103) for storing master information (103a) and an estimation unit (104). The master information (103a) associates a predetermined appearance of one worker or a worker group who conducts work, with the type of the work. The estimation unit (104) estimates the type of work to be conducted by one or more workers on the basis of the master information (103a) and appearance information about the appearance of the one or more workers obtained by analyzing a captured image of a working area.

Description

[Technical Field]

[0001] The present invention relates to a management device ,tube This document relates to a method and program for the management of electronic devices. [Background technology]

[0002] Various technologies have been proposed to support the improvement of efficiency in work processes at manufacturing bases.

[0003] For example, Non-Patent Document 1 describes how the system automatically measures value-added time (operating rate / operating time) and wasted time in the assembly and inspection processes within a factory from the position of a "hand" captured by a camera, thereby supporting numerical-based on-site kaizen.

[0004] For example, Non-Patent Document 2 describes platform software that converts skeletal information (posture such as body orientation and hand movements) from video captured by a camera or recorded video into data and recognizes movements.

[0005] Non-Patent Document 2 lists the patent numbers of Patent Documents 1 to 3.

[0006] Patent Document 1 describes a technique for analyzing a worker's posture. Patent Document 2 describes a technique for detecting unsafe whole-body posture. Patent Document 3 describes a technique for managing hand washing for medical staff.

[0007] Patent Document 4 describes a technology for reducing the burden on a creator regarding images of objects when creating a report using images that show the objects. The video editing device described in Patent Document 4 includes a recognition means for recognizing objects that appear in a predetermined period in a video, a registration means for registering the objects recognized by the recognition means in association with the period, and a display means for, when information is received, displaying an image in which the object identified by the information is recognized for the period associated with the object.

[0008] Patent Document 5 describes a technology that calculates the feature values ​​of each of multiple key points of a human body contained in an image, searches for images containing human bodies with similar postures or movements based on the calculated feature values, and classifies images with similar postures or movements together.

[0009] Non-Patent Document 3 describes a technique related to human skeleton estimation. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] Patent No. 6825041 [Patent Document 2] Patent No. 6884819 [Patent Document 3] Patent No. 6889228 [Patent Document 4] Japanese Patent Application Publication No. 2020-53729 [Patent Document 5] International Publication No. 2021 / 084677 [Non-patent literature]

[0011] [Non-Patent Document 1] NEC Platforms, Ltd., "Factory Value-Added Time Measurement Solution," [online], [Retrieved February 1, 2022], Internet<URL: https: / / www.necplatforms.co.jp / solution / i-iot / time-measurement / index.html> [Non-patent document 2] Hitachi Industry & Control Solutions, Ltd., "AI-based human posture and movement recognition solution," [online], [Retrieved February 1, 2022], Internet<https: / / info.hitachi-ics.co.jp / product / activity_evaluation / > [Non-patent document 3] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, P. 7291-7299 Summary of the Invention [Problem to be solved by the invention]

[0012] However, Patent Documents 1 to 5 and Non-Patent Documents 1 to 3 do not describe any technology for supporting the optimization of the entire work in a work area where one or more workers perform work.

[0013] In view of the above-mentioned problems, an example of an object of the present invention is to provide a management device, a management system, a management method, and a program that solve the difficulty of supporting the optimization of the entire work in a work area. [Means for solving the problem]

[0014] According to one aspect of the present invention, group Work jointly conduct Consists of multiple workers The predefined appearance of the worker group and the group a master storage means for storing master information relating the type of work to the work; Obtained by analyzing the video of the work area Complex Based on the appearance information relating to the appearance of the number of workers and the master information, The complex Number work The person conduct group an estimation means for estimating a type of work, The predetermined appearance is ,before and at least one of information indicating the clothing of the workers constituting the worker group and information indicating the items used for the work. and the positional relationship between the plurality of workers constituting the worker group. Contains A management device is provided.

[0016] According to one aspect of the present invention, The computer group Work jointly conduct Consists of multiple workers The predefined appearance of the worker group and the group storing master information relating to the type of work in a master storage means; Obtained by analyzing the video of the work area Complex Based on the appearance information relating to the appearance of the number of workers and the master information, The complex Number work The person conduct group including estimating the type of work; The predetermined appearance is ,before and at least one of information indicating the clothing of the workers constituting the worker group and information indicating the items used for the work. and the positional relationship between the plurality of workers constituting the worker group. Contains A management method is provided.

[0017] According to one aspect of the present invention, On the computer, group Work jointly conduct Consists of multiple workers The predefined appearance of the worker group and the group The master information relating to the type of work is stored, Obtained by analyzing the video of the work area Complex Based on the appearance information relating to the appearance of the number of workers and the master information, The complex Number work The person conduct group Performing an estimation of the type of work, The predetermined appearance is ,before and at least one of information indicating the clothing of the workers constituting the worker group and information indicating the items used for the work. and the positional relationship between the plurality of workers constituting the worker group. A program including the following is provided. [Effects of the Invention]

[0018] According to the present invention, it is possible to support the optimization of the entire work in the work area. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an overview of a management system according to a first embodiment of the present invention. [Figure 2] 1 is a flowchart showing an outline of a management process according to the first embodiment of the present invention. [Figure 3] 1 is a diagram showing an example of the configuration of a management system according to a first embodiment, together with a view of a work area A seen from above. [Figure 4] FIG. 3 is a diagram showing an example of master information according to the first embodiment. [Figure 5] FIG. 3 is a diagram illustrating a detailed example of the functional configuration of an estimation unit according to the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of history information according to the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of the physical configuration of a management device according to the first embodiment. [Figure 8] 10 is a flowchart showing a detailed example of an estimation process according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of tagging. [Figure 10] 5 is a flowchart showing an example of output processing according to the first embodiment of the present invention. [Figure 11] 10 is an example of a screen displaying history information on a display unit. [Figure 12] FIG. 10 is a diagram showing an example of the configuration of a management system according to a second embodiment of the present invention, together with a view of a work area A seen from above. [Figure 13] FIG. 10 is a diagram showing an example of the configuration of a management system according to a third embodiment of the present invention, together with a view of a work area A seen from above. [Figure 14] FIG. 11 is a diagram showing an example of master information according to the third embodiment. [Figure 15] FIG. 11 is a diagram illustrating a detailed example of the functional configuration of an estimation unit according to a third embodiment. [Figure 16] FIG. 10 is a diagram showing an example of the configuration of a management system according to a fourth embodiment of the present invention, together with a view of a work area A seen from above. [Figure 17] FIG. 13 is a diagram showing an example of the configuration of worker information according to the fourth embodiment. [Figure 18] FIG. 13 is a diagram showing an example of the configuration of unregistered information according to the fourth embodiment. [Figure 19] FIG. 10 is a diagram illustrating a detailed example of the functional configuration of an estimation unit according to a fourth embodiment. [Figure 20] 10 is a flowchart showing an example of a management process according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0021] <Embodiment 1> FIG. 1 is a diagram showing an overview of a management system 100 according to the first embodiment.

[0022] The management system 100 includes an image capturing device 101 and a management device 102 .

[0023] The image capturing device 101 captures an image of the work area and transmits image data including the captured image of the work area.

[0024] The management device 102 includes a master storage unit 103 for storing master information 103a, and an estimation unit 104. The master information 103a associates a predetermined appearance of a single worker or a group of workers performing a task with the type of task. The estimation unit 104 estimates the type of task performed by each of the one or more workers based on the master information 103a and appearance information related to the appearance of the one or more workers obtained by analyzing video footage of the work area.

[0025] This management system 100 makes it possible to support the optimization of the entire work performed by one or more workers. This management device 102 makes it possible to support the optimization of the entire work performed by one or more workers.

[0026] FIG. 2 is a flowchart showing an outline of the management process according to the first embodiment.

[0027] The management device 102 stores master information 103a in the master storage unit 103, which associates a predetermined appearance of one worker or a group of workers who will perform a task with the type of task (step S101).

[0028] The management device 102 estimates the type of work to be performed by each of the one or more workers based on appearance information regarding the appearance of the one or more workers obtained by analyzing the video of the work area and the master information 103a (step S102).

[0029] This management process makes it possible to support the optimization of the entire work performed by one or more workers.

[0030] A detailed example of the management system 100 according to the first embodiment will be described below.

[0031] FIG. 3 is a diagram showing an example of the configuration of the management system 100 according to the first embodiment of the present invention, together with a diagram showing the work area A as viewed from above.

[0032] 3 shows an example of a worker Pa performing task A in a work area A set up in a manufacturing factory for food, electronic devices, etc., as viewed from above. Task A is the attachment of three predetermined parts to a semi-finished product M1 that is placed on a conveyor C and moving in the direction of arrow AR (to the right in the figure).

[0033] While performing task A, worker P is in work area X, which is indicated by a roughly rectangular dotted line in Fig. 3. Worker P moves along conveyor C from position X1 to position X3 in work area X, repeatedly performing three processes (process A1, process A2, and process A3) by himself, in which parts are attached one by one to semi-finished product M1.

[0034] Worker Pa is one of the workers P who work in work area A. There may be one or more workers P. An example of group work in which multiple workers work together will be described later in another embodiment.

[0035] Furthermore, task A is an example of a task. The task is not limited to assembly work. The task may be, for example, transporting parts, semi-finished products, finished products, or inspection work, and is not limited to work in a manufacturing factory. The task may be comprised of one or more steps that are appropriately determined.

[0036] Furthermore, the work area A may be any area in which one or more types of work are performed, and the location in which it is set is not limited to a manufacturing factory for food, electronic equipment, etc. The work area A may include multiple work sites.

[0037] The management system 100 is a system for managing work in a work area.

[0038] As described above, the management system 100 includes the image capturing device 101 and the management device 102. The image capturing device 101 and the management device 102 are connected to each other via a network N. The network N is a communication network configured using wired or wireless means or a combination of these. Therefore, the image capturing device 101 and the management device 102 can transmit and receive information, data, and the like to each other via the network N.

[0039] The photographing device 101 is a device for photographing the work area A. The photographing device 101 is, for example, a camera.

[0040] The camera device 101 captures an image of the work area A. The camera device 101 captures an image of the work area A and generates video data including an image of the work area A. The camera device 101 transmits the video data to the management device 102 via the network N. It is desirable that the video include a whole-body image of the worker P in the work area A.

[0041] More specifically, for example, the camera device 101 continuously captures images of the working area A. The frame rate at which the camera device 101 captures images may be set as appropriate. The camera device 101 continuously generates image data including frame images generated in each capture. As a result, the camera device 101 generates video data including video (moving images) made up of multiple frame images.

[0042] (Functional configuration of management device 102) The management device 102 is a device for managing work in the work area A. In detail, the management device 102 includes a master storage unit 103, an estimation unit 104, a history storage unit 105, a management unit 106, a display unit 107, an output control unit 108, and an input reception unit 109.

[0043] The master storage unit 103 is a storage unit for storing master information 103a.

[0044] The master information 103a is information indicating criteria for determining the type of work to be performed by the worker P. Fig. 4 is a diagram showing an example of the master information 103a according to this embodiment.

[0045] In detail, the master information 103a associates the type of work, the appearance (master appearance information), the work area, the process ID (Identifier), and the work position / work posture.

[0046] The type of work is information for identifying the type of work.

[0047] The master appearance information is information that indicates a predetermined appearance of a worker P performing work. The master appearance information includes at least one of information indicating at least one of the worker P's whole body, clothing, posture, and behavior, and information indicating items used for the work. The behavior is at least one of a change or transition in posture, movement (change or transition in position), etc. The information included in the master appearance information is, for example, an image and image features.

[0048] Hereinafter, the "item used for work" will also be simply referred to as the "item."

[0049] The work area is information indicating the location where work is performed. In other words, the work area is information indicating the range in which the worker P performing the work is present while working. The work area may be indicated by a range within an image or by a range in real space.

[0050] The process ID is information for identifying each process included in the work.

[0051] The work position / work posture is information including the work position or the work posture.

[0052] The work position is information indicating the location where each process included in the work is performed. In other words, the work position is information indicating the range where a worker P performing each process included in the work is located while performing that process. The work position may be indicated as a range within an image or as a range in real space.

[0053] The working posture is information indicating the posture in each step included in the work. The information included in the working posture is, for example, an image or image feature amount.

[0054] For example, for work that requires moving positions during the work, the work position / work posture may include the work position. Posture changes For work involving the use of a hand, the working position / working posture may include the working posture.

[0055] 4 includes information indicating that, for example, for the task "task A," the predetermined appearance is "appearance A" and the work site is "work site X." Furthermore, for the task "task A," the master information 103a shown in FIG. 4 includes information indicating that the task "task A" includes three processes with process IDs "process A1" to "process A3," and that the respective work positions are "position X1" to "position X3."

[0056] For example, if task A is a task that requires wearing a hat, appearance A includes information indicating the outfit that includes the hat.

[0057] For example, if Work B involves wearing orange work clothes and carrying a load using a cart, Appearance B includes an image or image feature representing the orange work clothes. Appearance B also includes an image or image feature representing the cart, which is an item.

[0058] For example, if task C is a task in which the user repeatedly stands and squats in the same position, appearance C includes information indicating at least one of the posture, a change in posture, and a transition in posture.

[0059] Additionally, since tasks A and B involve movement of position, the "task position / task posture" includes the task position. Task C involves a change of posture, so the "task position / task posture" includes the task posture.

[0060] Referring again to FIG. The estimation unit 104 estimates the type of work performed by each of one or more workers P based on appearance information regarding the appearance of the one or more workers P obtained by analyzing footage of the work area A and the master information 103a.

[0061] FIG. 5 is a diagram showing a detailed example of the functional configuration of the estimation unit 104 according to this embodiment.

[0062] The estimation unit 104 includes an analysis unit 111 , an operation estimation unit 112 , and a history generation unit 113 .

[0063] The analysis unit 111 acquires the video data generated by the imaging device 101 from the imaging device 101 via the network N. The analysis unit 111 analyzes the video included in the acquired video data, i.e., the video of the working area A.

[0064] Specifically, the analysis unit 111 has one or more analysis functions that perform processing (analysis processing) for analyzing video. The analysis functions that the analysis unit 111 has are one or more of: (1) an object detection function, (2) a face analysis function, (3) a human figure analysis function, (4) a posture analysis function, (5) a behavior analysis function, (6) an appearance attribute analysis function, (7) a gradient feature analysis function, (8) a color feature analysis function, and (9) a movement line analysis function.

[0065] (1) The object detection function detects an object from an image. The object detection function can also determine the position of an object within an image. An example of a model that can be applied to object detection processing is YOLO (You Only Look Once). The object detection function detects, for example, a worker P, an object, etc. Also, for example, the object detection function determines the position of the worker P, an object, etc.

[0066] Here, "object" includes people and things, and the same applies hereinafter.

[0067] (2) The face analysis function detects human faces from images, extracts the features of the detected faces (facial feature values), and classifies the detected faces (classification). The face analysis function can also determine the position of the face within the image. The face analysis function can also determine the identity of people detected from different images based on the similarity between the facial feature values ​​of people detected from different images.

[0068] (3) The human morphology analysis function extracts the physical characteristics of people in images (for example, values ​​that indicate overall characteristics such as whether they are fat or thin, height, and clothing), and classifies (classifies) people in images. The human morphology analysis function can also identify the position of a person in an image. The human morphology analysis function can also determine the identity of people in different images based on the physical characteristics of people in different images.

[0069] (4) The posture analysis function detects the joint points of people from the image and creates a stick figure model by connecting the joint points. The posture analysis function then uses the information from the stick figure model to estimate the posture of the person, extract the feature values ​​of the estimated posture (posture feature values), and classify (classify) the people in the image. The posture analysis function can also determine the identity of people in different images based on the posture feature values ​​of the people in different images.

[0070] For example, the posture analysis function estimates postures such as standing, crouching, and bending from an image and extracts posture feature values ​​that indicate each posture. Also, for example, the posture analysis function can estimate the posture of an object detected using an object detection function or the like from an image and extract posture feature values ​​that indicate that posture.

[0071] For example, the techniques disclosed in Patent Document 5 and Non-Patent Document 3 can be applied to the posture analysis function. (5) The behavior analysis process can estimate a person's movements using information about the stick figure model, changes in posture, etc., extract features of the person's movements (movement features), and classify (classify) people included in the image. The behavior analysis process can also estimate a person's height and identify the person's position in the image using information about the stick figure model. The behavior analysis process can estimate behaviors such as changes or transitions in posture and movements (changes or transitions in position) from the image, and extract movement features of the behavior.

[0072] (6) The appearance attribute analysis function can recognize appearance attributes associated with people. The appearance attribute analysis function extracts features related to the recognized appearance attributes (appearance attribute features) and classifies (classifies) people in the image. Appearance attributes are attributes of appearance, and include one or more of, for example, clothing color, shoe color, hairstyle, and whether or not a hat, tie, or glasses are worn.

[0073] (7) The gradient feature analysis function extracts gradient features from an image. For gradient feature detection, technologies such as SIFT, SURF, RIFF, ORB, BRISK, CARD, and HOG can be applied.

[0074] (8) The color feature analysis function can detect objects from images, extract the color features of the detected objects, and classify the detected objects.

[0075] An example of a color feature is a color histogram. The color feature analysis function can detect, for example, people and work items included in an image. The color feature analysis function can also classify items into classes such as conveyor C, workbench, parts, semi-finished products, finished products, luggage to be transported, tools, instruments, and equipment (for example, carts and dollies for transporting luggage, and screwdrivers for fastening screws).

[0076] (9) The flow line analysis function can determine the flow line (trajectory of movement) of a person included in a video, for example, using the result of the identity determination in any of the analysis functions (2) to (6) above. In more detail, for example, by connecting people who are determined to be the same in different images in a time series, the flow line of the person can be determined. Note that, when acquiring videos captured by multiple image capture devices 101 capturing different shooting areas, the flow line analysis function can also determine the flow line across multiple videos captured in different shooting areas.

[0077] Image features include, for example, the results of object detection by the object detection function, facial features, human body features, posture features, movement features, appearance attribute features, gradient features, color features, and movement lines.

[0078] Each of the analysis functions (1) to (9) may appropriately use the results of analysis performed by other analysis functions.

[0079] The analysis unit 111 uses the above analysis functions (1) to (9) to analyze the video of the work area A and detects objects (worker P and objects) included in the video. The analysis unit 111 also generates appearance information related to the appearance of the detected worker P. The appearance information is time-series information related to the appearance of the worker.

[0080] The appearance information includes at least one of information indicating at least one of the whole body, clothing, posture, and behavior of the worker P, and information indicating items used for the work. The information included in the appearance information is, for example, an image or image features.

[0081] The task estimation unit 112 estimates the type of task that the worker P will perform based on the appearance information generated by the analysis unit 111 and the master information 103a.

[0082] Furthermore, the work estimation unit 112 estimates the work time of the worker P based on the appearance information generated by the analysis unit 111 and the master information 103a. The work time of the worker P is a time related to the work performed by the worker P.

[0083] The work time includes a time related to working hours, a time related to actual work time, and a time related to process execution time.

[0084] Working hours are the time that worker P is working, for example, the time that worker P is in work area A with a predetermined appearance. Actual working time is the time that worker P is actually performing work, for example, the time that worker P is in a work area corresponding to the work with a predetermined appearance. Process execution time is the time that worker P performs each process, and is the time that the work position or work posture corresponding to each process is maintained. The times related to working hours, actual working time, and process execution time each include a start time and an end time.

[0085] The work time is not limited to this, and may be, for example, a time related to one or two of working hours, actual work time, and process execution time.

[0086] The history generation unit 113 generates history information 105a based on the results of estimation by the operation estimation unit 112.

[0087] 6 is a diagram showing an example of history information 105a according to this embodiment. The history information 105a is information showing the work history of worker P. The history information 105a according to this embodiment includes a worker ID, a type of work, a time related to working hours, a time related to actual work time, and a time related to process execution time.

[0088] The worker ID included in the history information 105a is information for identifying the worker P. The worker ID may be assigned as appropriate.

[0089] The type of work included in the history information 105a is the type of work performed by the associated worker P. This type of work is the type of work estimated by the work estimation unit 112.

[0090] The history information 105a shown in FIG. 6 includes an example of a history of a worker Pa, whose worker ID is "Pa," performing "task A." The history information 105a shown in FIG. 6 indicates that worker Pa's working hours were from "ST1" to "ST2," during which there was actual work time from "WT1" to "WT2." It also indicates that processes A1 to A3 were performed during this actual work time. For example, "WT1" to "WT2" are an example of the time related to the process execution time of process A1, which was performed first during the actual work time.

[0091] The history information 105a may include multiple times related to actual work times for one working hour, since the worker P may take breaks, etc. Also, since a process included in a job is usually repeated multiple times during one actual working hour, the history information 105a may include multiple times related to process execution times for one actual working hour.

[0092] Referring again to FIG. The history storage unit 105 is a storage unit for storing history information 105a.

[0093] Based on the history information 105a, the management unit 106 generates management information for supporting the management of work in the work area A. For example, the management unit 106 generates the management information by statistically processing the history information 105a.

[0094] The display unit 107 displays various types of information.

[0095] The output control unit 108 performs control for outputting information. For example, the output control unit 108 may cause the display unit 107 to display the information, or may output electronic data including the information.

[0096] For example, the output control unit 108 displays the history information 105a, the management information, etc. on the display unit 107. For example, the output control unit 108 outputs electronic data including the history information 105a, the management information, etc.

[0097] The input receiving unit 109 receives input from the user.

[0098] Up to now, the functional configuration of the management system 100 according to the first embodiment has been mainly described. From here, the physical configuration of the management system 100 according to this embodiment will be described.

[0099] (Physical configuration of management system 100) The management system 100 is physically composed of an image capturing device 101 and a management device 102 connected via a network N. The image capturing device 101 and the management device 102 are each composed of a single, physically separate device.

[0100] The management device 102 may be physically composed of multiple devices connected via an appropriate communication line such as a network N. The image capturing device 101 and the management device 102 may be physically composed of a single device. When the management system 100 includes multiple image capturing devices 101, for example, one or more of the image capturing devices 101 may include at least a part of the management device 102.

[0101] The management device 102 is physically, for example, a general-purpose computer.

[0102] In detail, for example, the management device 102 physically includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070, as shown in FIG.

[0103] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and other components to each other is not limited to bus connection.

[0104] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0105] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0106] The storage device 1040 is an auxiliary storage device realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, a ROM (Read Only Memory), or the like. The storage device 1040 stores program modules for realizing the functions of the management device 102. The processor 1020 reads each of these program modules into the memory 1030 and executes them, thereby realizing the function corresponding to that program module.

[0107] The network interface 1050 is an interface for connecting the management device 102 to the network N.

[0108] The input interface 1060 is an interface such as a touch panel, keyboard, or mouse that allows the user to input information.

[0109] The output interface 1070 is a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like, which serves as an interface for presenting information to the user. The output interface 1070 constitutes the display unit 107. The output interface 1070 may be built into the management device 102, or may be provided externally to the management device 102.

[0110] Up to now, the explanation has been given mainly on the physical configuration of the management system 100 according to embodiment 1. From here, the operation of the management system 100 according to this embodiment will be explained.

[0111] (Operation of the management system 100) (Administrative processing) The management device 102 executes a management process (see FIG. 2). The management process is a process for managing work in the work area A. For example, when the management device 102 receives a start instruction from a user, the management device 102 starts the management process. For example, when the management device 102 receives an end instruction from a user, the management device 102 ends the management process.

[0112] Please refer to Figure 2. The input receiving unit 109 stores the master information 103a in the master storage unit 103 in accordance with the user's input (step S101).

[0113] The input receiving unit 109 may execute step S101 when initializing the management device 102 for the first time, when changing the master information 103a, etc. Therefore, the input receiving unit 109 may execute step S101, for example, when requested by a user.

[0114] The estimation unit 104 estimates the type of work to be performed by the worker P based on appearance information relating to the appearance of the worker P obtained by analyzing the video of the work area A and the master information 103a (step S102).

[0115] FIG. 8 is a flowchart showing a detailed example of the estimation process (step S102) according to this embodiment.

[0116] Please refer to Figure 8. The analysis unit 111 acquires the video data generated by the image capturing device 101 from the image capturing device 101 via the network N (step S102a).

[0117] The analysis unit 111 acquires, for example, video data for a predetermined period at a predetermined time from the imaging device 101. The predetermined period may be determined as appropriate, for example, one day. The predetermined time may be determined as appropriate, for example, a time when no work is being performed in the work area A.

[0118] As described above, the video data is a moving image made up of a plurality of frame images, and includes the shooting time of each frame image.

[0119] The analysis unit 111 may acquire the video data from the imaging device 101 in real time. In this case, the analysis unit 111 may hold the video data for a predetermined period. The analysis unit 111 may acquire the video data from the imaging device 101 via a device that holds the video data. The analysis unit 111 may acquire the video data via a storage medium, not limited to the network N.

[0120] The analysis unit 111 analyzes the image included in the image data acquired in step S102a, that is, the image captured of the working area A, and detects an object included in the image (step S102b).

[0121] For example, when analyzing a video of the work area A shown in FIG. 3, the analysis unit 111 detects the worker Pa, the conveyor C, and the like.

[0122] Referring again to FIG. The analysis unit 111 repeatedly executes steps S102d to S102g for all workers P detected in step S102c (loop A; step S102c).

[0123] The analysis unit 111 assigns a worker ID to the worker P to be processed and also performs tagging (step S102d).

[0124] In detail, the analysis unit 111 assigns the same worker ID to the same worker P included in multiple frame images based on the result of determining the identity of the person using, for example, facial feature amounts.

[0125] The analysis unit 111 tags the image of the worker P to be processed that is included in each frame image. In tagging, the analysis unit 111, for example, marks the image area of ​​the worker P (for example, a rectangular frame) and associates the worker ID with the mark. FIG. 9 is a diagram showing an example of tagging. FIG. 9 shows an example in which the image of the worker Pa is surrounded by a rectangular frame mark and the worker ID "Pa" of the worker Pa is associated with the frame.

[0126] The analysis unit 111 may detect a specific action instructed by a user and tag an image of a worker performing the specific action. The tag may include information (e.g., a wording, a mark, a symbol, etc.) that indicates that the specific action is instructed by the user.

[0127] Referring again to FIG. The analysis unit 111 analyzes the image included in the image data acquired in step S102a, that is, the image of the work area A, and generates appearance information of the worker P to be processed (step S102e).

[0128] For example, the analysis unit 111 analyzes the video using an object detection function, a human figure analysis function, a posture analysis function, an appearance attribute analysis function, a color feature analysis function, etc. Using the output results of these functions, the analysis unit 111 acquires appearance information indicating the whole body, clothing, posture, behavior, items, etc. of the worker P to be processed.

[0129] The task estimation unit 112 estimates the type of task to be performed by the processing target worker P based on the appearance information acquired in step S102e and the master information 103a (step S102f).

[0130] In detail, for example, the task estimation unit 112 identifies master appearance information that matches the appearance information acquired in step S102e. Here, "match" means substantially match, and includes cases where they differ within a predetermined range. The task estimation unit 112 identifies the type of task associated with the identified master appearance information in the master information 103a. The task estimation unit 112 estimates that the identified type of task is the type of task performed by worker P, who is the processing target.

[0131] 4, the master appearance information includes appearances A to C. The work estimation unit 112 identifies the master appearance information from among appearances A to C that matches the appearance information of the worker Pa who is the processing target.

[0132] For example, assume that appearance A matches the appearance information of the processing target worker Pa. In this case, the task estimation unit 112 estimates that task A associated with appearance A in the master information 103a is the type of task performed by worker Pa.

[0133] The task estimation unit 112 estimates the task time of the task to be performed by the worker P based on the appearance information acquired in step S102c and the master information 103a (step S102g).

[0134] (Estimation of working hours) The activity estimation unit 112 identifies a frame image that first appears in the video with an appearance corresponding to the type of activity estimated in step S102f. The activity estimation unit 112 estimates the capture time of the frame image as the start time of the workday.

[0135] The task estimation unit 112 identifies the frame image that last appeared in the video with an appearance corresponding to the task type estimated in step S102f. The task estimation unit 112 estimates the capture time of the frame image as the end time of the workday.

[0136] (Time estimation for actual work time) The work estimation unit 112 identifies a group of frame images of the person in the work area X with an appearance according to the type of work estimated in step S102f. At this time, if there are multiple actual work times, the work estimation unit 112 identifies multiple groups of frame images.

[0137] The activity estimation unit 112 estimates the shooting time of the first frame image in each of the identified one or more frame image groups as the start time of the actual activity time. The activity estimation unit 112 estimates the shooting time of the last frame image in each of the identified one or more frame image groups as the end time of the actual activity time.

[0138] (Time estimation for process execution time) The operation estimation unit 112 identifies the first frame image corresponding to the operation position or operation posture for each of one or more frame image groups identified for estimating the time related to the actual operation time. The operation estimation unit 112 estimates the capture time of the frame image as the start time of the process corresponding to the operation position or operation posture.

[0139] Next, for each of one or more frame image groups identified for estimating times related to actual work time, the task estimation unit 112 identifies a frame image in which the task position or task posture has changed to a task position or task posture corresponding to the next task. The task estimation unit 112 estimates the capture time of that frame image as the end time of the previous task and as the start time of the next task.

[0140] For example, the process of estimating the time related to the process execution time of task A will be described with reference to Fig. 4. The task estimation unit 112 identifies the first frame image corresponding to "position X1" for each of one or more frame image groups identified for estimating the time related to the actual task time. The task estimation unit 112 estimates the capture time of that frame image as the start time of "process A1."

[0141] Next, for each of one or more frame image groups identified for estimating times related to actual task time, the task estimation unit 112 identifies a frame image in which the task position has changed from "position X1" to "position X2." The task estimation unit 112 estimates the capture time of that frame image as the end time of the previous task, "step A1," and also as the start time of the next task, "step A2." The task estimation unit 112 repeats this process sequentially up to the last frame image in each of one or more frame image groups.

[0142] The work estimation unit 112 estimates the shooting time of the last frame image in each of one or more groups of frame images as the end time of the process corresponding to the work position in that frame image (usually, "process A3," the last process of work A).

[0143] 4, a process for estimating the time related to the process execution time of task C will be described. The task estimation unit 112 identifies the first frame image corresponding to "posture Z1" for each of one or more frame image groups identified for estimating the time related to the actual task time. The task estimation unit 112 estimates the capture time of that frame image as the start time of "process C1."

[0144] Next, the task estimation unit 112 identifies a frame image in which the task posture has changed from "posture Z1" to "posture Z2" for each of one or more frame image groups identified for estimating the time related to actual task time. The task estimation unit 112 estimates the capture time of that frame image as the end time of the previous step, "step C1," and also as the start time of the next step, "step C2." The task estimation unit 112 repeats this process sequentially up to the last frame image in each of one or more frame image groups.

[0145] The work estimation unit 112 estimates the shooting time of the last frame image in each of one or more frame image groups as the end time of the process corresponding to the work posture in that frame image (usually, “process C2,” the last process of work C).

[0146] Referring again to FIG. When the analysis unit 111 has executed steps S102d to S102g for all workers P detected in step S102c, it ends loop A (step S102c).

[0147] The analysis unit 111 generates history information 105a based on the results of steps S102f to S102g executed for each of one or more workers P as processing targets (step S102h).

[0148] The analysis unit 111 generates, for example, history information 105a for each worker P as shown in FIG.

[0149] The analysis unit 111 returns to the management process (see FIG. 2) and repeatedly executes the estimation process (step S102).

[0150] By executing the management process, it is possible to accurately collect and manage history information 105a including the types of work performed in work area A. Therefore, it is possible to support the optimization of all work in the work area.

[0151] (Output processing) 10 is a flowchart showing an example of the output process according to embodiment 1. The output process is a process of statistically processing the history information 105a and outputting the processed information.

[0152] The management unit 106 generates management information based on the history information 105a, for example, by performing statistical processing on the history information 105a (step S111).

[0153] In detail, for example, the management unit 106 obtains the number of workers P performing the same type of work and generates management information including that number. Also, for example, the management unit 106 obtains the number of workers P performing the same type of work by time period and generates management information including the number of workers by time period. The statistical processing in step S111 may be modified in various ways.

[0154] Furthermore, for example, the management unit 106 determines at least one of the required time, the proportion of the required time, etc. for each process performed by each worker during their actual working time, and generates management information including at least one of the required time, the proportion of the required time, etc. for each process.

[0155] The output control unit 108 outputs information such as the history information 105a and management information in response to a user instruction (step S112).

[0156] In detail, for example, the output control unit 108 outputs the information to the display unit 107, thereby displaying the information on the display unit 107. Also, for example, the output control unit 108 stores electronic data including the information in a storage location designated by the user in a storage unit (not shown) of the management device 102, or transmits the electronic data to a destination designated by the user.

[0157] 11 is an example of a screen displaying history information 105a on display unit 107. This figure includes an example in which, when the cursor is placed on a line indicating the actual work time being displayed, detailed information about the actual work time is displayed. In this figure, an example is shown in which the detailed information is the ratio of the required time for each process to the actual work time.

[0158] By executing the output process, it is possible to statistically process the history information 105a and obtain the management information necessary to optimize the entire work in the work area A. Therefore, it is possible to support the optimization of the entire work in the work area. Furthermore, it is possible to output information such as the history information 105a and management information in a format that is easy for the user to view. Therefore, it is possible to support the optimization of the entire work in the work area.

[0159] The first embodiment of the present invention has been described above.

[0160] (Actions and Effects) According to this embodiment, the management device 102 includes a master storage unit 103 for storing master information 103a, and an estimation unit 104. The master information 103a associates a predetermined appearance of a worker P performing a task with the type of task. The estimation unit 104 estimates the type of task performed by each of the one or more workers based on the master information 103a and appearance information related to the appearance of the one or more workers obtained by analyzing video footage of the task area A.

[0161] This makes it possible to estimate the type of work being performed in work area A. Therefore, for example, history information 105a including the type of work being performed in work area A can be accurately collected and managed. For example, by utilizing this history information 105a, it is possible to optimize all of the work in work area A. Therefore, it becomes possible to support the optimization of all of the work in work area A.

[0162] According to this embodiment, the predetermined appearance (master appearance information) includes at least one of information indicating the whole body, clothing, posture, and behavior of the worker P performing the work, and information indicating the items used for the work.

[0163] Generally, a worker P who performs work often wears predetermined clothing that is determined by a manufacturing factory or the like. The predetermined clothing is, for example, work clothes, a uniform, a hat, shoes, etc. that are determined according to the work that the worker P will be performing. In the case of work clothes or uniforms, the color may vary depending on the work that the worker P will be performing. Whether or not a hat is worn by the worker P is often determined according to the work that the worker P will be performing. Also, if a hat is worn, the color of the hat may vary depending on the work that the worker P will be performing. The shoes may be determined as safety shoes, work shoes, clean room shoes, etc. depending on the work that the worker P will be performing. In this case, the color of the shoes may vary depending on the work that the worker P will be performing.

[0164] Furthermore, in work, at least one of the postures or actions is often patterned into a certain style, such as moving within a certain range or repeatedly standing and squatting.

[0165] Furthermore, there are often items used for work, such as conveyor C, workbenches, parts, semi-finished products, finished products, cargo to be transported, tools, instruments, and equipment (for example, carts and dollies for transporting cargo, screwdrivers for fastening screws).

[0166] Therefore, it is possible to estimate the type of work that the worker P will be doing based on the video, based on at least one of information indicating the worker P's whole body, clothing, posture, and behavior, and information indicating the item. This makes it possible to support the optimization of the entire work in the work area A.

[0167] According to this embodiment, the management device 102 includes a management unit 106 that outputs management information based on the result of estimation by the estimation unit 104. For example, the management information includes the number of workers performing the same type of work. Also, for example, the management information includes the number of workers performing the same type of work by time period.

[0168] In particular, the whole-body information of worker P can include information indicating clothing, posture, etc. In this way, since the whole-body information of worker P can include multiple pieces of information, the effort required to set the master appearance information can be reduced compared to preparing each piece of information such as clothing and posture and setting it as master appearance information. Therefore, it is possible to support the optimization of the entire work in work area A while reducing the user's effort.

[0169] According to this embodiment, the estimation unit 104 further estimates the work time of each of one or more workers P based on the appearance information and the master information 103a.

[0170] This makes it possible to accurately collect and manage history information 105a including the type of work performed in work area A as well as the time of work. For example, by utilizing this history information 105a, it is possible to optimize all work in the work area. Therefore, it becomes possible to support the optimization of all work in work area A.

[0171] According to this embodiment, the video includes a whole-body image of the worker P in the work area A.

[0172] Generally, in a video, due to factors such as the orientation of the worker P's face, the face of the worker P may not be included clearly enough to determine the identity of the person based on facial features. Even in such a case, if the video includes a full-body image of the worker P, the identity of the person can be determined using, for example, anthropometric features, hairstyle, physique, etc. This makes it possible to accurately and reliably collect and manage history information 105a including the types of work performed in the work area A. This makes it possible to further support the optimization of the overall work in the work area A.

[0173] <Embodiment 2> 12 is a diagram showing an example of the configuration of a management system 200 according to a second embodiment of the present invention, together with a view of the working area A seen from above. In this embodiment, for the sake of brevity, explanations that overlap with those of the first embodiment will be omitted as appropriate, and differences from the first embodiment will be mainly described.

[0174] The management system 200 includes a plurality of image capturing devices 101 and a management device 102 .

[0175] The multiple image capturing devices 101 are devices for capturing an image of the entire working area A. The multiple image capturing devices 101 capture images of the working area A from different directions. Each of the multiple image capturing devices 101 is similar to the image capturing device 101 according to the first embodiment.

[0176] In the second embodiment, the estimation unit 104 estimates the type of work to be performed by each of one or more workers P based on appearance information regarding the appearance of the one or more workers P obtained by analyzing multiple images of the work area A and the master information 103a.

[0177] Except for these points, the management system 200 may be configured functionally and physically in the same manner as the management system 100 according to the first embodiment. In addition, the management system 200 may operate in the same manner as the management system 100 according to the first embodiment.

[0178] The second embodiment of the present invention has been described above.

[0179] (Actions and Effects) According to this embodiment, the estimation unit 104 estimates the type of work performed by each of one or more workers based on the master information 103a and appearance information relating to the appearance of one or more workers obtained by analyzing multiple videos of the work area A. The multiple videos are videos of the work area A taken from different directions.

[0180] In this way, by using images captured from multiple different directions, it is possible to obtain images that capture almost the entire work area A without any omissions. This makes it possible to recognize whether there is free space in the work area and change the placement of workers P, etc. This makes it possible to support the optimization of the entire work in work area A.

[0181] Furthermore, photographing worker P's face from different directions makes it easier to capture the face of worker P. This makes it easier to reliably determine the identity of the person based on facial features. This makes it possible to accurately and reliably collect and manage history information 105a including the types of work performed in work area A. This makes it possible to further support the optimization of the overall work in work area A.

[0182] <Embodiment 3> In the third embodiment, an example of group work in which multiple workers P work together will be described. In this embodiment, for the sake of brevity, explanations that overlap with the first embodiment will be omitted as appropriate, and differences from the first embodiment will be mainly described.

[0183] FIG. 13 is a diagram showing an example of the configuration of a management system 300 according to a third embodiment of the present invention, together with a diagram showing a working area A as viewed from above.

[0184] The management system 300 includes the same image capturing device 101 as in the first embodiment, and a management device 302 that replaces the management device 102 according to the first embodiment.

[0185] The management device 302 includes the history storage unit 105, management unit 106, display unit 107, output control unit 108, and input reception unit 109, which are the same as those in the first embodiment. The management device 302 includes a master storage unit 303 and an estimation unit 304, which replace the master storage unit 103 and the estimation unit 104 in the first embodiment.

[0186] The master storage unit 303 is a storage unit for storing master information 303a, as in the first embodiment.

[0187] As in the first embodiment, the master information 303a is information indicating criteria for determining the type of work to be performed by worker P. In this embodiment, the master information 303a differs from the master information 103a according to the first embodiment in that it includes master information 303a about group work.

[0188] FIG. 14 is a diagram showing an example of the master information 303a according to this embodiment.

[0189] The master information 303a according to this embodiment associates a predetermined appearance of a worker P who performs a task with the type of the task, similar to the master information 103a according to the first embodiment. In addition, the master information 303a associates a predetermined appearance of a worker group who performs a task with the type of the task.

[0190] A worker group is a group made up of multiple workers who work together.

[0191] The master appearance information for group work, that is, the predetermined appearance for group work, includes at least one of the number and positional relationship of the multiple workers P who make up the worker group in addition to the same appearance as in the first embodiment.

[0192] Master information 303a shown in FIG. 14 includes information associating master appearance information with the work area, process ID, and work position / work posture similar to those in the first embodiment for "Work D," which is a group work.

[0193] The master appearance information associated with "Task D" includes at least one of information indicating the whole body, clothing, posture, and behavior of each of the multiple workers constituting the worker group, and information indicating the items used for the task. This master appearance information also includes the number of workers constituting the worker group and their positional relationship. FIG. 14 shows an example in which the number of workers is "two" and the positional relationship is "facing each other at a distance of no more than XX pixels." "XX pixels" is an example in which the distance is expressed in terms of the number of pixels on the screen (distance on the screen). The distance may also be expressed as a distance in real space.

[0194] As in the first embodiment, the estimation unit 304 estimates the type of work to be performed by each of one or more workers P based on appearance information regarding the appearance of the one or more workers P obtained by analyzing the video footage of the work area A and the master information 103a.

[0195] FIG. 15 is a diagram showing a detailed example of the functional configuration of the estimation unit 304 according to this embodiment.

[0196] The estimation unit 304 includes the history generation unit 113 similar to that of the first embodiment, and an analysis unit 311 and an operation estimation unit 312 instead of the analysis unit 111 and the operation estimation unit 112 according to the first embodiment.

[0197] Similar to the first embodiment, the analysis unit 311 uses the analysis function to analyze the video of the work area A and detect objects (worker P and objects) included in the video. Also, similar to the first embodiment, the analysis unit 311 generates appearance information related to the appearance of the detected worker P.

[0198] When multiple workers P are included in the same frame image, the appearance information according to this embodiment includes at least one of the positional relationship and the number of the multiple workers P. For example, the appearance information includes the distance between the multiple workers P (e.g., the distance on the screen, the distance in real space), the orientation of each of the multiple workers P, etc.

[0199] The task estimation unit 312 estimates the type of task that the worker P will perform based on the appearance information generated by the analysis unit 111 and the master information 303a.

[0200] In detail, as in the first embodiment, the work estimation unit 312 estimates the type of work based on at least one of the information indicating at least one of the whole body, clothing, posture, and behavior of the worker P, and information indicating the items used for the work, which are included in the appearance information and the master appearance information.

[0201] The work estimation unit 312 in this embodiment, when the appearance information includes at least one of the positional relationship and the number of workers P, further estimates the type of work for group work based on at least one of the number of workers and the positional relationship included in the appearance information and the master appearance information.

[0202] Except for these points, the management system 300 may be configured functionally and physically in the same manner as the management system 100 according to the first embodiment.

[0203] In the estimation process according to this embodiment, in step S102e, the analysis unit 311 analyzes the video of the work area A, and generates appearance information including the whole body, clothing, posture, behavior, objects, etc., similar to embodiment 1. In addition, when there is a frame image including the processing target worker P and another worker P, the analysis unit 311 generates appearance information including at least one of the positional relationship and the number of multiple workers P.

[0204] In step S102f, the task estimation unit 112 estimates the type of task to be performed by the processing target worker P based on the appearance information acquired in step S102e and the master information 103a, as in the embodiment.

[0205] In detail, when the appearance information includes at least one of the positional relationship and the number of workers P, the work estimation unit 312 in this embodiment estimates the type of work for group work based on the appearance information and master appearance information.

[0206] More specifically, for example, if the appearance information includes the number of workers P, the work estimation unit 312 identifies master appearance information whose number of workers, whole body, clothing, posture, behavior, objects, etc. match the appearance information. Furthermore, for example, if the appearance information includes the positional relationship of workers P, the work estimation unit 312 identifies master appearance information whose positional relationship, whole body, clothing, posture, behavior, objects, etc. match the appearance information.

[0207] Furthermore, for example, if the appearance information includes the number and positional relationship of workers P, the work estimation unit 312 identifies master appearance information whose number of people, positional relationship, whole body, clothing, posture, behavior, items, etc. match the appearance information.

[0208] Even when the appearance information includes at least one of the positional relationship and the number of workers P, some or all of the multiple workers P may be performing work that should be done alone. Therefore, after performing processing to estimate the type of work for group work, for example, the work estimation unit 312 identifies master appearance information whose information other than the number of workers and positional relationship (such as the whole body, clothing, posture, behavior, and objects) matches the appearance information, as in the first embodiment.

[0209] In either case, the work estimation unit 312 determines whether the specified Master The type of work associated with the appearance information is identified, and the work estimation unit 112 estimates that the identified type of work is the type of work to be performed by the worker P who is the processing target.

[0210] Except for these points, the management process according to this embodiment may be the same as the management process according to the first embodiment.

[0211] The third embodiment of the present invention has been described above.

[0212] (Actions and Effects) According to this embodiment, the master information 103a associates a predetermined appearance of a worker group performing a task with the type of task. A worker group is made up of multiple workers who perform a task together.

[0213] This makes it possible to estimate the type of work performed in group work in work area A. Therefore, for example, history information 105a including the type of work performed in group work can be accurately collected and managed. For example, by utilizing this history information 105a, it is possible to optimize the overall work in work area A. Therefore, it becomes possible to support the optimization of the overall work in work area A.

[0214] According to this embodiment, the predetermined appearance (master appearance information) of a worker group performing a task includes at least one of the number of workers constituting the worker group and their positional relationships.

[0215] This makes it possible to estimate the type of work performed in group work in work area A. Therefore, for example, history information 105a including the type of work performed in group work can be accurately collected and managed. For example, by utilizing this history information 105a, it is possible to optimize the overall work in work area A. Therefore, it becomes possible to support the optimization of the overall work in work area A.

[0216] <Embodiment 4> Usually, a predetermined worker P often enters and leaves the work area A. However, there are cases where a person other than the predetermined worker P enters and leaves the work area A. In the fourth embodiment, an example of processing when a person other than the predetermined worker P is included in the video of the work area A will be described. In this embodiment, for the sake of brevity, explanations that overlap with the first embodiment will be omitted as appropriate, and differences from the first embodiment will be mainly described.

[0217] FIG. 16 is a diagram showing an example of the configuration of a management system 400 according to a fourth embodiment of the present invention, together with a diagram showing the work area A as viewed from above.

[0218] The management system 400 includes the same image capturing device 101 as in the first embodiment, and a management device 402 that replaces the management device 102 according to the first embodiment.

[0219] The management device 402 functionally includes an output control unit 408 that replaces the output control unit 108 according to the first embodiment. In addition, the management device 402 includes a registration storage unit 410. Except for these, the management device 402 may be configured similarly to the management device 102 according to the first embodiment.

[0220] The registration storage unit 410 is a storage unit for storing worker information 410a and unregistered information 410b.

[0221] The worker information 410a is information relating to the worker P. The worker information 410a is registered in the registration storage unit 410 in advance.

[0222] 17 is a diagram showing an example of the configuration of worker information 410a according to this embodiment. The worker information 410a associates a worker ID with an image feature amount.

[0223] The image feature amount is the image feature amount of the worker P indicated by the worker ID associated therewith. The image feature amount is, for example, a facial feature amount, but is not limited to this.

[0224] The unregistered information 410b is information about unregistered individuals. An unregistered individual is a person who enters and leaves work area A and is not registered in the worker information 410a.

[0225] 18 is a diagram showing an example of the configuration of unregistered information 410b according to this embodiment. The unregistered information 410b associates an unregistered ID, an image, and information related to behavior.

[0226] The unregistered ID is information for identifying an unregistered person.

[0227] The image is an image including an unregistered person identified by an unregistered ID associated therewith.

[0228] The information about the behavior is information about the behavior of the unregistered person indicated by the unregistered ID associated therewith.

[0229] The unregistered information 410b may include at least one of an image and information related to a behavior.

[0230] In addition to the functions of the estimation unit 104 according to the first embodiment, the estimation unit 404 generates unregistered information 410b about an unregistered person when a person included in the video captured of the working area A is an unregistered person. The estimation unit 404 stores the unregistered information 410b in the registration storage unit 410.

[0231] FIG. 19 is a diagram showing a detailed example of the functional configuration of the estimation unit 404 according to this embodiment.

[0232] The estimation unit 404 includes the operation estimation unit 112 and history generation unit 113 similar to those in the first embodiment, and an analysis unit 411 that replaces the analysis unit 111 according to the first embodiment.

[0233] The analysis unit 411 has the same functions as the analysis unit 411 according to the first embodiment. Furthermore, the analysis unit 411 determines whether or not the person included in the video of the work area A is the worker P. If the analysis unit 411 determines that the person is not the worker P, it generates unregistered information 410b and stores it in the registration storage unit 410.

[0234] Referring again to FIG. The output control unit 408 displays unregistered information 410b on the display unit 107 in addition to the history information 105a and the management information. The output control unit 408 also outputs electronic data including the unregistered information 410b. In detail, for example, the output control unit 408 makes a list of information about unregistered persons included in the unregistered information 410b, and displays the list of unregistered persons on the display unit 107 or outputs electronic data including the list of unregistered persons.

[0235] The management system 400 may be physically configured in the same manner as the management system 100 according to the first embodiment.

[0236] The management device 402 according to this embodiment executes management processing in the same manner as in embodiment 1. The management processing according to this embodiment includes estimation processing that is partially different from the estimation processing (step S102) according to embodiment 1.

[0237] Fig. 20 is a flowchart showing an example of the management process according to this embodiment. Fig. 20 shows an example of the management process according to this embodiment that differs from the estimation process (step S102) according to the first embodiment.

[0238] The analysis unit 411 executes steps S102a to S102b similar to those in the first embodiment.

[0239] The analysis unit 411 determines whether the person included in the video of the work area A is the worker P based on the worker information 410a (step S402a).

[0240] In more detail, for example, the analysis unit 411 determines whether or not the worker information 410a contains facial features that match the facial features of the person detected in step S102b.

[0241] If the worker information 410a does not contain any facial feature amount that matches the facial feature amount of the detected person, the analysis unit 411 determines that the person is not worker P. If the worker information 410a contains any facial feature amount that matches the facial feature amount of the detected person, the analysis unit 411 determines that the person is worker P.

[0242] If it is determined that the person is not the worker P (step S402a; No), the analysis unit 411 generates unregistered information 410b (step S402b). The analysis unit 411 stores the generated unregistered information 410b in the registration storage unit 410.

[0243] In detail, the analysis unit 411 assigns an unregistered ID to a person (unregistered person) determined not to be the worker P. At this time, the analysis unit 411 may assign the unregistered ID according to, for example, a predetermined rule. The analysis unit 411 associates information related to an image and behavior including the person (unregistered person) determined not to be the worker P with the unregistered ID to generate unregistered information 410b.

[0244] The image of the person determined not to be worker P is an image of the person included in the video captured of the work area A. This image is a still image and / or a moving image. This image includes at least one of a face image, a whole-body image, etc. Information related to behavior is, for example, movement feature amounts, posture feature amounts, and transitions or changes in posture feature amounts, but is not limited to these.

[0245] If it is determined that the person is worker P (step S402a; Yes), the analysis unit 411 performs the same processes from step S102c onwards as in embodiment 1. However, in step S102d, the analysis unit 111 may use the worker ID included in the worker information 410a as the worker ID to be assigned to the worker to be processed.

[0246] The fourth embodiment of the present invention has been described above.

[0247] (Actions and Effects) According to this embodiment, if a person included in the video captured of the work area A is an unregistered person who is not registered in the worker information 410a, unregistered information 410b relating to the unregistered person is generated.

[0248] This makes it possible to manage people other than worker P who enter and leave work area A. Therefore, it becomes possible to improve safety management, information management, and the like in work area A.

[0249] According to this embodiment, the unregistered information 410b includes at least one of an image including an unregistered person and information regarding the behavior of the unregistered person.

[0250] This makes it possible to manage people other than worker P who enter and exit work area A based on at least one of their images and behavior, thereby improving safety management, information management, and the like in work area A.

[0251] According to this embodiment, the output control unit 408 causes the display unit 107 to display the unregistered information 410b.

[0252] This allows the user to view the unregistered information 410b and manage people other than worker P who enter and leave work area A. This makes it possible to improve safety management, information management, and the like in work area A.

[0253] Although the embodiments and modifications of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0254] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps performed in each embodiment is not limited to the order shown. In each embodiment, the order of steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments and variations can be combined as long as the content is not contradictory.

[0255] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0256] 1. a master storage means for storing master information that associates a predetermined appearance of a worker or a group of workers performing a task with the type of task; and an estimation means for estimating the type of work to be performed by each of the one or more workers based on appearance information relating to the appearance of the one or more workers obtained by analyzing a video of the work area and the master information. Management device. 2. The predetermined appearance includes at least one of information showing the whole body, clothing, posture, and behavior of the one worker performing the work or the plurality of workers constituting the worker group, and information showing an item used for the work. 1. The management device described in 3. The worker group is made up of a plurality of workers who will jointly perform the work. The management device according to 1. or 2. 4. The predetermined appearance of the worker group performing the work includes at least one of the number of workers constituting the worker group and their positional relationship. 3. The management device according to any one of 1. to 3. 5. The apparatus further includes a management unit that outputs management information based on the result of estimation by the estimation unit. 5. The management device according to any one of 1. to 4. 6. The management information includes the number of workers who perform the same type of work. 5. The management device described in. 7. The management information includes the number of workers performing the same type of work by time period. 6. The management device described in. 8. The estimation means further estimates the work time of each of the one or more workers based on the appearance information and the master information. 8. The management device according to any one of 1. to 7. 9. the image is one of a plurality of images of the work area taken from different directions; The estimation means estimates the type of work to be performed by each of the one or more workers based on appearance information relating to the appearance of the one or more workers obtained by analyzing the plurality of images of the work area and the master information. 9. The management device according to any one of 1. to 8. 10. The estimation means further generates unregistered information about a person included in the video if the person is an unregistered person who is not registered in the worker information. 10. The management device according to any one of 1. to 9. 11. The unregistered information includes at least one of an image including the unregistered person and a behavior of the unregistered person. 10. The management device described in the above. 12. The device further includes an output control unit for displaying the unregistered information on a display unit. 10. The management device according to claim 11. 13. a management device according to any one of 1. to 12.; an imaging device that captures an image of the work area and transmits image data including the image of the work area; Management system. 14. The computer storing master information in a master storage means that associates a predetermined appearance of one worker or a group of workers performing a task with the type of task; and estimating the type of work to be performed by each of the one or more workers based on appearance information relating to the appearance of the one or more workers obtained by analyzing a video of the work area and the master information. Management method. 15. On the computer, storing master information that associates a predetermined appearance of a worker or a group of workers performing a task with the type of task; A program for estimating the type of work performed by each of one or more workers based on appearance information regarding the appearance of one or more workers obtained by analyzing footage of the work area and the master information. [Explanation of symbols]

[0257] 100,200,300,400 Management System 101 Imaging equipment 102,302,402 Management device 103,303 Master storage unit 103a, 303a Master information 104,304,404 Estimation part 105 History memory unit 105a Historical Information 106 Management Department 107 Display section 108,408 Output control section 109 Input reception section 111,311,411 Analysis Department 112,312 Work Estimation Department 113 History Generation Unit 410 Registration memory unit 410a Worker Information 410b Unregistered information A work area P,Pa worker

Claims

1. A master storage means for storing master information that associates a predetermined appearance of a worker group consisting of a plurality of workers who perform a group task with the type of the group task; an estimation means for estimating the type of group work to be performed by the plurality of workers based on appearance information relating to the appearances of the plurality of workers obtained by analyzing a video of the work area and the master information; The predetermined appearance includes at least one of the clothing of the workers constituting the worker group and information indicating an item used for the work, and the positional relationship of the workers constituting the worker group. Management device.

2. The predetermined appearance further includes at least one of information indicating the whole body, posture, and behavior of the plurality of workers constituting the worker group. The management device according to claim 1 .

3. The predetermined appearance of the worker group performing the work further includes the number of workers constituting the worker group. The management device according to claim 1 or 2.

4. The apparatus further includes a management unit that outputs management information based on the result of estimation by the estimation unit. The management device according to claim 1 .

5. The management information includes the number of workers who perform the same type of work. The management device according to claim 4 .

6. The management information includes the number of workers performing the same type of work by time period. The management device according to claim 5 .

7. The estimation means further estimates the work time of each of the plurality of workers based on the appearance information and the master information. The management device according to claim 1 .

8. The computer storing master information in a master storage means that associates a predetermined appearance of a worker group made up of a plurality of workers who will jointly perform a group task with the type of the group task; and estimating the type of group work to be performed by the plurality of workers based on appearance information relating to the appearances of the plurality of workers obtained by analyzing a video of the work area and the master information, The predetermined appearance includes at least one of the clothing of the workers constituting the worker group and information indicating an item used for the work, and the positional relationship of the workers constituting the worker group. Management method.

9. On the computer, storing master information that associates a predetermined appearance of a worker group made up of a plurality of workers who will jointly perform a group work with the type of the group work; estimating the type of group work to be performed by the plurality of workers based on appearance information relating to the appearances of the plurality of workers obtained by analyzing the video of the work area and the master information; The predetermined appearance is a program that includes at least one of the clothing of the multiple workers that make up the worker group, information indicating the items used for the work, and the positional relationship of the multiple workers that make up the worker group.

Images