Equipment monitoring device and equipment monitoring method

Abstract

In production processes carried out by production equipment, it supports monitoring of the operating status of production equipment, including the work logs of workers. [Solution] The equipment monitoring device includes an acquisition unit that acquires video data of an imaging area corresponding to production equipment and operation logs of production equipment collected by the production equipment; a detection unit that detects a specific person performing a predetermined task from the acquired video data; a work log generation unit that generates a work log of the specific person for the production equipment based on the detection time when the specific person was detected; and a performance log generation unit that generates and outputs a performance log showing the operation record of the production equipment including the predetermined task based on the generated work log and operation log.

Description

Technical Field

[0001] The present disclosure relates to an equipment monitoring device and an equipment monitoring method.

Background Art

[0002] In Patent Document 1, an operation analysis device is disclosed that analyzes the operation status of a production facility that processes or assembles a product to be produced or a production line to which the production facility is connected. The operation analysis device collects tact information including production tact actual values from the production facility or production line, and as a result of analyzing the collected tact information, if it is determined that there is a tact loss, which is an extra tact that extends beyond the standard production tact in the production operation of the production facility, the NC data for operating the production facility is corrected to reduce the tact loss and fed back to the production facility or production line.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In view of the above-described conventional circumstances, the present disclosure has been devised, and an object thereof is to provide an equipment monitoring device and an equipment monitoring method that assist in monitoring the operation status of a production facility including the work log of an operator in the production performed by the production facility.

Means for Solving the Problems

[0005] This disclosure provides an equipment monitoring device comprising: an acquisition unit that acquires video data of an imaging area corresponding to production equipment and an operation log of the production equipment collected by the production equipment; a detection unit that detects a specific person performing a predetermined task from the acquired video data; a work log generation unit that generates a work log of the specific person for the production equipment based on the detection time when the specific person is detected; and a performance log generation unit that generates and outputs a performance log showing the operation record of the production equipment including the predetermined task based on the generated work log and the operation log.

[0006] Furthermore, this disclosure provides an equipment monitoring method performed by one or more processors, which includes acquiring video data in which an imaging area corresponding to production equipment is captured and operation logs of the production equipment collected by the production equipment, detecting a specific person performing a predetermined task from the acquired video data, generating a work log of the specific person for the production equipment based on the detection time when the specific person was detected, and generating and outputting a performance log showing the operation record of the production equipment including the predetermined task based on the generated work log and operation log. [Effects of the Invention]

[0007] According to this disclosure, it is possible to support monitoring of the operating status of production equipment, including the work logs of workers, during production performed by the production equipment. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing an example of an equipment monitoring system according to Embodiment 1. [Figure 2] A diagram showing an example of factory equipment. [Figure 3] Block diagram showing an example of the internal configuration of the terminal device in Embodiment 1. [Figure 4] A diagram showing an example of an equipment monitoring system according to Embodiment 2. [Figure 5] Block diagram showing an example of the internal configuration of the terminal device in Embodiment 2. [Figure 6]Block diagram showing an example of the internal configuration of the terminal device in Embodiment 2. [Figure 7] Table showing the correspondence between terminal devices in Embodiments 1 and 2 [Figure 8] Diagram showing an example of the overall operation procedure of the terminal device according to Embodiments 1 and 2. [Figure 9] Figure showing an example of a switching time analysis procedure for terminal devices according to Embodiments 1 and 2. [Figure 10] Diagram illustrating an example of determining the switching start time. [Figure 11] Diagram illustrating an example of determining the switchover completion time. [Figure 12] This figure shows an example of the procedure for updating the performance log of a terminal device according to Embodiments 1 and 2. [Figure 13] A diagram showing an example of switching time data. [Figure 14] A diagram comparing the performance logs displayed on the conventional dashboard with the performance logs displayed on the dashboard in this disclosure. [Figure 15] A diagram illustrating the actual production status and an example of how it would be displayed on a conventional dashboard. [Modes for carrying out the invention]

[0009] (Background leading to this disclosure) Conventionally, there is a technology (Patent Document 1) that collects production cycle information (hereinafter referred to as "logs") performed by production equipment or production lines, analyzes the operating status of the production equipment or production line, and improves cycle loss in production. Here, the cycle of production equipment or production lines includes the cycle performed by the production equipment or production line itself, and the cycle performed by workers on the production equipment or production line (for example, work for switching products produced by the production equipment (hereinafter referred to as "switching work")). Tasks performed by workers in this way may include periods when the work is not performed, such as breaks or holidays. However, information (logs) about these periods when workers are not performing tasks could not be obtained from the logs of the production equipment or production line.

[0010] Here, referring to FIG. 15, a display example of a dashboard showing the operating status of production equipment or a production line based on the log of the production equipment or production line will be described. FIG. 15 is a diagram for explaining the actual operating status and an example displayed on a conventional dashboard. In FIG. 15, as an example, the log of production equipment that produces different types of substrates A and B respectively will be described.

[0011] When the switching time does not include a non-operating time zone, after the production of a predetermined number of substrates A is completed ( "Substrate A (final)" shown in FIG. 15), the production equipment is switched by an operator from the production of substrate A to substrate B. After the switching operation by the operator is completed, the production equipment starts the production of substrate B ( "Substrate B (first sheet~)" shown in FIG. 15). In such a case, the same log as the operating status of the production equipment is displayed on the dashboard.

[0012] On the other hand, when the switching time includes a non-operating time zone, after the production of a predetermined number of substrates A is completed ( "Substrate A (final)" shown in FIG. 15), the production equipment is switched from substrate A to substrate B by an operator. The switching operation is interrupted by the operator's break, holiday, etc. during the switching operation in the time zone of "no flag" shown in FIG. 15 and resumes after a long non-operating time zone. After the switching operation by the operator is completed, the production equipment starts the production of substrate B ( "Substrate B (first sheet~)" shown in FIG. 15). In such a case, in the log obtained from the production equipment, the log of switching from the production of substrate A ( "operation" shown in FIG. 15) to the switching operation and the log of the completion of the switching operation are recorded respectively, but the log regarding the non-operating time zone is not included. Therefore, on the dashboard, the time zone including the non-operating time zone is displayed as the time zone of the switching operation.

[0013] Therefore, in the present disclosure, an equipment monitoring device and an equipment monitoring method for assisting in monitoring the operating status of production equipment including the work log of an operator in the production performed by the production equipment will be described.

[0014] Hereinafter, each embodiment specifically disclosing the facility monitoring device and the facility monitoring method according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, detailed descriptions that are more detailed than necessary may be omitted. For example, detailed descriptions of well-known matters and duplicate descriptions of substantially the same configurations may be omitted. This is to avoid making the following descriptions unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the attached drawings and the following descriptions are provided for those skilled in the art to fully understand the present disclosure, and it is not intended to limit the subject matter described in the claims thereby.

[0015] <Configuration of the Facility Monitoring System According to Embodiment 1> First, referring to each of FIGS. 1 to 3, the facility monitoring system 100 according to Embodiment 1 will be described. FIG. 1 is a diagram showing an example of the facility monitoring system 100 according to Embodiment 1. FIG. 2 is a diagram showing an example of the factory devices L1, L2, L3, and L4. FIG. 3 is a block diagram showing an example of the internal configuration of the facility monitoring system 100 in Embodiment 1.

[0016] The facility monitoring system 100 according to Embodiment 1 is installed in the factory FCT, and includes each of the factory devices L1 to L4 that produce products, a terminal device P1 that generates a performance log indicating the operation performance of each of the factory devices L1 to L4, a cloud server CS that stores the performance logs of each of the factory devices L1 to L4, and a management terminal AP that generates and outputs a dashboard in which the performance logs of each of the factory devices L1 to L4 are visualized. In the present disclosure, as an example of the product, an example of producing a substrate on which electronic components are mounted will be described, but the product is not limited thereto. Also, in the present disclosure, as an example of the factory device, an example of analyzing the performance log of any one of a plurality of production devices included in one production line for producing a substrate will be described, but the factory device is not limited thereto.

[0017] The factory FCT includes at least one factory machine L1 to L4, at least one camera C1 to C2 that images a specific worker (hereinafter referred to as "specific worker") performing any operation (changeover operation in this disclosure) necessary for the factory machine L1 to L4 to perform production, and a factory terminal FP that collects logs from each of the factory machines L1 to L4 and transmits them to a terminal device P1. Note that the factory FCT shown in Figure 2 is an example and is not limited thereto.

[0018] Each of the factory equipment L1 to L4 is connected to the factory terminal FP via wireless or wired communication and transmits logs. In this disclosure, the logs of L10, L20, L30, and L40, which are production equipment included in factory equipment L1 to L4 and from which the produced circuit boards are discharged, are transmitted as logs for the production line, which is factory equipment L1 to L4. Factory equipment L1 to L4 produces circuit boards, which are the products, based on the production data. After the production of the circuit board currently being produced (e.g., "Circuit Board A") is completed, a specific worker performs a switchover operation to produce a different circuit board (e.g., "Circuit Board B").

[0019] Cameras C1 and C2 are installed within the factory FCT and capture images of their respective imaging areas AR1, AR2, AR3, and AR4, which correspond to factory equipment L1 to L4, the targets of performance log monitoring. Cameras C1 and C2 transmit the captured video data to terminal device P1. Alternatively, the video data may be transmitted to the factory terminal FP, and then from the factory terminal FP to terminal device P1.

[0020] In the example shown in Figure 2, camera C1 captures an area including the imaging area AR1 of factory equipment L1 and the imaging area AR2 of factory equipment L2. Camera C2 captures an area including the imaging area AR3 of factory equipment L3 and the imaging area AR4 of factory equipment L4.

[0021] Terminal device P1 is connected to the factory terminal FP, cameras C1-C2 installed in the factory FCT, and the cloud server CS, enabling data communication between them. Terminal device P1 acquires logs from factory equipment L1-L4 transmitted from factory terminal FP and converts the acquired logs into a data format compatible with the dashboard. Terminal device P1 also acquires video data transmitted from cameras C1-C2. Terminal device P1 generates performance logs for factory equipment L1-L4 using the detection results of specific workers detected in the video data and the logs of factory equipment L1-L4 after the data format conversion. Terminal device P1 sends (uploads) the generated performance logs for each of the factory equipment L1-L4 to the cloud server CS for storage.

[0022] The terminal device P1 includes a communication unit 10, a processor 11, and a memory 12. Although not shown in Figure 3, data transmission from the performance log generation unit 114B to the cloud server CS is performed via the communication unit 10.

[0023] The communication unit 10 is connected to the factory terminal FP, each of the cameras C1 to C2, and the cloud server CS to enable data communication and perform data transmission and reception. The communication unit 10 outputs various data acquired from the factory terminal FP and each of the cameras C1 to C2 to the processor 11. The communication unit 10 transmits the various data output from the processor 11 to the cloud server CS.

[0024] The processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU") or a Field Programmable Gate Array (hereinafter referred to as "FPGA"), and works in cooperation with the memory 12 to perform various processing and control. Specifically, the processor 11 refers to the programs and data held in the memory 12 and executes those programs to realize various functions such as the image acquisition unit 111, the movement path extraction unit 112, the equipment log conversion unit 113, and the production analysis unit 114.

[0025] The image acquisition unit 111 acquires video data transmitted from cameras C1 to C2 via the communication unit 10. The image acquisition unit 111 captures the image data contained in the acquired video data frame by frame and outputs it to the movement path extraction unit 112. The image acquisition unit 111 also stores the acquired video data in the video data Dt11 of the memory 12.

[0026] The movement path extraction unit 112 detects workers from each of the image data output from the image acquisition unit 111 and extracts the movement path of each detected worker. The movement path extraction unit 112 stores the extracted movement path data for each worker in the movement path data Dt12 of the memory 12.

[0027] The equipment log conversion unit 113 converts the logs of factory equipment L1 to L4 transmitted from the factory terminal FP via the communication unit 10 into a data format suitable for generating a dashboard. The equipment log conversion unit 113 stores the converted logs of factory equipment L1 to L4 in the factory equipment log data Dt13 of the memory 12.

[0028] The production analysis unit 114 generates performance logs for each of the factory machines L1 to L4 based on the data stored in the video data Dt11, movement data Dt12, and factory equipment log data Dt13 of the memory 12. The production analysis unit 114 includes a changeover time analysis unit 114A and a performance log generation unit 114B.

[0029] The switching time analysis unit 114A detects a specific worker performing a switching operation based on the video data Dt11 and the movement data Dt12, respectively. Based on the detection time of the detected specific worker, the switching time analysis unit 114A analyzes the time period during which the switching operation is being performed. The switching time analysis unit 114A generates switching time data (i.e., a log of the switching operation), which is the result of the analysis of the time period of the switching operation, and stores it in the switching time database Dt14 in the memory 12.

[0030] Furthermore, the switching time analysis unit 114A updates the switching time data stored in the switching time database Dt14 based on the analysis results of the latest switching work time period. The switching time analysis unit 114A stores the updated switching time data in the switching time database Dt14.

[0031] The performance log generation unit 114B generates performance logs showing the operating status of each of the factory machines L1 to L4 based on the switching time data stored in the switching time database Dt14 and the logs of each of the factory machines L1 to L4 stored in the factory machine log data Dt13. The performance log generation unit 114B sends (uploads) the generated performance log data to the cloud server CS for storage.

[0032] Memory 12 includes, for example, Random Access Memory (hereinafter referred to as "RAM"), which serves as work memory used when executing each process of the processor 11, and Read Only Memory (hereinafter referred to as "ROM"), which stores programs and data that define the operation of the processor 11. Data or information generated or acquired by the processor 11 is temporarily stored in RAM. Programs that define the operation of the processor 11 are written in ROM. Memory 12 stores video data Dt11, movement data Dt12, factory equipment log data Dt13, and switching time database Dt14, respectively. In Figures 3 and 5, blocks of each data are illustrated within the processor 11 for clarity of explanation.

[0033] The equipment monitoring system 100 according to Embodiment 1 described above is an example in which a single terminal device P1 generates performance data for factory equipment L1 to L4. The equipment monitoring system 100A according to Embodiment 2 will describe an example in which multiple terminal devices generate performance data for factory equipment L1 to L4.

[0034] The equipment monitoring system 100A according to Embodiment 2 has the same configuration and functions as the equipment monitoring system 100 according to Embodiment 1. Therefore, in the following description of the equipment monitoring system 100A according to Embodiment 2, the same configuration and functions as those of the equipment monitoring system 100 according to Embodiment 1 will be omitted.

[0035] <Configuration of the equipment monitoring system according to Embodiment 2> Next, the equipment monitoring system 100A according to Embodiment 2 will be described with reference to Figures 4 to 6. Figure 4 is a diagram showing an example of the equipment monitoring system 100A according to Embodiment 2. Figure 5 is a block diagram showing an example of the internal configuration of terminal device P1A in Embodiment 2. Figure 6 is a block diagram showing an example of the internal configuration of terminal devices P2 and P2A in Embodiment 2.

[0036] The equipment monitoring system 100A according to Embodiment 2 is installed in the factory FCT and includes each of the factory equipment L1 to L4 that produce products, a terminal device P1A that acquires logs for each of the factory equipment L1 to L4, terminal devices P2 and P2A that generate performance logs, a cloud server CS that stores the performance logs for each of the factory equipment L1 to L4, and a management terminal AP that generates and outputs a dashboard that visualizes the performance logs for each of the factory equipment L1 to L4.

[0037] As shown in Figure 7, the equipment monitoring system 100A according to Embodiment 2 is divided into two configurations: one including terminal devices P1A and P2, and another including terminal devices P1A and P2A, depending on the timing of converting the logs of factory equipment L1 to L4 into a data format suitable for generating a dashboard. When the equipment monitoring system 100A is configured to include terminal devices P1A and P2, the conversion of the data format of the logs of factory equipment L1 to L4 is performed by terminal device P1A. When the equipment monitoring system 100A is configured to include terminal devices P1A and P2A, the conversion of the data format of the logs of factory equipment L1 to L4 is performed by terminal device P2A.

[0038] First, we will describe the configuration of terminal devices P1A and P2 when the equipment monitoring system 100A includes terminal devices P1A and P2.

[0039] Terminal device P1A is connected to the factory terminal FP, cameras C1-C2 installed in the factory FCT, terminal device P2, and the cloud server CS, enabling data communication between them. Terminal device P1A acquires and stores logs from factory equipment L1-L4 transmitted from factory terminal FP and video data transmitted from cameras C1-C2. Based on the acquired video data, terminal device P1A calculates the movement of workers.

[0040] Terminal device P1A includes a communication unit 10A, a processor 11A, and a memory 12. Although not shown in Figure 5, data transmission between terminal device P1A and terminal devices P2 and P2A is performed via the communication unit 10A.

[0041] The communication unit 10A is connected to the factory terminal FP, each of the cameras C1 to C2, and the cloud server CS to enable data communication and perform data transmission and reception. The communication unit 10 outputs various data acquired from the factory terminal FP and each of the cameras C1 to C2 to the processor 11. The communication unit 10 transmits the various data output from the processor 11 to terminal devices P2 and P2A.

[0042] The processor 11A is configured, for example, using a CPU or FPGA, and works in cooperation with the memory 12 to perform various processing and control. Specifically, the processor 11A refers to the programs and data held in the memory 12 and executes those programs to realize various functions such as the image acquisition unit 111, the movement path extraction unit 112, and the equipment log conversion unit 113.

[0043] The equipment log conversion unit 113 converts the logs of each factory equipment L1 to L4 transmitted from the factory terminal FP into a data format corresponding to the dashboard. The equipment log conversion unit 113 stores the converted logs of each factory equipment L1 to L4 in the factory equipment log data Dt13 in the memory 12.

[0044] Terminal device P2 acquires the detection result of detecting a specific worker in the video data and the logs of factory equipment L1 to L4 after the data format has been changed, and generates performance logs for factory equipment L1 to L4. Terminal device P1 sends (uploads) the generated performance logs for factory equipment L1 to L4 to the cloud server CS for storage.

[0045] The terminal device P2 includes a communication unit 20, a processor 21, and a memory 22. Although not shown in Figure 5, data transmission from the performance log generation unit 114B to the cloud server CS is performed via the communication unit 20.

[0046] The communication unit 20 is connected to the terminal device P1A and the cloud server CS to enable data communication and performs data transmission and reception. The communication unit 20 outputs various data acquired from the factory terminal FP and cameras C1 to C2 to the processor 21. The communication unit 20 transmits the various data output from the processor 21 to the cloud server CS.

[0047] The processor 21 is configured, for example, using a CPU or FPGA, and works in cooperation with the memory 22 to perform various processing and control. Specifically, the processor 21 refers to the programs and data held in the memory 22 and executes those programs to realize various functions such as the production analysis unit 211. The processor 21 periodically acquires video data and movement data from the terminal device P1A and stores them in the video data Dt21 and movement data Dt22 of the memory 22, respectively. The processor 21 also periodically acquires logs from the factory equipment L1 to L4 from the terminal device P1A and stores them in the factory equipment log data Dt23 of the memory 22.

[0048] The production analysis unit 211 generates performance logs for each of the factory machines L1 to L4 based on the data stored in the video data Dt21, movement data Dt22, and factory equipment log data Dt23 of the memory 22. The production analysis unit 211 includes a switchover time analysis unit 211A, an equipment log conversion unit 211B, and a performance log generation unit 211C.

[0049] The switching time analysis unit 211A detects a specific worker performing a switching operation based on the video data Dt21 and the movement data Dt22, respectively. Based on the detection time of the detected specific worker, the switching time analysis unit 211A analyzes the time period during which the switching operation is being performed. The switching time analysis unit 211A generates switching time data, which is the result of the analysis of the time period of the switching operation, and stores it in the switching time database Dt24 in memory 22.

[0050] Furthermore, the switching time analysis unit 211A updates the switching time data stored in the switching time database Dt24 based on the analysis results of the latest switching operation time period. The switching time analysis unit 211A stores the updated switching time data in the switching time database Dt24.

[0051] The equipment log conversion unit 211B converts the logs of each factory equipment L1 to L4 stored in the factory equipment log data Dt23 in the memory 22 into a data format corresponding to the dashboard. The equipment log conversion unit 211B outputs the converted logs of each factory equipment L1 to L4 to the performance log generation unit 211C.

[0052] The performance log generation unit 211C generates performance logs for each of the factory equipment L1 to L4 based on the switching time data stored in the switching time database Dt24 and the logs for each of the factory equipment L1 to L4 stored in the factory equipment log data Dt23. The performance log generation unit 211C sends (uploads) the generated performance log data to the cloud server CS for storage.

[0053] Memory 22 includes, for example, RAM as work memory used when executing each process of processor 21, and ROM which stores programs and data that define the operation of processor 21. Data or information generated or acquired by processor 21 is temporarily stored in RAM. Programs that define the operation of processor 21 are written in ROM. Memory 22 stores video data Dt21, movement data Dt22, factory equipment log data Dt23, and switching time database Dt24, respectively. Note that in Figure 6, blocks of each data are shown within processor 21 for clarity of explanation.

[0054] Next, we will describe the configurations of terminal devices P1A and P2A when the equipment monitoring system 100A includes terminal devices P1A and P2A. Note that we will omit explanations of configurations or functions similar to those described above when the equipment monitoring system 100A includes terminal devices P1A and P2.

[0055] Terminal device P1A is connected to the factory terminal FP, cameras C1-C2 installed in the factory FCT, terminal device P2, and the cloud server CS, enabling data communication between them. Terminal device P1A acquires and stores logs from factory equipment L1-L4 transmitted from factory terminal FP and video data transmitted from cameras C1-C2. Based on the acquired video data, terminal device P1A calculates the movement path of a specific worker.

[0056] The terminal device P1A includes a communication unit 10A, a processor 11A, and a memory 12. In the processor 11A shown in Figure 5, the configuration of the device log conversion unit 211B is omitted.

[0057] After acquiring logs from each of the factory equipment L1 to L4, the processor 11A stores them in the factory equipment log data Dt13 without converting the data format of each log from the factory equipment L1 to L4.

[0058] Terminal device P2A acquires the detection result of detecting a specific worker in the video data from terminal device P1A, as well as the logs of factory equipment L1-L4 after the data format has been changed, and generates performance logs for factory equipment L1-L4. Terminal device P1 sends (uploads) the generated performance logs for each of the factory equipment L1-L4 to the cloud server CS for storage.

[0059] Terminal device P2A includes a communication unit 20, a processor 21A, and a memory 22. The processor 21A also has a production analysis unit 211 which includes a device log conversion unit 211B.

[0060] The equipment log conversion unit 211B converts the logs of each factory equipment L1 to L4 stored in the factory equipment log data Dt23 in the memory 22 into a data format corresponding to the dashboard. The equipment log conversion unit 211B outputs the converted logs of each factory equipment L1 to L4 to the performance log generation unit 211C.

[0061] Next, with reference to Figure 7, the configuration and functions of the terminal devices in Embodiment 1 and Embodiment 2 will be described. Figure 7 is a table TB showing the correspondence between terminal devices P1, P1A, P2, and P2A in Embodiments 1 and 2.

[0062] The equipment monitoring system 100 according to Embodiment 1 generates performance logs for each of the factory equipment L1 to L4 using a single terminal device P1, P1A. When terminal devices P1, P1A acquire the logs for each of the factory equipment L1 to L4 transmitted from the factory terminal FP, the equipment log conversion unit 113 converts the data format of the acquired logs for each of the factory equipment L1 to L4 into a data format corresponding to the dashboard of the management terminal AP.

[0063] The equipment monitoring system 100A according to Embodiment 2 generates performance logs for each of the factory equipment L1 to L4 using two terminal devices (specifically, terminal device P1A and terminal device P2, or terminal device P1A and terminal device P2A). Furthermore, when performance logs are generated using two terminal devices, the conversion of the data format of each log for factory equipment L1 to L4 is performed by terminal device P1A, which acquired the logs for each of the factory equipment L1 to L4 from the factory terminal FP, or by terminal device P2A, which acquired the logs for each of the factory equipment L1 to L4 from terminal device P1A.

[0064] As a result, the equipment monitoring systems 100 and 100A according to Embodiments 1 and 2 can generate performance logs converted into a data format suitable for generating dashboards.

[0065] <Overall operating procedure for terminal devices> Next, with reference to Figure 8, an example of the overall operation procedure for one or two terminal devices will be described. Figure 8 is a diagram showing an example of the overall operation procedure for terminal devices according to Embodiments 1 and 2. In Embodiment 1, the overall operation procedure shown in Figure 8 is executed by the switching time analysis unit 114A and the performance log generation unit 114B of terminal devices P1 and P1A, respectively, while in Embodiment 2, it is executed by the switching time analysis unit 211A and the performance log generation unit 211C of terminal devices P2 and P2A.

[0066] The switching time analysis units 114A and 211A acquire periodic execution commands (St11) at an execution timing different from that of the actual log update process (step St18) described later (for example, once every 15 minutes), for example, once every hour. The periodic execution command referred to here is a command (control command) that instructs the factory terminal FP to acquire logs showing the operating status of each of the factory equipment L1 to L4 installed in the factory FCT.

[0067] Based on the periodic execution command, the switching time analysis units 114A and 211A acquire video data (i.e., 1 hour's worth of video data) for each of the factory equipment L1 to L4 recorded in accordance with the execution timing of the periodic execution command from the video data Dt11 and Dt21 (St12). Based on the periodic execution command, the switching time analysis units 114A and 211A acquire worker movement data (i.e., 1 hour's worth of video data) recorded in accordance with the execution timing of the periodic execution command from the movement data Dt12 and Dt22 (St13).

[0068] The switching time analysis units 114A and 211A generate switching time data (i.e., switching operation logs) based on the acquired video data (St14). Here, the switching time data generation process (step St14) will be explained with reference to Figures 9 to 11. Figure 9 is a diagram showing an example of the switching time analysis procedure for terminal devices P1, P1A, P2, and P2A according to Embodiments 1 and 2. Figure 10 is a diagram illustrating an example of determining the switching start time Ts. Figure 11 is a diagram illustrating an example of determining the switching end time Te.

[0069] The switching time data generation process (step St14) is executed only when the brightness inside the factory FCT based on image data (video data) is above a certain level, that is, when it is determined that the factory FCT is operating. If the brightness inside the factory FCT based on image data (video data) is below a certain level, it may be omitted. Similarly, in such cases, the performance log update process (step St18), which will be described later, may also be omitted.

[0070] Furthermore, the switching time analysis units 114A and 211A may determine that a switching operation has been performed when they determine, based on the logs of factory equipment L1 to L4, that the products produced by each of the factory equipment L1 to L4 have changed, and may execute a switching time data generation process (step St14) and a performance log update process (step St18), described later, based on video data from the time when production of the previous product ended to the time when production of the next product started.

[0071] <Procedure for generating switching time data> The switching time analysis units 114A and 211A read the video data (image data) and movement path data for one frame from each of the multiple image data (frames) contained in the acquired video data (St141).

[0072] The switching time analysis units 114A and 211A perform AI image analysis on one frame of video data (image data) to detect a specific worker and detect features indicating that a switching operation is being performed in each of the imaging areas AR1 to AR4 of each factory equipment L1 to L4 that are visible in the image data (St142). Here, in this disclosure, the feature for detecting the switching operation is a specific worker wearing a hat of a specific color. Furthermore, feature detection may be performed by determining the distribution of colors using an RGB histogram. In addition, if cameras C1 and C2 are capable of capturing the worker's face, feature detection may be further performed by performing face recognition of the worker and based on the face recognition result.

[0073] The features used to detect the switching operation are not limited to the examples described above. The features only need to be detectable by image analysis, and may include, for example, the color or pattern of the clothing or hat worn by the worker, a 2D code or symbol (e.g., ☆, ◎, etc.) attached to the clothing or hat worn by the worker, or the color or lighting pattern of a lamp installed on the factory equipment or production line where the switching operation is performed. Furthermore, the features may also be the work actions of a specific worker that can be recognized by image analysis using AI.

[0074] The switching time analysis units 114A and 211A record the time data in which a specific worker, which is a characteristic indicating that a switching operation is being performed, is detected (St143). The switching time analysis units 114A and 211A determine whether or not the detection of the characteristic (specific worker) has been completed from all image data contained in the video data (St144).

[0075] If the switching time analysis units 114A and 211A determine that the detection of features (specific workers) from all image data is complete (St144, YES), they determine the start time of the switching time during which the switching work was performed (hereinafter referred to as the "switching start time") and the end time of the switching time during which the switching work was performed (hereinafter referred to as the "switching end time") (St145). The switching time analysis units 114A and 211A store (register) the switching time data (i.e., the switching work log) containing the information of the determined switching start time Ts or switching end time Te in the switching time databases Dt14 and Dt24 (St146).

[0076] Furthermore, the switching time analysis units 114A and 211A determine only the switching start time Ts corresponding to the video data, and if they determine that the switching end time Te cannot be determined, they determine the capture time of the last image data (frame) among the multiple image data included in the video data as the switching end time Te. The switching time analysis units 114A and 211A determine only the switching end time Te corresponding to the video data, and if they determine that the switching start time Ts cannot be determined, they determine the capture time of the first image data (frame) among the multiple image data included in the video data as the switching start time Ts.

[0077] Furthermore, if the switching time analysis units 114A and 211A cannot determine the switching start time Ts or switching end time Te because they do not meet the conditions described later, they may omit the processing in steps St145 to St146.

[0078] On the other hand, if the switching time analysis units 114A and 211A determine that the detection of features (specific workers) from all image data has not been completed (St144, NO), they return to the process of step St141.

[0079] Specifically, in determining the switching start time Ts, the switching time analysis units 114A and 211A count the number of times a specific worker is detected during a first predetermined time (e.g., 1 minute). If the switching time analysis units 114A and 211A determine that the number of times a specific worker is detected during the first predetermined time (e.g., 1 minute) is greater than or equal to a predetermined number (e.g., 10 times), they determine the time at which the image data of the first detection of the specific worker during this first predetermined time is captured as the switching start time Ts.

[0080] For example, in the example shown in Figure 10, the image data IMG11 captured at time t=0 only shows the unspecified worker HM11. Also, the image data IMG12 captured at time t=Ts and the image data IMG13 captured at time t=T11 show both the unspecified worker HM11 and the specified worker HM2. If the switching time analysis units 114A and 211A determine that a feature (specified worker) has been detected from the image data, they set the detection flag for the specified worker (detection flag = 1).

[0081] The switching time analysis units 114A and 211A determine a first predetermined time (e.g., 1 minute) in which the number of times a specific worker is detected (detection flag = 1) is equal to or greater than a predetermined number (e.g., 10 times), based on the detection flags derived from the analysis results of each image data. The switching time analysis units 114A and 211A determine the capture time of the image data in which the specific worker is first detected during the first predetermined time (e.g., 1 minute) in which the number of detections (detection flag = 1) is equal to or greater than a predetermined number, as the switching start time Ts.

[0082] Specifically, in determining the switching time completion time Te, the switching time analysis units 114A and 211A determine whether the number of times a specific worker is detected during a second predetermined time period (e.g., 10 minutes) is less than a predetermined number (e.g., 5 times). If the switching time analysis units 114A and 211A determine that the number of times a specific worker is detected during the second predetermined time period is less than a predetermined number, they count the number of times a specific worker is detected during a third predetermined time period (e.g., 10 minutes) in a time period prior to the second predetermined time period. If the switching time analysis units 114A and 211A determine that the number of times a specific worker is detected during the second predetermined time period is less than a predetermined number (e.g., 5 times), and the number of times a specific worker is detected during the third predetermined time period is 10 times or more, they determine the capture time of the image data in which the specific worker was first detected during the third predetermined time period (i.e., the image data in which the specific worker was last detected during the second predetermined time period) as the switching time completion time Te.

[0083] Furthermore, for example, in the example shown in Figure 11, the image data IMG21 captured at time t=Te within the second predetermined time period shows both the non-specified worker HM11 and the specified worker HM2. Also, the image data IMG22 captured at time t=21 shows only the non-specified worker HM11. If the switching time analysis units 114A and 211A determine that a feature (specified worker) has been detected from the image data, they set the detection flag for the specified worker (detection flag = 1).

[0084] The switching time analysis units 114A and 211A determine a second predetermined time period (e.g., 10 minutes) in which the number of times a specific worker has been detected is less than a predetermined number (e.g., 5 times), based on the detection flags derived from the analysis results of each image data. The switching time analysis units 114A and 211A determine a third predetermined time period that is in the past time period consecutive to the determined second predetermined time period, and in which the number of times a specific worker has been detected (detection flag = 1) is less than a predetermined number (e.g., 10 times). The switching time analysis units 114A and 211A determine the capture time of the image data in which the specific worker was first detected during the determined second predetermined time period (e.g., 5 minutes) as the switching end time Te.

[0085] After the switching time analysis unit 114A and 211A completes the switching time data generation process (step St14), they generate a control command instructing the start of the actual log update process and output it to the actual log generation unit 114B and 211C. In addition, the switching time analysis unit 114A and 211A automatically generate a control command instructing the start of the actual log update process (step St18) at the execution timing of the actual log update process (for example, once every 15 minutes) and output it to the actual log generation unit 114B and 211C.

[0086] The performance log generation units 114B and 211C acquire the logs for each of the factory equipment L1 to L4 stored in the factory equipment log data Dt13 and Dt23 in the memory 12 and 22 based on the control commands output from the switching time analysis units 114A and 211A (St16). The performance log generation units 114B and 211C also acquire the switching time data (i.e., information on the switching start time Ts and switching end time Te) stored in the switching time database Dt14 and Dt24 in the memory 12 and 22 (St17).

[0087] The performance log generation units 114B and 211C perform a performance log update process (St18) based on the factory equipment log data Dt13 and Dt23, and the switchover start time Ts and switchover end time Te, respectively. The performance log update process (step St18) is repeatedly executed based on a control command that instructs the start of the performance log update process, which is generated at a predetermined execution timing (for example, once every 15 minutes). Now, the process of updating the switchover time data (step St18) will be explained with reference to Figures 12 and 13, respectively. Figure 12 is a diagram showing an example of the performance log update procedure for terminal devices P1, P1A, P2, and P2A according to Embodiments 1 and 2. Figure 13 is a diagram showing an example of switchover time data.

[0088] Furthermore, the performance log update process (step St18) may be executed only if the switching time analysis unit 114A, 211A determines that the time period of the switching work analyzed is longer than a predetermined time (for example, 60 minutes). Also, the performance log update process (step St18) may be executed only if the switching time analysis unit 114A, 211A detects a specific worker, that is, determines that the switching work has been performed. In addition, an upper limit may be set on the number of time periods that are merged as non-switching time periods in a single switching operation, based on the length of the switching time period.

[0089] <Procedure for updating performance logs> The performance log generation units 114B and 211C read the logs of factory equipment L1 to L4 obtained from the factory equipment log data Dt13 and Dt23 (St181), and the switching time data obtained from the switching time databases Dt14 and Dt24 (St182).

[0090] The performance log generation units 114B and 211C merge switching time data (i.e., logs of switching operations, containing information on the switching start time Ts and switching end time Te) into the logs of each of the factory equipment L1 to L4 (St183) to generate performance logs for each of the factory equipment L1 to L4 that include the switching operation data (logs), or update the generated performance logs for each of the factory equipment L1 to L4 by merging the latest switching time data (St184). The performance log generation units 114B and 211C upload (send) the updated or generated performance logs for each of the factory equipment L1 to L4 to the cloud server CS (St185).

[0091] Specifically, the performance log generation units 114B and 211C generate logs of switching operations, indicating that switching operations were performed during the time period from the switching start time Ts to the switching end time Te, based on the read switching time data. The performance log generation units 114B and 211C generate a performance log by merging the generated switching operation logs with the logs or performance logs of each of the factory equipment L1 to L4. Here, we will explain the switching time data stored in the switching time databases Dt14 and Dt24.

[0092] The switching time data is the switching time data generated by a single switching time data generation process (step St14), and represents the switching start time Ts and switching end time Te respectively detected from one hour of video data. The switching time analysis units 114A and 211A generate the switching start time Ts and switching end time Te detected from one hour of video data as one set of switching time data. As an example, the switching time data is generated with the switching start time Ts and switching end time Te each in the data format of the switching start or switching end date "YYYYmmdd" and time "HHMMSS".

[0093] For example, the switching time data shown in Figure 13 includes three sets of switching time data from three days' worth of video data stored in switching time databases Dt14 and Dt24. The first set of switching time data indicates that the switching operation started on date "20230204" at time "021052" (switching start time Ts) and ended on date "20230205" at time "053129" (switching end time Te). The second set of switching time data indicates that the switching operation started on date "20230205" at time "223344" (switching start time Ts) and ended on date "20230206" at time "021532" (switching end time Te). The third set of switchover time data indicates that the switchover operation started on date "20230206" at time "102233" (switchover start time Ts) and ended on date "20230206" at time "141241" (switchover end time Te).

[0094] In the example shown in Figure 13, the performance log generation units 114B and 211C merge the information on the start time Ts and end time Te of the switchover time data from three sets of switchover time data from each of the logs or performance logs of factory equipment L1 to L4, with the log indicating that the time period from the start time Ts to the end time Te is a switchover operation, and then generate and update the performance data.

[0095] Furthermore, if the performance log generation units 114B and 211C have two consecutive switching time data sets in a time series, and the time between the switching end time Te of the first detected switching time data set and the switching start time Ts of the later detected switching time data set is within a certain period of time (for example, 5 minutes, 10 minutes, etc.), they may determine that the switching operations indicated by these two consecutive switching time data sets are a series of switching operations, and may merge (integrate) the switching start time Ts of the first detected switching time data set and the switching end time Te of the later detected switching time data set into a single set of switching time data.

[0096] For example, in the example shown in Figure 13, if the performance log generation units 114B and 211C determine that the time between the first set of switching time data and the second set of switching time data is within 5 minutes, they may merge the first set of switching time data and the second set of switching time data to generate a single switching time data indicating that the switching operation started on date "20230204" and time "021052" (switching start time Ts), and ended on date "20230206" and time "021532" (switching end time Te). This merging process may also be performed by the switching time analysis units 114A and 211A. As a result, even if a single switching operation is analyzed as two separate switching operations due to reasons such as a failure to detect a specific person or the specific person being partially obscured by cameras C1 and C2, the merging allows the performance log generation units 114B and 211C to recognize these two switching operations as a single switching operation. Therefore, the performance log generation units 114B and 211C can analyze the time period of the switching operation with greater accuracy.

[0097] In the merge process described above, if the actual log generation units 114B and 211C determine that the time between the start time Ts based on the switching time data and the end time of the work performed before the switching operation (e.g., production of board A), or the time between the end time Te based on the switching time data and the start time of the work performed after the switching operation (e.g., production of board B), is within a predetermined time (e.g., less than 15 minutes), they may determine that the logs from factory equipment L1 to L4 obtained from factory equipment L1 to L4 are more accurate than the switching time data analyzed based on the video data. The predetermined time here is set to a time that can be ignored as an error in the actual log. In such cases, the actual log generation units 114B and 211C use the logs from factory equipment L1 to L4 as they are and generate an actual log in which the switching operation started from the end time of the work performed before the switching operation (e.g., production of board A), or an actual log in which the switching operation ended at the start time of the work performed after the switching operation (e.g., production of board B).

[0098] As described above, terminal devices P1, P1A, P2, and P2A in Embodiment 1 and Embodiment 2 can acquire performance logs of production equipment or production lines, including logs of tasks performed by specific workers that are difficult to acquire by production equipment or production lines (in this disclosure, factory equipment L1 to L4) (in this disclosure, switching operations). As a result, the management terminal AP can generate a dashboard that visualizes the operating status of production equipment or production lines based on the generated performance logs of production equipment or production lines, thereby supporting the management work of managers who manage production equipment or production lines.

[0099] Furthermore, as a result, terminal devices P1, P1A, P2, and P2A in Embodiments 1 and 2 can generate performance logs that distinguish between periods when work is being performed and periods when work is interrupted, even if the work performed by a specific worker (in this disclosure, switching work) is temporarily interrupted due to breaks or holidays, by determining the switching start time Ts and the switching end time Te. Therefore, the management terminal AP can distinguish and visualize the periods when work is being performed by a specific worker and periods when work is not being performed due to breaks or holidays of that specific worker, based on the generated performance logs of the production equipment or production line, thereby obtaining more accurate performance logs of the production equipment or production line.

[0100] The method for determining the switching start time Ts or switching end time Te described above is not limited to the examples given. For example, the switching time analysis units 114A and 211A may determine the switching start time Ts or switching end time Te based on the detection probability of a specific worker, which is determined from multiple image data captured during a predetermined time period, based on the number of image data in which a specific worker (feature) is detected, or based on the number of image data in which a specific worker (feature) is detected consecutively. Alternatively, the switching start time Ts or switching end time Te may be determined based on the number of specific workers (features) detected from a single image data. For example, if the switching time analysis units 114A and 211A determine that the detection probability of a specific worker is greater than or equal to a predetermined probability, that the number of image data in which a specific worker (feature) is detected consecutively is greater than or equal to a predetermined number, or that the number of specific workers (features) is greater than or equal to a predetermined number, they will determine the switching start time Ts based on this determined timing. Furthermore, for example, if the switching time analysis units 114A and 211A determine that the detection probability of a specific worker is less than a predetermined probability, the number of image data in which a specific worker (feature) is detected consecutively is less than a predetermined number, or the number of specific workers (features) is less than a predetermined number, they determine the switching end time Te based on this determined timing.

[0101] Furthermore, in the above example, the switching time analysis units 114A and 211A determine the switching start time Ts when the number of detected specific workers or the detection probability, etc., exceeds a predetermined value, and determine the switching end time Te when the number of detected specific workers or the detection probability, etc., falls below a predetermined value. However, the system is not limited to this example. The switching start time Ts and switching end time Te may be determined by arbitrary conditions depending on the content (process) of the switching work, the number of people, etc. For example, if the switching work is performed by first entering and exiting the imaging area and finally remaining inside the imaging area, the conditions for determining the switching start time Ts and switching end time Te may be the reverse of the conditions shown in the examples in Figures 10 and 11.

[0102] Furthermore, for example, when the switching time analysis units 114A and 211A detect a specific worker, they determine the time when the movement data corresponding to that specific worker was first generated within the imaging area of ​​the corresponding factory equipment as the switching start time Ts, based on the movement data corresponding to that specific worker. This allows for the analysis of the time period during which the switching work was performed, even if the specific worker forgot to put on their distinctive hat and put it on partway through the switching work.

[0103] Furthermore, the switching time analysis units 114A and 211A may improve the detection accuracy of specific workers by retraining the AI ​​using image data in which specific workers were detected.

[0104] Next, with reference to Figure 14, the performance logs generated by terminal devices P1, P1A, P2, and P2A will be explained. Figure 14 is a comparison of the performance log DB11 displayed on the conventional dashboard and the performance log DB12 displayed on the dashboard of this disclosure. In Figure 14, "Start" indicates the switchover start time Ts, and "End" indicates the switchover end time Te.

[0105] In the example shown in Figure 14, the factory equipment (production equipment) operates from 7:00 to 7:22 to produce board A, after which a specific worker performs a switchover operation for the production of board B. After the switchover operation by the specific worker is completed, the factory equipment (production equipment) starts producing board B at 9:02. Furthermore, after the production of board A is completed, the specific worker performs a predetermined switchover operation from 7:22 to 7:36, and then, after a predetermined time interval, performs a switchover operation for the production of board B from 7:48:45 to 8:37:41, and after a predetermined time interval, allows the factory equipment (production equipment) to start producing board B.

[0106] In this case, after the production of board A is completed, the factory equipment (production equipment) generates and records a log at the time "7:22" indicating that the production of board A has been completed, and records a log indicating that the predetermined switchover operation started at the time "7:22". The factory equipment (production equipment) records a log indicating that the switchover operation for another model started at the time "7:22", and records a log indicating that the switchover operation for the other model was completed at the time "7:36", as well as a log indicating that the switchover operation for producing board B had started. At the time "9:02", the factory equipment (production equipment) records a log indicating that the switchover operation for producing board B was completed, as well as a log indicating that the production of board B had started.

[0107] Therefore, the performance log DB11 displayed on the dashboard, which is generated based on logs sent from factory equipment (production equipment), does not show the time periods when the switching operation is interrupted.

[0108] In this disclosure, terminal devices P1, P1A, P2, and P2A analyze the time period during which the switching work necessary for the production of substrate A to substrate B is performed, based on the detection results of features (specific workers) based on video data. In the example shown in Figure 14, terminal devices P1, P1A, P2, and P2A generate switching time data (i.e., a log of the switching work) where the switching start time Ts is "07:48:45" and the switching end time Te is "08:37:58", and merge it with the log of the factory equipment (production equipment).

[0109] Specifically, terminal devices P1, P1A, P2, and P2A record the time "7:36" in the log where the switchover work for producing board B began, and merge it with the time "07:48:45" (switchover start time Ts) based on the switchover time data. They also merge the time "9:02" in the log where the switchover work for producing board B ended with the time "08:37:41" (switchover end time Te) based on the switchover time data.

[0110] As a result, the performance log DB12 displayed on the dashboard, which is generated based on the performance logs generated by terminal devices P1, P1A, P2, and P2A, will show the time period from "07:48:45" to "08:37:58" as the time period for the switchover work. In addition, the performance log DB12 will show the time from "7:36", when the switchover work for another model was completed, to "07:48:45", when the switchover work for the production of board B began, and the time from "08:37:41", when the switchover work for the production of board B ended, to "9:02", when production of board B began, as time periods when the factory equipment (production equipment) was not operating for production, i.e., when it was stopped.

[0111] (Note) Based on the descriptions of the embodiments described above, the following technologies are disclosed.

[0112] (Technology 1) An acquisition unit (communication unit 10, 10A) acquires video data of imaging areas AR1 to AR4 corresponding to production equipment (factory equipment L1 to L4) and operation logs of the production equipment (logs of factory equipment L1 to L4) collected by the production equipment (factory equipment L1 to L4), A detection unit (switching time analysis unit 114A) detects a specific person (specific worker) performing a predetermined task from the acquired video data, Based on the detection time when the specified person (specified worker) was detected, a work log generation unit (performance log generation unit 114B) generates work logs (switching time data) of the specified person (specified worker) for the production equipment (factory equipment L1 to L4), The system includes a performance log generation unit 114B that generates and outputs a performance log showing the operational performance of the production equipment (factory equipment L1 to L4) including the predetermined work, based on the generated work log (switching time data) and the operation log (logs of factory equipment L1 to L4). Equipment monitoring device (terminal devices P1, P1A). As a result, terminal devices P1 and P1A can acquire performance logs of production equipment or production lines, including work logs of tasks performed by specific workers (specific workers) that are difficult to obtain by production equipment or production lines (in this disclosure, factory equipment L1 to L4). Furthermore, even if a specific worker interrupts their work for a long period of time due to a break, holiday, etc., terminal devices P1 and P1A can acquire logs of the time period in which the specific worker actually performed the work, excluding the interrupted time. Therefore, terminal devices P1 and P1A can support monitoring of the operating status of production equipment, including worker work logs.

[0113] (Technology 2) The system further includes a work time recording unit (switching time database Dt14) that records the start time (switching start time Ts) and end time (switching end time Te) of the predetermined work, The work log generation unit (performance log generation unit 114B) determines the start time (switching start time Ts) and end time (switching end time Te) of the predetermined work based on the detection time of the specific person (specific worker), and records them in the work time recording unit (switching time database Dt14). (Technical 1) Equipment monitoring device (terminal devices P1, P1A). As a result, terminal devices P1 and P1A can determine the switching start time Ts and switching end time Te, and even if the work performed by a specific worker (switching work in this disclosure) is temporarily interrupted due to a break or holiday, etc., they can generate a performance log that distinguishes between the time period during which work is being performed and the time period during which work is interrupted, based on the switching start time Ts and switching end time Te before the work interruption and the switching start time Ts and switching end time Te after the work interruption, thereby obtaining a more accurate performance log of the production equipment or production line.

[0114] (Technology 3) The work log generation unit (performance log generation unit 114B) counts the number of times the specific person (specific worker) is detected, and if it determines that the number of detections within a first predetermined time is equal to or greater than a first threshold, it determines the timing at which the specific person (specific worker) was first detected within the first predetermined time as the work start time (switching start time Ts). (Technical 2) Equipment monitoring device (terminal devices P1, P1A). As a result, terminal devices P1 and P1A can more effectively suppress false detections of whether or not a switchover operation is being performed due to a specific worker's erroneous detection, or false detections of the switchover start time Ts.

[0115] (Technology 4) The work log generation unit (performance log generation unit 114B) counts the number of times the specific person (specific worker) is detected, and if it determines that the number of detections within the second predetermined time is less than the second threshold, it determines the timing at which the specific person (specific worker) was first detected within the second predetermined time as the work end time (switching end time Te). Equipment monitoring device (terminal device P1, P1A) as described in (Technology 2) or (Technology 3). As a result, terminal devices P1 and P1A can more effectively suppress erroneous detection of the switching completion time Te for switching operations due to erroneous detection of specific workers.

[0116] (Technology 5) The system further includes an operation log recording unit (factory equipment log data Dt13) that records the operation logs (logs of factory equipment L1 to L4) of the aforementioned production equipment (factory equipment L1 to L4), The work log generation unit (performance log generation unit 114B) generates the performance log based on the operation log (logs of factory equipment L1 to L4) recorded in the operation log recording unit (factory equipment log data Dt13) and the start time of the predetermined work (switching start time Ts) and the end time of the work (switching end time Te) of the predetermined work recorded in the work time recording unit (factory equipment log data Dt13). Equipment monitoring device (terminal device P1, P1A) as described in any one of (Technology 2) to (Technology 4). As a result, terminal devices P1 and P1A can generate an actual log by merging the logs of factory equipment L1 to L4 with the logs of the switching operation (i.e., information on the switching start time Ts and switching end time Te).

[0117] (Technology 6) The detection unit (switching time analysis unit 114A) detects the specific person (specific worker) from each of the multiple image data included in the video data. Equipment monitoring device (terminal device P1, P1A) as described in any one of (Technology 1) to (Technology 5). This allows terminal devices P1 and P1A to detect specific workers from the image data.

[0118] (Technology 7) The work log generation unit (performance log generation unit 114B) determines that the number of specific persons (specific workers) detected by the detection unit (switching time analysis unit 114A) is equal to or greater than a predetermined number, and generates the work log (switching time data) based on the detection time at which the predetermined number of specific persons (specific workers) were detected. Equipment monitoring device (terminal device P1, P1A) as described in any one of (Technology 1) to (Technology 6). As a result, terminal devices P1 and P1A can detect a switchover operation based on whether the required number of specific workers for the switchover operation are present and whether the necessary specific workers are ready to perform the switchover operation.

[0119] (Technology 8) The detection unit (switching time analysis unit 114A) generates movement data of the detected specific person (specific worker), The work log generation unit (performance log generation unit 114B) generates the work log (switching time data) in which the time the specific person (specific worker) entered the imaging area AR1 to AR4 is determined to be the start time of the predetermined work (switching start time), based on the movement data of the specific person (specific worker). Equipment monitoring device (terminal device P1, P1A) as described in any one of (Technology 1) to (Technology 7). As a result, terminal devices P1 and P1A can analyze the time period during which the switchover work was performed, based on the movement data of the specific worker, even if the specific worker forgot to wear their distinctive hat and put it on partway through the switchover process.

[0120] (Technology 9) The performance log generation unit 114B, based on the generated work log (switching time data), determines that the time of the predetermined work is equal to or greater than the predetermined work time (e.g., 60 minutes), and generates the performance log based on the work log (switching time data) and the operation log (logs of factory equipment L1 to L4). If it determines that the time of the predetermined work is less than the predetermined work time, it outputs the operation log (logs of factory equipment L1 to L4) as the performance log. Equipment monitoring device (terminal device P1, P1A) as described in any one of (Technology 1) to (Technology 8). As a result, terminal devices P1 and P1A can only generate an actual log by merging the switching time data with the logs from factory equipment L1 to L4 if they determine that the switching operation is longer than a predetermined time (e.g., 60 minutes) and may include non-working time such as breaks or holidays for specific workers.

[0121] (Technology 10) When the performance log generation unit (performance log generation unit 114B) generates multiple work logs (switching time data), it calculates the gap time between two consecutive work logs (switching time data) in the time series and, if it determines that the calculated gap time is less than a predetermined gap time (for example, 5 minutes, 10 minutes, etc.), it generates one work log (switching time data) by merging the two work logs (switching time data). An equipment monitoring device (terminal device P1, P1A) described in any one of (Technology 1) to (Technology 9). As a result, terminal devices P1 and P1A can merge the time between two consecutive switching operations if the time between them (gap time) is less than a predetermined time (i.e., short). In other words, even if a single switching operation is analyzed as two separate operations due to reasons such as a failure to detect a specific person or the specific person being partially obscured by cameras C1 and C2, terminal devices P1 and P1A can recognize these two switching operations as a single operation through merging. Therefore, terminal devices P1 and P1A can analyze the timing of switching operations with greater accuracy.

[0122] (Technology 11) A method for monitoring equipment performed by one or more processors 11, 11A, 21, 21A, The system acquires video data of imaging areas AR1 to AR4 corresponding to production equipment (factory equipment L1 to L4), and operation logs of the production equipment (factory equipment L1 to L4) collected by the production equipment (factory equipment L1 to L4 logs). From the acquired video data, a specific person (specific worker) performing a predetermined task is detected. Based on the detection time when the aforementioned specific person (specific worker) was detected, a work log (switching time data) of the aforementioned specific person (specific worker) for the production equipment (factory equipment L1~L4) is generated. Based on the generated work log (switching time data) and the operation log (logs of factory equipment L1 to L4), a performance log showing the operational performance of the production equipment (factory equipment L1 to L4) including the predetermined work is generated and output. Equipment monitoring method. As a result, processors 11, 11A, 21, and 21A can acquire performance logs of production equipment or production lines, including work logs of tasks performed by specific workers (specific workers) that are difficult to obtain by production equipment or production lines (in this disclosure, factory equipment L1 to L4). Furthermore, even if a specific worker interrupts their work for a long period of time due to a break, holiday, etc., processors 11, 11A, 21, and 21A can acquire logs of the time period during which the specific worker actually performed the work, excluding the interrupted time. Therefore, processors 11, 11A, 21, and 21A can support monitoring of the operating status of production equipment, including worker work logs.

[0123] Although various embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these will also be understood to fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention. [Industrial applicability]

[0124] This disclosure is useful as an equipment monitoring device and equipment monitoring method for supporting the monitoring of the operating status of production equipment, including the work logs of workers, in production performed by production equipment. [Explanation of Symbols]

[0125] 10,10A,20 Communications Department 11, 11A, 21, 21A processors 12.22 memory 100,100A Equipment Monitoring System 111 Image acquisition unit 112 Flow line extraction part 113,211B Equipment log conversion unit 114,211 Production Analysis Department 114A, 211A Switching Time Analysis Unit 114B, 211C Performance log generation unit AP Management Terminal AR1, AR2, AR3, AR4 imaging area C1, C2 Camera CS Cloud Server DB11, DB12 Performance Log Dt11, Dt21 video data Dt12, Dt22 Movement path data Dt13, Dt23 Factory Equipment Log Data Dt14, Dt24 Switching Time Database FCT Factory FP factory terminal HM2 Specific worker IMG11, IMG12, IMG13, IMG21, IMG22 Image data L1,L2,L3,L4 Factory equipment P1, P1A, P2, P2A Terminal Devices

Claims

1. An acquisition unit that acquires video data of an imaging area corresponding to a production machine and an operation log of the production machine collected by the production machine, A detection unit that detects a specific person performing a predetermined task from the acquired video data, A work log generation unit generates a work log of the specified person to the production equipment based on the detection time when the specified person was detected, The system includes a performance log generation unit that generates and outputs a performance log showing the operational performance of the production equipment, including the predetermined work, based on the generated work log and the operation log. Equipment monitoring device.

2. The system further includes a work time recording unit that records the start time and end time of the predetermined work, The work log generation unit determines the start time and end time of the predetermined work based on the detection time of the specific person, and has the work time recording unit record them. The equipment monitoring device according to claim 1.

3. The work log generation unit counts the number of times the specific person is detected, and if it determines that the number of detections within a first predetermined time is equal to or greater than a first threshold, it determines the timing at which the specific person was first detected within the first predetermined time as the work start time. The equipment monitoring device according to claim 2.

4. The work log generation unit counts the number of times the specific person is detected, and if it determines that the number of detections within a second predetermined time is less than a second threshold, it determines the timing at which the specific person was first detected within the second predetermined time as the work completion time. The equipment monitoring device according to claim 2.

5. The machine further comprises an operation log recording unit for recording the operation logs of the aforementioned production equipment, The work log generation unit generates the actual log based on the operation log recorded in the operation log recording unit and the start time and end time of the predetermined work recorded in the work time recording unit. The equipment monitoring device according to claim 2.

6. The detection unit detects the specific person from each of the multiple image data contained in the video data. The equipment monitoring device according to claim 1.

7. If the work log generation unit determines that the number of specific persons detected by the detection unit is equal to or greater than a predetermined number, it generates the work log based on the detection time for which the predetermined number of specific persons were detected. The equipment monitoring device according to claim 1.

8. The detection unit generates movement data of the detected specific person, The work log generation unit generates the work log in which the time the specific person entered the imaging area is determined to be the start time of the predetermined work, based on the movement data of the specific person. The equipment monitoring device according to claim 1.

9. The performance log generation unit, based on the generated work log, determines that the time for the predetermined work is equal to or greater than the predetermined work time, and generates the performance log based on the work log and the operation log. If it determines that the time for the predetermined work is less than the predetermined work time, it outputs the operation log as the performance log. The equipment monitoring device according to claim 1.

10. When the work log generation unit generates multiple work logs, it calculates the gap time between two consecutive work logs in the time series and, if it determines that the calculated gap time is less than a predetermined gap time, it generates a single work log by merging the two work logs. The equipment monitoring device according to claim 1.

11. A method for monitoring equipment performed by one or more processors, The system acquires video data of the imaging area corresponding to the production equipment and the operation logs of the production equipment collected by the production equipment. From the acquired video data, a specific person performing a predetermined task is detected, Based on the detection time when the specified person was detected, a work log of the specified person for the production equipment is generated. Based on the generated work log and operation log, a performance log showing the operation performance of the production equipment including the predetermined work is generated and output. Equipment monitoring method.

Images