Component mounting system, image processing device, image processing method, and image processing system

Abstract

This component mounting system comprises: a plurality of production equipment installed on a component mounting line for mounting components on substrates; imaging units installed on the component mounting line for imaging one or a plurality of workers deployed to the plurality of production equipment and the component mounting line; and an image processing unit for performing image processing on images captured by the imaging units in a prescribed production period when products are produced on the component mounting line. The image processing unit displays at least one person from among the one or plurality of workers in a manner where it is possible to visualize the work status of the worker for at least one of the plurality of production equipment at each of a preset plurality of points in time or plurality of periods within the production period.

Description

[Technical field]

[0001] The present specification relates to a component mounting line that mounts components on a board, and more particularly to a technology for visualizing the working status of workers deployed on the component mounting line. [Background technology]

[0002] A component mounting line where components are mounted on a board is equipped with multiple production equipment (e.g., a printer, a component mounter, a reflow oven, a board inspection machine, etc.). The board is transported to the multiple production equipment in sequence, and each production equipment performs a predetermined process on the board, thereby mounting components on the board. In order to mount components on the board, each production equipment requires preparation work (e.g., supplying raw materials (components, etc.)). For this reason, workers are assigned to the component mounting line, and the workers perform the necessary work depending on the operating status of the production equipment. In order to improve the production efficiency of the component mounting line, the working status of the workers must be appropriate for the operating status of the production equipment. For this reason, a technology for evaluating the working status of the workers has been developed (e.g., JP 2019-191748 A). Summary of the Invention [Problem to be solved by the invention]

[0003] In the technology of the above publication, the production equipment to be analyzed and the worker who works on the production equipment are photographed. Then, the photographed image is analyzed to obtain the operating status of the production equipment and the position and orientation of the worker, and it is determined whether the worker's work on the production equipment is appropriate. Although the technology of the above publication can evaluate whether the work status of the worker on the production equipment to be analyzed is appropriate or not, it is common for multiple production equipment to be installed on a component mounting line, and workers work on these multiple production equipment. Therefore, even if the work status on one production equipment is appropriate, the work status on other production equipment is not necessarily appropriate. The technology of the above publication could not evaluate whether the work status of the worker is appropriate or not for multiple production equipment installed on the component mounting line.

[0004] This specification discloses a technique for evaluating whether the working conditions of a worker are appropriate for a plurality of production equipments installed in a component mounting line. [Means for solving the problem]

[0005] The component mounting system disclosed in this specification comprises: a plurality of production equipments installed in a component mounting line where components are mounted on a board; a photography unit installed in the component mounting line and photographing the plurality of production equipments and one or more workers stationed in the component mounting line; and a display unit that visualizes and displays, for at least one of the one or more workers, the work status of that worker for at least one of the plurality of production equipments at each of multiple time points or multiple periods that are preset within the production period by processing images photographed by the photography unit during a predetermined production period in which products are produced on the component mounting line.

[0006] In the above component mounting system, multiple pieces of production equipment and one or more workers deployed on the component mounting line are photographed, and the photographed images are processed to visualize the working status of the workers. By photographing multiple pieces of production equipment, it is possible to evaluate whether the work of the workers is appropriate in relation to the operating status of these multiple pieces of production equipment. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a component mounting system. [Diagram 2] FIG. 4 is a flowchart showing a process executed by the image processing apparatus. [Diagram 3] FIG. 3 is a flowchart showing the image processing executed in S20 of FIG. 2. [Figure 4] FIG. 11 is a diagram for explaining a procedure for calculating a distance between an operator and production equipment. [Diagram 5] FIG. 13 is a diagram showing an example of a time chart visualizing the work status of an operator. [Figure 6] FIG. 13 is a diagram showing another example of a time chart visualizing the work status of an operator. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The technology disclosed in this specification may further include an operation status acquisition unit that acquires the operation status of at least one of the multiple production equipment at multiple points in time or multiple periods. The display unit may display the acquired operation status of the production equipment in correspondence with the work status. With this configuration, the operation status of the production equipment and the work status of the worker are displayed in correspondence with each other, so that it is easy to evaluate whether the work status of the worker is appropriate.

[0009] In the technology disclosed in this specification, the operation status acquisition unit may acquire the operation status by image processing. For example, some production equipment is equipped with a signal light that notifies a worker that a specific task is required. In such a case, when photographing the production equipment, the signal light is photographed together with the production equipment, so that the operation status of the production equipment can be acquired.

[0010] In the technology disclosed in this specification, at least one of the multiple production facilities may include a status output unit that outputs its own status. The operation status acquisition unit may acquire the operation status from the status output from the status output from the status output unit. With this configuration, the operation status of the production facility can be accurately acquired based on the information output from the production facility.

[0011] The technology disclosed in this specification may further include a determination unit that determines whether or not the worker's work is delayed based on the transition of the operating status of the production equipment and the transition of the worker's work status. With this configuration, the determination unit determines whether or not the worker's work is delayed, making it easier to evaluate the worker's work status.

[0012] The technology disclosed in this specification may further include a first designation device that designates a production facility to be analyzed from among a plurality of production facilities. The operation status acquisition unit may acquire the operation status of the production facility designated by the first designation device. With this configuration, the operation status of the worker for the designated production facility can be evaluated.

[0013] The technology disclosed in this specification may further include a second designation device that designates a worker to be analyzed from one or more workers. The display unit may visualize and display the work status of the worker designated by the second designation device. With this configuration, the work status of the designated worker can be evaluated.

[0014] In the image processing disclosed in this specification, when multiple pieces of production equipment are captured near a worker at multiple times or multiple periods, the distance between the worker and each piece of production equipment may be calculated, and it may be determined that the worker is working on the production equipment with the smallest distance among the calculated distances. With this configuration, even if multiple pieces of production equipment are captured in the captured image, it is possible to identify the production equipment on which the worker is working.

[0015] In the image processing disclosed in this specification, when it is determined that a worker is working on a production facility having a minimum distance, if the time the worker has been working on the production facility is less than a predetermined time, the determination that the worker is working on the production facility may be discarded, and if the time the worker has been working on the production facility is equal to or longer than the predetermined time, the determination that the worker is working on the production facility may be maintained. With this configuration, even if a worker is photographed simply passing in front of a production facility, it is possible to avoid the worker being determined to be working on the production facility.

[0016] In the image processing disclosed in this specification, when it is determined that a worker is performing work on a specific production facility, the content of the work performed by the worker on the specific production facility may be identified based on the operation status of the specific production facility. With this configuration, the work content of the worker is identified taking into account the operation status of the production facility, so that the work status of the worker can be accurately evaluated.

[0017] (Example) Hereinafter, a component mounting system 10 according to Example 1 will be described with reference to the drawings. As shown in Fig. 1, the component mounting system 10 includes a plurality of pieces of production equipment 12-28 installed in a component mounting line, a management device 30 that manages the plurality of pieces of production equipment 12-28, photographing devices (34a, 34b) installed in the component mounting line, and an image processing device 36 that processes images photographed by the photographing devices (34a, 34b).

[0018] The production equipment 12-28 constitute a component mounting line that mounts components on a board. The component mounting line mounts components on a board that is input, and manufactures a board on which the components are mounted. Hereinafter, the board after the components are mounted is referred to as a circuit board, and the board before or during the component mounting may be simply referred to as a board. In the component mounting line of this embodiment, a plurality of types of circuit boards are manufactured. Different types of circuit boards have different board sizes, different solder patterns printed on the board, and different types of components mounted on the board. For this reason, when changing the type of circuit board produced in the component mounting line, a setup operation (preparation operation) is required in each of the production equipment 12-28. In addition, when the production of the circuit board is started in the component mounting line, boards, solder, components, etc. are used in the production of the circuit board, and it becomes necessary to replenish them. For these reasons, a plurality of operators 50a-50c (an example of "workers") are deployed in the component mounting line. In the mounting line shown in FIG. 1, three operators 50a-50c are deployed.

[0019] The component mounting line includes a board loader 12, a printer 14, a print inspection machine (SPI) 16, a first component mounter 18, a second component mounter 20, a first board visual inspection machine (AOI) 22, a reflow furnace 24, a second board visual inspection machine (AOI) 26, and a board unloader 28. These production facilities 12 to 28 can be known machines used in known component mounting lines, so they will be briefly described. The board loader 12 puts boards into the component mounting line. The board loader 12 stores a plurality of boards, and carries the stored boards out one by one to the printer 14. The printer 14 prints a solder pattern on the board carried in from the board loader 12. The board on which the solder pattern has been printed is transported from the printer 14 to the print inspection machine 16. The print inspection machine 16 inspects whether the solder pattern printed on the board is normal or not. If an abnormality occurs in the printed solder pattern (for example, if a printing defect occurs due to clogging of the mask), the board is discarded. On the other hand, if the printed solder pattern is normal, the board is sent from the print inspection machine 16 to the first component mounter 18. The first component mounter 18 mounts a predetermined number of components on the board sent from the print inspection machine 16. Specifically, the first component mounter 18 is detachably attached with a plurality of component feeders, and mounts the components supplied from these component feeders on the board. The board on which the components have been mounted by the first component mounter 18 is sent to the second component mounter 20. The second component mounter 20, like the first component mounter 18, mounts a predetermined number of components on the board sent from the first component mounter 18. The board on which the components have been mounted is sent to the first board appearance inspection machine 22. The first board appearance inspection machine 22 inspects whether the components have been normally mounted on the board. If the components are not mounted correctly on the board (for example, if the components are mounted in the wrong location), the board is discarded. On the other hand, if the components are mounted correctly on the board, the board is transferred from the first board visual inspection machine 22 to a reflow furnace 24. The reflow furnace 24 heats the board that is transferred in to melt the solder and solder the components to the board. The board that is transferred out of the reflow furnace 24 is transferred to a second board visual inspection machine 26. The second board visual inspection machine 26 inspects whether the board is soldered in the correct location.If the components are not soldered in the correct position on the board (for example, if the components become misaligned for some reason when heated in reflow furnace 24), the board is discarded. On the other hand, if the components are soldered in the correct position on the board, the board is carried out from second board visual inspection machine 22 to board unloader 28. Board unloader 28 carries the board transported from second board visual inspection machine 22 (i.e., the circuit board on which components are mounted) out of the component mounting line.

[0020] Each of the production equipment 12-28 includes a communication circuit 12a-28a (an example of a "status output unit"). The communication circuits 12a-28a are communicatively connected to the management device 30. Each of the communication circuits 12a-28a outputs status information indicating the status of the production equipment 12-28 equipped with the communication circuit to the management device 30. For example, the board loader 12 outputs the number of boards accommodated to the management device 30. This allows the management device 30 to determine whether or not it is necessary to replenish boards in the board loader 12. Also, for example, the first component mounter 18 and the second component mounter 20 output the number of components used for each type of component to the management device 30. This allows the management device 30 to determine whether or not it is necessary to replenish components in the first component mounter 18 and the second component mounter 20.

[0021] Furthermore, the component feeders attached to the first component mounter 18 and the second component mounter 20 are transported by a feeder rack 32. That is, component feeders before use are loaded onto the feeder rack 32 in a storage location (not shown) and transported to the first component mounter 18 and / or the second component mounter 20 by the feeder rack. Furthermore, component feeders that have been used by the first component mounter 18 and / or the second component mounter 20 are loaded onto the feeder rack 32. The used component feeders loaded onto the feeder rack 32 are transported to a storage location (not shown) and prepared for the next use.

[0022] The management device 30 is configured by a computer having a CPU and a memory. The management device 30 controls the production of circuit boards by controlling the operations of the production equipment 12 to 28. For example, the management device 30 transmits a solder print pattern to the print inspection machine 16, and the print inspection machine 16 inspects the board using the received print pattern. Also, for example, the management device 30 transmits a mounting program (mounting job) that specifies the type, mounting order, and mounting position of components to be mounted to the first component mounter 18 and the second component mounter 20, and the first component mounter 18 and the second component mounter 20 mount the components on the board based on the received mounting program.

[0023] Furthermore, the management device 30 determines the operating status of each piece of production equipment based on the status information output from each piece of production equipment 12-28. For example, when the first component mounter 18 and the second component mounter 20 need to replace their component feeders, they output information to that effect to the management device 30. Based on the information output from the first component mounter 18 and the second component mounter 20, the management device 30 can determine that the first component mounter 18 and the second component mounter 20 need to replace their component feeders.

[0024] The photographing devices (34a, 34b (an example of an "photographing unit")) are disposed in a factory in which a component mounting line is installed. The photographing devices (34a, 34b) are capable of photographing a plurality of pieces of production equipment 12-28 installed in the component mounting line, a plurality of workers 50a-50c arranged in the component mounting line, and a feeder rack 32 used in the component mounting line. Specifically, the photographing devices (34a, 34b) include a first camera 34a disposed on the upstream side (board loader 12 side) of the component mounting line, and a second camera 34b disposed on the downstream side (board unloader 28 side) of the component mounting line. The first camera 34a is disposed so that its optical axis is oblique to the component mounting line (the direction in which the production equipment 12-28 is disposed), and is capable of photographing the entirety of the production equipment 12-28 from the upstream side of the component mounting line. The second camera 34b is disposed so that its optical axis is oblique to the component mounting line, and is capable of photographing the entirety of the production equipment 12-28 from the downstream side of the component mounting line. The first camera 34a and the second camera 34b photograph the component mounting line from different directions, making it possible to identify the positions (x, y) of the operators 50a-50c, as described below. The first camera 34a and the second camera 34b are connected to an image processing device 36. Video data photographed by the first camera 34a and the second camera 34b is input to the image processing device 36.

[0025] The image processing device 36 is configured by a computer having a CPU and a memory. The image processing device 36 can function as a "display unit" or an "operational status acquisition unit" by executing a program (including an image processing program) stored in the memory. The first camera 34a, the second camera 34b, and the management device 30 are connected to the image processing device 36. The image processing device 36 receives the video captured by the first camera 34a and the video captured by the second camera 34b, and also receives the operational status of the production equipment 12 to 28 from the management device 30. The image processing device 36 visualizes the work status of the operators 50a to 50c based on the input video and the input operational status of the production equipment 12 to 28. The processing performed by the image processing device 36 will be described in detail later.

[0026] The image processing device 36 is further connected to a display device 38, an input device 40, and an image registration device 42. The display device 38 displays various information output from the image processing device 36 (for example, a time chart visualizing the operating status of the production equipment 12-28 and the work status of the operators 50a-50c). A known display can be used for the display device 38. The input device 40 can specify (input) the conditions of the image processing performed by the image processing device 36, and the production equipment and operators to be the targets of the image processing. That is, the input device 40 is an example of a "first specification device" and an example of a "second specification device". The information input from the input device 40 is stored in the memory of the image processing device 36. A known pointing device such as a keyboard or a mouse can be used for the input device 40. The image registration device 42 inputs (registers) images (reference images) of the production equipment 12-28 and the operators 50a-50c to be recognized in the image processing performed by the image processing device 36. The reference image input from the image registration device 42 is stored in the memory of the image processing device 36. A digital camera, a smartphone with a digital camera, a tablet PC with a digital camera, a scanner, etc. can be used as the image registration device 42. The image registration device 42 is an example of an "image input unit."

[0027] Next, a process executed by the image processing device 36 when visualizing the working status of the operators 50a to 50c based on the video captured by the photographing devices (34a, 34b) will be described. As shown in Fig. 2, first, the image processing device 36 registers (inputs) reference images of the production equipment 12 to 28 and the feeder rack 32 that are the subject of image processing (S10). In this embodiment, the reference images of all the production equipment 12 to 28 that constitute the component mounting line and the reference image of the feeder rack 32 used in the component mounting line are input. An image registration device 42 is used to input these reference images.

[0028] Next, the image processing device 36 registers (inputs) reference images of the operators 50a to 50c to be subjected to image processing (S12). In this embodiment, the image processing device 36 inputs reference images of all the operators 50a to 50c arranged in the component mounting line. The image registration device 42 is used to input these reference images, as in S10.

[0029] Next, the image processing device 36 imports the video data captured by the photographing devices (34a, 34b) (S14). That is, the photographing devices (34a, 34b) capture the production equipment 12-28, the feeder rack 32, and the operators 50a-50c during a predetermined production period in which the circuit boards are produced on the component mounting line. The video data captured by the photographing devices (34a, 34b) is stored in the memory of the photographing devices (34a, 34b). In S14, the video data stored in the memory of the photographing devices (34a, 34b) is input to the image processing device 36. The user can arbitrarily set the period during which the photographing devices (34a, 34b) capture the component mounting line. For example, the photographing period may be the operating hours (e.g., 9:00 a.m. to 5:00 p.m.) of the factory in which the component mounting line is installed, or, if the factory in which the component mounting line is installed operates on a 24-hour system, the photographing period may be one day from the time when the operator is changed.

[0030] Next, the image processing device 36 captures the operation status of the production equipment 12-28 during the period when the video was captured by the imaging devices (34a, 34b) (S16). That is, the production equipment 12-28 outputs status information indicating its own status to the management device 30 via the communication circuits 12a-28a. Therefore, the management device 30 acquires operation information of the production equipment 12-28 (for example, operating normally, stopped (including error code), undergoing maintenance, etc.) based on the status information output from the production equipment 12-28, and stores the operation information in memory. In S16, the operation information of the production equipment 12-28 stored in the memory of the management device 30 is input to the image processing device 36.

[0031] Next, the image processing device 36 performs image processing on the video data input in S14 to create a time chart that visualizes the work status of the operators 50a to 50c (S18). The image processing in S18 will be described in detail with reference to FIG.

[0032] As shown in FIG. 3, first, the image processing device 36 selects image data (one video data) to be subjected to image processing (analysis target) from the video data input in S14 (S22). That is, the photographing device (34a, 34b) photographs video data at a predetermined time interval (predetermined frame rate). Therefore, the video data input in S14 is composed of a plurality of image data photographed at a predetermined time interval. In S22, one image data to be subjected to image processing is selected from a plurality of image data included in the video data. For example, all image data included in the photographed video data may be selected and image processed. In this case, since all image data are processed, the working status of the operators 50a to 50c can be visualized with high accuracy. On the other hand, only image data selected at a predetermined time interval (i.e., a time interval longer than the photographing time interval of the photographing device (34a, 34b)) from the image data included in the photographed video data may be image processed. In this case, the number of image data to be subjected to image processing is reduced, so that the time required for image processing can be shortened.

[0033] Next, the image processing device 36 automatically recognizes whether the production equipment 12-18 to be recognized is included in the image data selected in S22 (S24). In this embodiment, all of the production equipment 12-28 constituting the component mounting line, the feeder rack 32 used in the component mounting line, and all of the operators 50a-50c arranged in the component mounting line are recognized. The first camera 34a and the second camera 34b of the photographing device (34a, 34b) are installed so that all of the production equipment 12-28 can be photographed. Therefore, all of the production equipment 12-28 is photographed in the video data photographed by the first camera 34a and the second camera 34b. On the other hand, the feeder rack 32 may be located outside the photographing range of the photographing device (34a, 34b), and the operators 50a-50c may also be located outside the photographing range of the photographing device (34a, 34b). Therefore, in S24, the production equipment 12-28 shown in the image data selected in S22 is recognized, and when the feeder rack 32 and the operators 50a-50c are located within the photographing range of the photographing devices (34a, 34b), the feeder rack 32 and the operators 50a-50c are recognized. A known automatic recognition program can be used as a method for automatically recognizing whether or not the object to be recognized is shown in the image data.

[0034] Next, the image processing device 36 judges whether or not any of the operators 50a to 50c was recognized in S24 (S26). If the image processing device 36 does not recognize any of the operators 50a to 50c (i.e., if none of the operators 50a to 50c is present on the component mounting line), S26 becomes NO, and the process proceeds to S36.

[0035] If the image processing device 36 recognizes any of the operators 50a to 50c (i.e., if any of the operators 50a to 50c are present on the component mounting line (YES in S26)), the process proceeds to S28, where one of the recognized operators is selected. For example, if the three operators 50a to 50c are recognized, one of the operators 50a to 50c to be analyzed is selected.

[0036] Next, the image processing device 36 calculates the distance between the operator selected in S28 and each of the production facilities 12 to 28 (S30). The procedure for calculating the distance between the operator and the production facilities will be described with reference to FIG. 4. FIG. 4 shows an operator 50a and the production facilities 22 and 24. As shown in FIG. 4, the position (x1, y1) where the first camera 34a is installed is known, and the position (x2, y2) where the second camera 34b is installed is also known. The optical axis of the first camera 34a is fixed, and the optical axis of the second camera 34b is also fixed. Therefore, the direction (angle θ1) where the operator 50a is located relative to the first camera 34a can be calculated from the position of the operator 50a in the image captured by the first camera 34a. Similarly, the direction (angle θ2) where the operator 50a is located relative to the second camera 34b can be calculated from the position of the operator 50a in the image captured by the second camera 34b. The position (x1, y1) of the first camera 34a and the position (x2, y2) of the second camera 34b are known, and the above-mentioned angles θ1 and θ2 are calculated, so that the position (x3, y3) of the operator 50a can be calculated. Here, the front center position (x4, y4) of the production equipment 22 and the front center position (x5, y5) of the production equipment 24 are also known, so that the distance l1 between the operator 50a and the production equipment 22 and the distance l2 between the operator 50a and the production equipment 24 can be calculated. In S30, the distance between the operator selected in S28 and each of the production equipment 12 to 28 is calculated by the above-mentioned procedure. As is clear from the above explanation, the distance calculated in S30 means the distance when the production equipment 12 to 28 and the operators 50a to 50c are viewed in a plane. In addition, if the feeder rack 32 is recognized in S24, the distance between the operator and the feeder rack 32 is also calculated. The position of the feeder rack 32 can be calculated in the same way as the operator, so the distance between the feeder rack 32 and the operator can also be calculated.

[0037] In S30, the image processing device 36 associates the operator selected in S28 with the production equipment (feeder rack 32 in some cases) that is closest to the operator (S32). That is, since the distances between the operator and each piece of production equipment 12 to 28 are calculated in S30, the production equipment that has the shortest calculated distance is associated with the operator selected in S28. For example, in the case shown in FIG. 4, the operator 50a is associated with the production equipment (first board visual inspection machine) 22.

[0038] Next, the image processing device 36 judges whether the processes of S28 to S32 have been executed for all the operators recognized in S24 (S34). If the processes have not been executed for all the operators recognized in S24 (NO in S34), the process returns to S28 and executes the processes from S28. As a result, the distances between the production equipment 12 to 28 and all the operators recognized in S24 are calculated, and each operator is associated with the production equipment with the shortest distance.

[0039] If processing has been performed for all operators recognized in S24 (YES in S34), the process proceeds to S36. In S36, the image processing device 36 determines whether or not all image data to be subjected to image processing among the moving image data input in S14 has been processed. If not all image data has been processed (NO in S36), the image processing device 36 returns to S22 and repeats the processing from S22. As a result, all image data to be subjected to image processing among the moving image data input in S14 is processed.

[0040] If all image data has been processed (YES in S36), the image processing device 36 selects one operator to be analyzed (S38). In this embodiment, each of the three operators 50a to 50c deployed on the component mounting line is the analysis target. Therefore, in S38, one of the three operators 50a to 50c (for example, operator 50a) is selected.

[0041] Next, the image processing device 36 judges whether the operator selected in S38 has been associated with the same production equipment 12-28, 32 continuously for a predetermined time or more during the period when the video was captured by the imaging device (34a, 34b) (S40). That is, by the above-mentioned processing of S22-S36, all image data to be analyzed among the video data input in S14 are image-processed, and the operators 50a-50c are associated with any of the production equipment 12-28, 32 at the time when each image data was captured. (However, if the operator is not shown in the image data (i.e., if the operator is not recognized), the operator is not associated with any of the production equipment 12-28, 32 at that time.) When the operator is associated with the production equipment, it can be determined that the operator is located near the production equipment and is performing some work on the production equipment. However, even if the operator is located near any production equipment at a certain time, he / she may simply be passing in front of the production equipment and not performing work on the production equipment. In this embodiment, when an operator is continuously associated with the same production equipment for a predetermined time (e.g., 20 seconds) or more, it is determined that the operator is working on the production equipment. For this purpose, in S40, it is determined whether the operator has been continuously associated with the same production equipment 12-28, 32 for a predetermined time or more.

[0042] The judgment in S40 can be made based on the time interval of the image data selected in S22. For example, if the time interval of the image data selected in S22 is 5 seconds and the predetermined time used in the judgment in S40 is 20 seconds, when the operator is associated with the same production equipment for 5 consecutive image data, it can be judged that the operator has been associated with the same production equipment for a period of time equal to or longer than the predetermined time. From the viewpoint of reducing the amount of image processing, the time interval of the image data selected in S22 may be made to match the predetermined time used in the judgment in S40. In this case, when the operator is associated with the same production equipment for 2 consecutive image data, it can be judged that the operator has been associated with the same production equipment for a period of time equal to or longer than the predetermined time.

[0043] If the operator has not been associated with the same production equipment for a period of time equal to or longer than the predetermined time (NO in S40), it is determined that the operator has not worked on any of the production equipment 12-28, 32, and the result of associating the operator with the production equipment is reset (S44). On the other hand, if the operator has been associated with the same production equipment for a period of time equal to or longer than the predetermined time (YES in S40), it is determined that the operator has worked on the production equipment during that period, and the result of associating the operator with the production equipment is maintained (S42). Note that, for periods other than the period in which the operator has been associated with the same production equipment for a period of time equal to or longer than the predetermined time, it is determined that the operator has not worked on any of the production equipment 12-28, 32, and the result of associating the operator with the production equipment is reset.

[0044] Next, the image processing device 36 judges whether the processes of S40 to S44 have been executed for all the operators to be analyzed (S46). If the processes have not been executed for all the operators to be analyzed (NO in S46), the process returns to S38 and executes the processes from S38. As a result, the processes of S40 to S44 are executed for all the operators to be analyzed, and it is determined which production equipment the relevant operators have been working on.

[0045] When S46 is completed, the process returns to S20 in FIG. 2, and the image processing device 36 displays on the display device 38 a time chart in which the operators 50a-50c to be analyzed are associated with each of the production facilities 12-28, 32 (S20). That is, the display device 38 displays a time chart that visualizes the work status of which production facility the operators 50a-50c are working on at each time during the period (production period) during which the video was taken by the imaging devices (34a, 34b). Since the image processing device 36 has acquired the operation status of each of the production facilities 12-28 in S16, the time chart displayed in S20 may also display the operation status of each of the production facilities 12-28. By displaying the operation status of each of the production facilities 12-28, the work content that the operators 50a-50c are working on each of the production facilities 12-28 can be estimated. For example, if an operator is working on the first component mounter 18 and the first component mounter 18 is issuing a warning that it is out of components, it can be assumed that the operator is working to replenish components.

[0046] An example of a time chart displayed on the display device 38 in S20 will now be described with reference to FIGS.

[0047] 5 is a time chart showing the working status of operators 50a to 50c at the production facilities 12 to 28, 32 from time 9:50 to time 11:05. Note that the working status of operator 50a is displayed as "operator 1" in the figure, the working status of operator 50b is displayed as "operator 2" in the figure, and the working status of operator 50c is displayed as "operator 3" in the figure.

[0048] In FIG. 5, the working status of the operators 50a to 50c is displayed for each minute, indicating which production equipment the operator worked on. For example, the 9:50 column displays which production equipment the operator worked on between 9:50:00 and 9:51:00. In this embodiment, when the operator is associated with the production equipment 12 to 28, 32 for 20 consecutive seconds or more, it is determined that the operator is working on the production equipment. For this reason, there are cases where the operator works on multiple production equipment in one minute. For example, the operator 50b (operator 2) works on both the first component mounter 18 and the second component mounter 20 at 10:08.

[0049] 5 also shows the operating status of the production equipment 12-28. Specifically, the printer 14 outputs a setup work guide at 9:51, and similarly, the print inspection machine (SPI) 16 outputs a setup work guide (alarm) at 9:52, the first component mounter 18 and the second component mounter 20 outputs a setup work guide (alarm) at 9:53, the first board visual inspection machine (AOI) 22 outputs a setup work guide at 9:54, the reflow furnace 24 and the second board visual inspection machine (AOI) 26 outputs a setup work guide at 9:55, and the board unloader 28 outputs a setup work guide (alarm) at 9:56. Therefore, the time chart in FIG. 5 is a time chart when setup work guide is output to the production equipment 12-28 and the operators 50a-50c perform setup work for each of the production equipment 12-28 in response to the guide.

[0050] From the time chart shown in Fig. 5, it can be seen that specific measures are necessary to improve the working conditions of the operators 50a to 50c. For example, as shown in A of Fig. 5, the operator 50a (operator 1) takes a long time to set up the printer 14. Therefore, in order to shorten the time it takes for the operator 50a to set up the printer 14, it can be seen that measures such as kneading the solder in advance and preparing a replacement mask in advance are necessary.

[0051] 5B, in the first component mounter 18 and the second component mounter 20, it takes a long time from when the setup work guide is output until the operator 50b (operator 2) actually starts the work. Therefore, in order to shorten the time until the operator 50b starts the work, it is necessary to take measures such as preparing the feeder rack 32 in advance.

[0052] 5C, the workload of operator 50b (operator 2) is high during this time period, and operator 50b is working on multiple pieces of production equipment 18-28, 32. Meanwhile, operator 50c (operator 3) is not working during this time period. Therefore, in order to reduce the workload of operator 50b, it is necessary to take measures to have another operator 50c share the work.

[0053] 5D and 5E, operator 50a (operator 1) and operator 50b (operator 2) work together on the first component mounter 18 and the second component mounter 20 from time 10:44 to time 10:54, so operator 50a's work on the printing press 14 is delayed until time 10:57. Therefore, it can be seen that by changing the work schedule so that operator 50b and operator 50c (operator 3) work together on the first component mounter 18 and the second component mounter 20, operator 50a's work on the printing press 14 can be started promptly.

[0054] Fig. 6 is a time chart showing the working status of operators 50a-50c at production facilities 12-28, 32 from 13:15 to 13:54. Fig. 6 also shows the operating status of production facilities 12-28 (specifically, displays on signal towers and monitors). As is clear from Fig. 6, the time chart in Fig. 6 shows displays (alarms) issued by the production facilities on signal towers and monitors, and the working status of operators 50a-50c responding to these displays.

[0055] From the time chart shown in Fig. 6, measures for improving the working conditions of the operators 50a to 50c can be derived. For example, as shown in Fig. 6F, alarms occurred simultaneously in the printer 14 and the first component mounter 18 at time 13:33, so that the work of the operator 50a (operator 1) on the first component mounter 18 is delayed. Therefore, when alarms occur simultaneously in multiple pieces of production equipment, it is necessary for multiple operators to respond. In the above example, it is only necessary to change so that the operators 50b and 50c (operators 2 and 3) work on the first component mounter 18.

[0056] Also, as shown in Figure 6G, it takes about 2 minutes for operator 50c (operator 3) to complete the work after an alarm occurs in the print inspection machine (SPI) 16. This suggests the possibility that operator 50c is not familiar with dealing with the print inspection machine 16. Therefore, possible countermeasures to this problem include standardizing (manualizing) the work of the print inspection machine 16 and educating operator 50c.

[0057] As described above, in the component mounting system 10 of this embodiment, the multiple production equipment 12-28, 32 and the multiple operators 50a-50c arranged in the component mounting line are photographed, and the photographed video is image-processed to visualize the working status of the operators 50a-50c. Therefore, it is possible to evaluate whether the work of the operators 50a-50c is appropriate or not with respect to the operating status of the multiple production equipment 12-28, 32 installed in the component mounting system. Furthermore, if the work of the operators 50a-50c is not appropriate, the cause can be easily estimated, and as a result, it becomes possible to find a remedy.

[0058] Although specific examples of the technology disclosed in this specification have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples exemplified above.

[0059] For example, in the above embodiment, each production facility 12-28 outputs status information indicating its own status, and the management device 30 acquires the operation status of each production facility 12-28 from the output status information, and the acquired operation status is input to the image processing device 36. However, the technology disclosed in this specification is not limited to such a form. For example, the image processing device 36 may acquire the operation status of each production facility by image processing video data captured by a shooting device. That is, the operation status of each production facility may be acquired from the guidance of a signal tower or monitor installed in each production facility by image processing the video data. In this case, since it is not necessary to acquire the operation status from the management device 30, the system configuration can be simplified.

[0060] In the above embodiment, the producer who produces the circuit boards on the component mounting line and the person who visualizes the working status of the workers with the image processing device 36 are the same person, but the technology disclosed in this specification is not limited to this form. For example, the producer who produces the circuit boards on the component mounting line and the person who visualizes the working status of the workers with the image processing device 36 may be different people. In this case, the producer who produces the circuit boards on the component mounting line only registers reference images of the production equipment and the operators to be image-processed, and inputs video data of the component mounting line. The registration of the reference images and the input of the videos can be performed from a PC or tablet PC owned by the producer. Meanwhile, the person who visualizes the working status of the workers with the image processing device 36 analyzes the registered reference images and the input video data, and outputs a time chart that visualizes the working status of the workers to the producer. According to this form, the operator of the image processing device 36 can support the producer who produces the circuit boards on the component mounting line from a remote location.

[0061] In the above embodiment, it is determined whether the operators 50a to 50c are working on each of the production equipment 12 to 28, 32, but the technology disclosed in this specification is not limited to this form. For example, the image processing device may estimate (determine) the work content from the operating status of the production equipment where the operator is working. By the image processing device estimating the work content of the operator, the user can easily evaluate the work status of the operator. Furthermore, the image processing device may determine whether the operator is delayed in work from the transition of the operating status of the production equipment and the transition of the work status of the operator. With such a configuration, the delay in the operator's work can be easily grasped, and the convenience of the manufacturer who produces circuit boards on the component mounting line can be improved.

[0062] In the above embodiment, the operators 50a to 50c are always associated with the closest production equipment, but the technology disclosed in this specification is not limited to this form. For example, if the distance between an operator and the closest production equipment is longer than a predetermined distance, the operator does not need to be associated with the production equipment. In other words, the operator is not associated with any production equipment. This is because, if the distance from the operator to the production equipment exceeds the predetermined distance, it is difficult to think that the operator is working on the production equipment.

[0063] In the above embodiment, all the production equipment installed in the component mounting line and all the operators assigned to the component mounting line are analyzed, but the technology disclosed in this specification is not limited to this. For example, only the work status of a specific operator at a specific production equipment may be analyzed.

[0064] In the above embodiment, the positions of the operators 50a to 50c are calculated by photographing the operators 50a to 50c with the two cameras 34a and 34b, but the technology disclosed in this specification is not limited to this form. For example, one camera and a distance sensor (laser sensor) that measures the distance to the operator may be installed, and the position of the operator may be calculated using the image of the camera and the distance measured by the distance sensor.

[0065] In the above embodiment, an image captured by a camera is used to calculate the position of the operator, but the position of the operator may be acquired by having the operator carry a GPS sensor. In this case, it is not necessary to capture images of the production equipment installed on the component mounting line, and the working status of the operator may be estimated from the position of the operator alone.

[0066] In addition, the technical elements described in this specification or drawings exert technical utility alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technologies exemplified in this specification or drawings achieve multiple objectives simultaneously, and achieving one of those objectives is itself technically useful.

Claims

1. A plurality of production equipments installed in a component mounting line that mounts components on a board; an imaging unit that is installed in the component mounting line and images the plurality of pieces of production equipment and one or more workers arranged in the component mounting line; a display unit that performs image processing on images captured by the imaging unit during a predetermined production period in which products are produced on the component mounting line, thereby visualizing and displaying, for at least one of the one or more workers, a work status of the worker with respect to at least one of the plurality of pieces of production equipment at each of a plurality of time points or a plurality of time periods that are preset within the production period; an operation status acquisition unit that acquires an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods, The display unit displays the acquired operation status of the production equipment in correspondence with the work status, The component mounting system further includes a judgment unit that judges whether or not there is a delay in the work of the worker based on a transition in the operating status of the production equipment and a transition in the work status of the worker.

2. A plurality of production equipments installed in a component mounting line that mounts components on a substrate; an imaging unit that is installed in the component mounting line and images the plurality of pieces of production equipment and one or more workers arranged in the component mounting line; a display unit that performs image processing on images captured by the imaging unit during a predetermined production period in which products are produced on the component mounting line, thereby visualizing and displaying, for at least one of the one or more workers, a work status of the worker with respect to at least one of the plurality of pieces of production equipment at each of a plurality of time points or a plurality of time periods that are preset within the production period; an operation status acquisition unit that acquires an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods, The display unit displays the acquired operation status of the production equipment in correspondence with the work status, The method further includes a first designation device that designates a production facility to be analyzed from among the plurality of production facilities, A component mounting system, wherein the operation status acquisition unit acquires the operation status of a production facility designated by the first designated device.

3. A plurality of production equipments installed in a component mounting line that mounts components on a substrate; an imaging unit that is installed in the component mounting line and images the plurality of pieces of production equipment and one or more workers arranged in the component mounting line; a display unit that performs image processing on images captured by the imaging unit during a predetermined production period in which products are produced on the component mounting line, thereby visualizing and displaying, for at least one of the one or more workers, a work status of the worker with respect to at least one of the plurality of pieces of production equipment at each of a plurality of time points or a plurality of time periods that are preset within the production period; an operation status acquisition unit that acquires an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods, The display unit displays the acquired operation status of the production equipment in correspondence with the work status, The method further includes a second designation device that designates a worker to be analyzed from among the one or more workers, The display unit visualizes and displays the work status of the worker designated by the second designation device.

4. A plurality of production equipments installed in a component mounting line that mounts components on a substrate; an imaging unit that is installed in the component mounting line and images the plurality of pieces of production equipment and one or more workers arranged in the component mounting line; a display unit that performs image processing on images captured by the imaging unit during a predetermined production period in which products are produced on the component mounting line, thereby visualizing and displaying, for at least one of the one or more workers, a work status of the worker with respect to at least one of the plurality of pieces of production equipment at each of a plurality of time points or a plurality of time periods that are preset within the production period; an operation status acquisition unit that acquires an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods, The display unit displays the acquired operation status of the production equipment in correspondence with the work status, In the image processing, if it is determined that the worker is working on a specific production equipment, the component mounting system identifies the content of the work performed by the worker on the specific production equipment based on the operating status of the specific production equipment.

5. 5. The component mounting system according to claim 1, wherein the operation status acquisition unit acquires the operation status by performing image processing.

6. At least one of the plurality of production facilities includes a status output unit that outputs a status of the production facilities, 5. The component mounting system according to claim 1, wherein the operation status acquisition unit acquires the operation status from the status output from the status output unit.

7. In the image processing, When a plurality of pieces of production equipment are photographed in the vicinity of the worker at any of the plurality of time points or the plurality of time periods, a distance between the worker and each of the plurality of pieces of production equipment is calculated; 5. The component mounting system according to claim 1, wherein it is determined that the worker is performing work on a production facility having a minimum distance among the calculated distances.

8. In the image processing, when it is determined that the worker is performing work on the production equipment having the smallest distance, If the time during which the worker is performing work on the production equipment is less than a predetermined time, discard the determination that the worker is performing work on the production equipment; 8. The component mounting system according to claim 7, wherein when the time during which the worker is performing work on the production facility is equal to or longer than the predetermined time, the determination that the worker is performing work on the production facility is maintained.

9. An image input unit that inputs images taken of a plurality of pieces of production equipment installed on a component mounting line and one or more workers stationed on the component mounting line during a specified production period in which products are produced on the component mounting line in which components are mounted on a board; an image processing unit that performs image processing on the image input from the image input unit to visualize, for at least one of the one or more workers, a work status of the worker at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods that are preset within the production period, The image processing unit includes: Acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of points in time or the plurality of periods; The acquired operation status of the production equipment is visualized in correspondence with the work status; The image processing device further includes, when a production facility to be analyzed is designated from among the plurality of production facilities, acquiring an operating status of the designated production facility.

10. An image input unit that inputs images taken of a plurality of pieces of production equipment installed on a component mounting line and one or more workers assigned to the component mounting line during a specified production period in which products are produced on the component mounting line in which components are mounted on a board; an image processing unit that performs image processing on the image input from the image input unit to visualize, for at least one of the one or more workers, a work status of the worker at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods that are preset within the production period, The image processing unit includes: Acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of points in time or the plurality of periods; The acquired operation status of the production equipment is visualized in correspondence with the work status; The image processing device further comprises: when a worker to be analyzed is designated from the one or more workers, the image processing unit visualizes a work status of the designated worker.

11. An image capturing step of capturing images of a plurality of pieces of production equipment installed on a component mounting line and one or more workers deployed on the component mounting line during a predetermined production period in which a product is produced on the component mounting line in which components are mounted on a board; and an image processing step of visualizing a work status of at least one of the one or more workers at at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods preset within the production period by image processing the image taken in the image taking step, The image processing step includes: An acquisition step of acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods; and visualizing the operation status of the production equipment acquired in the acquisition step in correspondence with the work status, The image processing method includes, when a production facility to be analyzed is designated from among the plurality of production facilities, acquiring an operating status of the designated production facility.

12. An image capturing step of capturing images of a plurality of pieces of production equipment installed on a component mounting line and one or more workers deployed on the component mounting line during a predetermined production period in which a product is produced on the component mounting line in which components are mounted on a board; and an image processing step of visualizing a work status of at least one of the one or more workers at at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods preset within the production period by image processing the image taken in the image taking step, The image processing step includes: An acquisition step of acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods; The method further includes a step of visualizing the operation status of the production equipment acquired in the acquisition step in correspondence with the work status, The image processing method further includes, when a worker to be analyzed is designated from the one or more workers, visualizing a work status of the designated worker.

13. An image capturing step of capturing images of a plurality of pieces of production equipment installed on a component mounting line and one or more workers deployed on the component mounting line during a predetermined production period in which a product is produced on the component mounting line in which components are mounted on a board; an image processing step of visualizing a work status of at least one of the one or more workers at at least one of the plurality of production facilities at each of a plurality of time points or a plurality of time periods preset within the production period by image processing the image taken in the image taking step; A program for causing a computer to execute the above, The image processing step includes: An acquisition step of acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods; and visualizing the operation status of the production equipment acquired in the acquisition step in correspondence with the work status, The acquisition step is a program for acquiring an operating status of a specified production facility when the production facility to be analyzed is specified from among the plurality of production facilities.

14. An image capturing step of capturing images of a plurality of pieces of production equipment installed on a component mounting line and one or more workers deployed on the component mounting line during a predetermined production period in which a product is produced on the component mounting line in which components are mounted on a board; an image processing step of visualizing a work status of at least one of the one or more workers at at least one of the plurality of production facilities at each of a plurality of time points or a plurality of time periods preset within the production period by image processing the image taken in the image taking step; A program for causing a computer to execute the above, The image processing step includes: An acquisition step of acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of time points or the plurality of time periods; and visualizing the operation status of the production equipment acquired in the acquisition step in correspondence with the work status, The image processing step further includes a program for visualizing, when a worker to be analyzed is designated from the one or more workers, the work status of the designated worker.

15. An input device for inputting images taken of a plurality of pieces of production equipment installed on a component mounting line and one or more workers stationed on the component mounting line during a specified production period in which products are produced on the component mounting line in which components are mounted on a board; an image processing device that is communicatively connected to the input device and processes an image input from the input device to visualize a work status of at least one of the one or more workers at at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods that are preset within the production period, The image processing device includes: Acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of points in time or the plurality of periods; The acquired operation status of the production equipment is visualized in correspondence with the work status; The method further includes a first designation device that designates a production facility to be analyzed from among the plurality of production facilities, The image processing system includes an image processing device that acquires an operating status of a production facility designated by the first designated device.

16. An input device for inputting images taken of a plurality of pieces of production equipment installed on a component mounting line and one or more workers stationed on the component mounting line during a specified production period in which products are produced on the component mounting line in which components are mounted on a board; an image processing device that is communicatively connected to the input device and processes an image input from the input device to visualize a work status of at least one of the one or more workers at at least one of the plurality of production equipments at each of a plurality of time points or a plurality of time periods that are preset within the production period, The image processing device includes: Acquiring an operation status of at least one of the plurality of production facilities at each of the plurality of points in time or the plurality of periods; The acquired operation status of the production equipment is visualized in correspondence with the work status; The method further includes a second designation device that designates a worker to be analyzed from among the one or more workers, The image processing system includes: an image processing device that visualizes a work status of a worker designated by the second designation device;

Images