Work improvement support system, work improvement support method, and program

Abstract

A work improvement support system (1) comprises: a work data generation unit (21) that generates work data by attaching, to a microscope image acquired by a microscope device (10) during the work of assembling or inspecting equipment using the microscope device (10), tag information relating to the work at the time of the acquisition of the microscope image; a quality information generation unit (22) that, on the basis of a plurality of sets of work data generated by the work data generation unit (21), generates quality information relating to the quality of the work for each work data classification classified by the tag information; and a display control unit (23) that displays the quality information generated by the quality information generation unit (22) on a display device (30).

Description

Work improvement support system, work improvement support method, and program

[0001] The disclosure of this specification relates to a work improvement support system, a work improvement support method, and a program, and in particular to a system, method, and program that support the improvement of work performed under a microscope, such as the assembly of precision equipment.

[0002] Even today, when automation of work is progressing with the use of robots and other technologies, there are still many products that require manual assembly. Medical equipment is one example. Assembly and inspection of precision equipment such as medical equipment involves a lot of detailed work, which is often done under a microscope, making the work more difficult. Therefore, it is difficult to avoid variations in work efficiency and quality due to differences in the proficiency level of workers.

[0003] A technology related to such technical issues is described, for example, in Patent Document 1. The work instruction system described in Patent Document 1 generates and displays work instruction information based on information input by a worker and information pre-recorded in a database. By having the worker perform the work while checking the displayed work instructions, errors in the work procedure can be reduced.

[0004] Japanese Patent Application Publication No. 9-262745

[0005] However, simply displaying work instructions has limited effect on work improvement. For highly difficult work performed under a microscope, there is a need for proposals for new technologies that can further improve work efficiency and work quality variations that arise from the level of worker proficiency.

[0006] In view of the above circumstances, an object of one aspect of the present invention is to provide a technique for improving the efficiency and quality variations of work performed under a microscope.

[0007] A work improvement support system according to one aspect of the present invention is a work improvement support system that supports the improvement of work in assembling or inspecting equipment using a microscope device, and includes: a work data generation unit that generates work data by attaching tag information related to the work at the time the microscope image was acquired to a microscope image acquired by the microscope device during the work of assembling or inspecting the equipment; a quality information generation unit that generates quality information related to the quality of the work for each classification of work data classified by the tag information based on multiple work data generated by the work data generation unit; and a display control unit that displays the quality information generated by the quality information generation unit on a display device.

[0008] In one aspect of the present invention, a work improvement support method is provided in which a computer generates work data by attaching tag information related to the work performed at the time the microscopic image was acquired to a microscopic image acquired by a microscope device during the assembly or inspection of equipment using the microscope device, and then generates quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information based on the multiple work data generated, and displays the generated quality information on a display device.

[0009] A program according to one aspect of the present invention causes a computer to perform the following processes: generate work data by attaching tag information related to the work performed at the time the microscopic image was acquired to a microscopic image acquired by the microscope device during the assembly or inspection of equipment using the microscope device; generate quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information based on the generated multiple work data; and display the generated quality information on a display device.

[0010] According to the above aspect, it is possible to improve the efficiency and quality of work performed under a microscope.

[0011] FIG. 1 is a diagram showing the configuration of a work improvement support system according to an embodiment. FIG. 2 is a diagram explaining tag information that the work improvement support system attaches to a microscope image. FIG. 3 is a diagram explaining the flow of data processing performed by the work improvement support system. FIG. 4 is a diagram explaining assembly master information held by the work improvement support system. FIG. 5 is a diagram illustrating the configuration of a microscope system according to an embodiment. FIG. 6 is a diagram illustrating the configuration of an optical system included in the microscope system. FIG. 7 is a diagram illustrating the configuration of a slide set. FIG. 8 is an example of an application screen displayed on a display device. FIG. 9 is a diagram showing an example of tag information. FIG. 10 is a diagram showing another example of tag information. FIG. 11 is a diagram showing an example of means for selecting tag information. FIG. 12 is a diagram showing yet another example of tag information. FIG. 13 is another example of an application screen displayed on a display device. FIG. 14 is yet another example of an application screen displayed on a display device. FIG. 15 is yet another example of an application screen displayed on a display device. FIG. 16 is an example of an image observed by looking through an eyepiece. FIG. 17 is another example of an image observed by looking through an eyepiece. FIG. 18 is a diagram illustrating the hardware configuration of a computer for realizing a control device.

[0012] 1 is a diagram showing the configuration of a work improvement support system 1 according to one embodiment. The work improvement support system 1 is a system that supports the improvement of work performed using a microscope device 10, such as the assembly or inspection of precision equipment, typified by medical equipment. However, the use of the work improvement support system 1 is not limited to precision equipment. The system can be applied to any work performed using the microscope device 10.

[0013] The importance of work improvement efforts has long been recognized, and improvement activities consisting of the following steps are generally recommended at precision equipment manufacturing sites: (Step 1) Find the defect. (Step 2) Analyze the defect. (Step 3) Use the analysis results to improve subsequent work.

[0014] In addition, images (including videos) have also been collected as work records necessary for the above-mentioned improvement activities. However, in many cases, images collected as work records have simply been stored on a server, and are managed in a manner that makes them extremely difficult to search. This makes it difficult to find defects from the work records after the fact, and the reality is that it is difficult to effectively carry out improvement activities that utilize the work records.

[0015] The work improvement support system 1 was devised in light of this situation, and it tags work records to enable their effective use, automatically classifies and analyzes them, and creates and displays quality information that contributes to work improvement for each classification. This makes it possible to improve the variations in work quality and efficiency caused by the proficiency of workers, even in the manufacture of precision equipment, which requires highly difficult work.

[0016] Fig. 2 is a diagram illustrating tag information 50 that the work improvement support system 1 attaches to a microscopic image 40. Fig. 3 is a diagram illustrating the flow of data processing performed by the work improvement support system 1. Fig. 4 is a diagram illustrating assembly master information 90 held by the work improvement support system 1. The configuration and operation of the work improvement support system 1 will be described below with reference to Figs. 1 to 4.

[0017] 1, the work improvement support system 1 includes an information processing device 20 that processes microscope images acquired by a microscope device 10 during the assembly or inspection of precision equipment. The work improvement support system 1 may further include the microscope device 10 and a display device 30.

[0018] The microscope device 10 is a microscope device equipped with an imaging device, and acquires a microscope image 40 during the assembly or inspection of a precision instrument. The microscope device 10 may be, for example, a binocular stereo microscope, a metallurgical microscope, a measuring microscope, a digital microscope, a laser microscope, or an AR (Augmented Reality) microscope (described later).

[0019] The display device 30 is, for example, a display device connected to the information processing device 20, a display device within a projector equipped in an AR microscope, or a display device of another information processing device (e.g., a tablet terminal or smart glasses) that communicates with the information processing device 20.

[0020] As shown in FIG. 1, the information processing device 20 includes a work data generating unit 21, a quality information generating unit 22, and a display control unit 23.

[0021] 2 and 3 , in the information processing device 20, the work data generation unit 21 generates work data 60 by attaching tag information 50 related to the work at the time the microscopic image 40 was acquired to a microscopic image 40 acquired by the microscope device 10 during the assembly or inspection of a precision instrument. The microscopic image 40 to which the tag information 50 is attached by the work data generation unit 21 may be a still image or a moving image. As the work data generation unit 21 repeatedly generates the work data 60, multiple pieces of work data 60 are accumulated in the information processing device 20 (step S1). Note that hereinafter, a collection of work data 60 will be referred to as a work data set 70.

[0022] The tag information 50 attached to the microscopic image 40 by the work data generation unit 21 is used at least to classify the work data. As shown in Fig. 2, the tag information 50 preferably includes work content information 51 that identifies the work performed when the microscopic image 40 was acquired, and work result information 53 that indicates an evaluation of the results of the work performed when the microscopic image 40 was acquired. As shown in Fig. 2, the tag information 50 preferably further includes work condition information 52 that identifies the conditions of the work performed when the microscopic image 40 was acquired.

[0023] The hierarchical structure of the multiple assemblies that make up the precision device, and the work and procedures for assembling and manufacturing each assembly are known in advance and are managed as assembly master information 90 shown in Fig. 3 in the information processing device 20. The work content information 51 may be information that identifies the work performed when the microscopic image 40 was acquired by referring to the assembly master information 90. The work content information 51 may include, for example, work type information that identifies the type of work R1 performed when the microscopic image 40 was acquired, work procedure information that identifies the work procedure R2 performed when the microscopic image 40 was acquired, and work stage information that identifies the work stage R3 performed when the microscopic image 40 was acquired, as shown in Figs. 2 and 4, respectively.

[0024] 2, the work condition information 52 may include worker information identifying the worker, procedure manual information identifying the procedure manual to be referred to by the worker during work, camera setting information typified by the gain and exposure time of the imaging device, microscope status information typified by the microscopy method, magnification, and illumination light intensity, specimen information typified by the specimen serial number and lot number, specimen status information typified by the specimen posture, time information such as the allowable work time, information on the start and end times of work (e.g., 9:00 AM, 11:00 AM, etc.), and any other character information.

[0025] In addition, the work result information 53 is information that qualitatively or quantitatively evaluates the results of the work and information that serves as the basis for that evaluation, and may include, for example, as shown in Figure 2, information on the time required for the work, the time (including date and time) when the work started and the time (including date and time) when the work was completed (for example, 9:00 AM and 11:00 AM on May 1, 2024), information indicating the quality of the work, information that serves as the basis for judging whether the work was good or bad, information on measured work results, and information indicating the quality of the product that is the final result of the work.

[0026] There are no particular limitations on the timing at which the work data generation unit 21 performs the process of attaching tag information 50. For example, the work data generation unit 21 may attach tag information 50 each time a microscopic image 40 is acquired. Furthermore, the work data generation unit 21 may attach tag information 50 periodically or irregularly to one or more microscopic images 40 acquired and accumulated by the microscope device 10. Furthermore, the tag information 50 attached to a microscopic image 40 may be changed later. For example, after a worker attaches tag information 50, an administrator may update and change the tag information 50.

[0027] The work data generation unit 21 may generate tag information 50 to be attached to the microscopic image 40 based on an input operation by the worker, and attach the generated tag information 50 to the microscopic image 40. Alternatively, the work data generation unit 21 may automatically generate tag information 50 to be attached to the microscopic image 40 based on information such as the analysis results of the microscopic image 40 and the status of the microscope device 10 and the information processing device 20, and attach the generated tag information 50 to the microscopic image 40.

[0028] 3, the quality information generation unit 22 generates quality information 80 related to the quality of the work for each classification of the work data 60 classified by the tag information 50, based on the multiple work data (work data set 70) generated by the work data generation unit 21. Furthermore, the display control unit 23 causes the display device 30 to display the quality information 80 generated by the work data generation unit 21.

[0029] More specifically, as shown in FIG. 3, the quality information generation unit 22 classifies the working data set 70 into a plurality of working data sets 70 (working data set 71, working data set 72, working data set 73, working data set 74) based on the tag information 50 (step S2), and generates quality information 80 for each classified working data set (step S3).

[0030] It is desirable that the quality information generating unit 22 analyzes the plurality of pieces of work result information 53 included in the plurality of pieces of work data 60, and generates quality information 80 for each classification based on the plurality of pieces of work content information 51 included in the plurality of pieces of work data 60. For example, the quality information generating unit 22 may classify the work data 60 into each process of precision equipment assembly work based on the work content information 51, and generate quality information 80 for each process.

[0031] Furthermore, the quality information generating unit 22 may analyze the multiple pieces of work result information 53 included in the multiple pieces of work data 60, and generate quality information 80 for each classification based on the multiple pieces of work content information 51 and the multiple pieces of work condition information 52 included in the multiple pieces of work data 60. For example, the quality information generating unit 22 may classify the work data 60 into combinations of precision equipment assembly work processes and workers based on the work content information 51 and the work condition information 52, and generate quality information 80 for each combination of process and worker.

[0032] There are no particular limitations on the timing at which the quality information generating unit 22 performs the process of generating the quality information 80. The process of generating the quality information 80 may be performed before the display control unit 23 causes the display device 30 to display the quality information 80. The quality information generating unit 22 may generate the quality information 80, for example, in synchronization with the process of displaying the quality information 80. Furthermore, the quality information generating unit 22 may generate the quality information 80 periodically or irregularly, for example, regardless of the process of displaying the quality information 80.

[0033] There are no particular limitations on the timing at which the display control unit 23 performs the process of displaying the quality information 80 on the display device 30. For example, the display control unit 23 may cause the display device 30 to display the quality information 80 of a classification corresponding to a current task during the assembly or inspection of a precision device using the microscope device 10. Furthermore, the display control unit 23 may cause the display device 30 to display the quality information 80 outside of the work period for assembling or inspecting the precision device. Furthermore, when the quality information 80 is to be displayed on the display device 30 outside of the work period for assembling or inspecting the precision device, the display control unit 23 may, for example, cause the display device 30 to display the quality information 80 of a classification corresponding to a task identified based on input information in response to an input operation by a worker using an input device.

[0034] As described above, in the work improvement support system 1, work records are managed as work data 60 in which work-related tag information 50 is attached to microscopic images 40. The work data 60 (work data set 70), which is the accumulated work records, is then automatically categorized based on the tag information, and quality information 80 is generated and displayed for each category.

[0035] Therefore, with the work improvement support system 1, not only is data accumulation possible, but the entire process from data accumulation to information generation and information display (feedback) is designed in advance, making it possible to easily provide feedback from work records to relevant parties such as workers and managers that contribute to work improvement. As a result, it is possible to improve the efficiency and quality of work performed under a microscope.

[0036] Furthermore, the work improvement support system 1 classifies the work data 60 by work content information 51 and generates quality information 80 from the work result information 53 attached to the work data 60 (microscope images 40) for each classification. This allows information according to the work process and the work target to be fed back to the relevant parties. This allows the relevant parties to specifically recognize areas that need improvement for each individual work, making it easier to improve the work.

[0037] Furthermore, the work improvement support system 1 classifies work data 60 by work content information 51 and work condition information 52, and generates quality information 80 from work result information 53 attached to the work data 60 (microscope images 40) for each classification. This allows information according to the work conditions, in addition to the work process and work target, to be fed back to the relevant parties. This allows the relevant parties to specifically recognize areas that need improvement for individual work under specific work conditions, making work improvements even easier.

[0038] Furthermore, the work improvement support system 1 displays quality information 80 of the category corresponding to the current work on the display device 30 while the worker is assembling or inspecting a precision device. This allows the worker to check the quality information 80 related to the work while he or she is working. Therefore, the work improvement support system 1 can more reliably improve the efficiency and quality of the work.

[0039] Furthermore, the task improvement support system 1 displays on the display device 30 quality information 80 for the category corresponding to the task identified based on information input using the input device. This makes it possible to obtain information even outside the task period. Therefore, the task improvement support system 1 can provide necessary information not only to workers but also to other relevant parties such as managers.

[0040] Although not specifically mentioned, the work data and the various data that make it up (microscope images, tag information, quality information) may be recorded in the recording device of the information processing device 20, or may be recorded in other storage devices (for example, storage in a dedicated database server, storage on the cloud).

[0041] The configuration of a microscope system 2 in which the work improvement support system 1 is applied to an AR microscope will be described below. Fig. 5 is a diagram illustrating an example of the configuration of the microscope system 2 according to one embodiment. Fig. 6 is a diagram illustrating an example of the configuration of an optical system 110 included in the microscope system 2. Fig. 7 is a diagram illustrating an example of the configuration of a slide set 600.

[0042] The microscope system 2 shown in Fig. 5 uses a prepared slide set to provide appropriate information to an operator at the necessary timing while working under a microscope, and is an example of the work improvement support system described above. The configuration of the microscope system 2 will be described with reference to Figs. 5 to 7.

[0043] The microscope system 2 includes a microscope device 100 , a control device 200 , a display device 300 , a plurality of input devices 400 (a mouse 401 , a keyboard 402 , a foot switch 403 , and a barcode reader 404 ), and a web camera 500 .

[0044] The microscope apparatus 100 is an example of the microscope apparatus 10 described above. The microscope apparatus 100 is a stereo microscope capable of stereoscopically viewing a sample, and is an AR microscope that projects the above-described quality information 80 onto an image plane where an optical image is formed. The microscope apparatus 100 includes a microscope optical system 110 shown in FIG. 6 .

[0045] The microscope optical system 110 is an optical system for a stereomicroscope. A user can observe the optical image formed by the microscope optical system 110 on the object side of the eyepieces 106 (eyepieces 106a and 106b) with both their left and right eyes via the eyepieces 106, allowing them to observe the sample in three dimensions. For this reason, the microscope device 100 is suitable for applications such as precision equipment assembly work.

[0046] The microscope device 100 includes zoom lenses 102 (zoom lenses 102a and 102b) that can be operated with a zoom handle 130. By operating the zoom handle 130, the observation magnification can be changed while continuing to observe a sample by looking through the eyepiece 106.

[0047] The microscope device 100 includes a focusing handle 140. By operating the focusing handle 140, the distance between the sample and the objective lens 101 can be changed to bring the sample into focus.

[0048] The microscope apparatus 100 includes an imaging device 112 that captures an image of a sample and acquires a microscopic image of the sample. An eyepiece tube 120 to which the eyepiece 106 is attached is a triocular tube, and the imaging device 112 is attached to the eyepiece tube 120. The imaging device 112 is provided with a two-dimensional image sensor. The image sensor is not particularly limited, but may be, for example, a CCD image sensor or a CMOS image sensor. The microscopic image 40 acquired by the imaging device 112 is output to the control device 200. Alternatively, the microscopic image 40 may be output directly to the display device 300.

[0049] 6, light split by a beam splitter 103a, such as a half mirror, from one of the left and right optical paths of the microscope optical system 110 enters the imaging device 112 via an imaging lens 111. Note that an ND prism 103b is provided in the other optical path to compensate for the optical path length between the left and right optical paths caused by the beam splitter 103a and to suppress the difference in light intensity.

[0050] The microscope device 100 includes a projector 113 that projects information onto an image plane where the imaging lens 105 (imaging lens 105a, imaging lens 105b) forms an optical image. The projector 113 is a device that projects and superimposes information onto the image plane in accordance with commands from the control device 200. More specifically, the projector 113 superimposes, for example, the above-described quality information 80 onto the image plane. Note that the type of the projector 113 is not particularly limited. The projector 113 may be configured using a display device such as a liquid crystal device or a digital mirror device. The projector 113 is an example of the above-described display device 30.

[0051] The projector 113 is provided inside the eyepiece tube 120. Light from the projector 113 is guided to the left and right optical paths of the microscope optical system 110 via a projection lens 114 and a plurality of beam splitters (beam splitter 115, beam splitter 104a, beam splitter 104b).

[0052] 5, the eyepiece tube 120 is provided with an operation unit 121. By operating the operation unit 121, the user can switch the projector 113 on and off and instruct the projector to start or stop superimposing information on the image surface.

[0053] The control device 200 is a device that controls the microscope device 100 and the display device 300, and is an example of the above-mentioned information processing device 20. The control device 200 generates screen data for an application during the assembly or inspection of a precision device, and displays the application screen on the display device 300 based on the screen data. The screen data is generated using a slide set 600, exemplified in FIG. 7 , which is stored in advance in the control device 200.

[0054] Slide set 600 is made up of multiple slides (slide set 601, slide set 602, slide set 603, slide set 604, slide set 605, slide set 606). Slide set 600 is, for example, information that supports the assembly or inspection of precision equipment, and more specifically, corresponds to a procedure manual for assembly or inspection work. The order of the multiple slides is also predetermined. In other words, slide set 600 is made up of multiple slides that have been ordered in advance.

[0055] Each of the multiple slides contains information on each process of assembling or inspecting a precision device, and more specifically, the information includes the work content of each process.

[0056] The control device 200 generates screen data based on information about a target slide selected from the multiple slides included in the slide set 600. By the control device 200 appropriately switching the target slide, the microscope system 2 can switch the application screen displayed on the display device 300 to provide the user with appropriate information according to the work process.

[0057] Similarly to the information processing device 20, the control device 200 includes a work data generation unit 21, a quality information generation unit 22, and a display control unit 23. The control device 200 causes the display device 300 and the display device of the projector 113 to display quality information 80.

[0058] Specifically, the control device 200 controls the display device 300 so that, during the assembly or inspection of the precision device, the quality information 80 is displayed on an application screen displayed on the display device 300. Furthermore, during the assembly or inspection of the precision device, the control device 200 outputs the quality information 80 to the projector 113 and causes the display device of the projector 113 to display the quality information 80. As a result, the quality information 80 is superimposed on the image plane by the projector 113, and the user of the microscope device 100 can view the quality information 80 superimposed on the optical image by looking through the eyepiece 106.

[0059] The display device 300 and the input device 400 are connected to the control device 200. The display device 300 is, for example, a liquid crystal display or an organic EL display, and is an example of the above-mentioned display device 30. The web camera 500 transmits captured images to the control device 200 via a network such as the Internet. The web camera 500 captures, for example, an image of a user using the microscope system 2.

[0060] Fig. 8 is an example of an application screen displayed on the display device. Fig. 9 is a diagram showing an example of tag information. Fig. 10 is a diagram showing another example of tag information. Fig. 11 is a diagram showing an example of means for selecting tag information. Fig. 12 is a diagram showing yet another example of tag information. Hereinafter, with reference to Figs. 8 to 12, a specific description will be given of the process of generating work data performed by the microscope system 2 during assembly work.

[0061] The control device 200 executes a software application (hereinafter referred to as a work support application) that supports assembly or inspection work performed under a microscope. As a result, an application screen corresponding to the work process based on the procedure manual is displayed on the display device 300. An instruction to switch the work process can be given, for example, by pressing a switch provided on the zoom handle 130 or by operating the mouse 401 or foot switch 403. Note that the instruction to switch the work process may be given based on the results of analysis of a microscope image (live image), and artificial intelligence may be used for the image analysis.

[0062] 8 is an example of a screen displayed during the work of attaching a chip resistor to assembly C1. Window 301 includes a display area 305 for selecting a slide, a display area 310 for displaying the work procedure, a display area 320 for displaying the latest microscope image acquired by microscope device 100, a display area 330 for displaying an explanation of the work procedure, an input area 340 for inputting the work results, and a button 350 for inputting image capture instructions.

[0063] While performing chip resistor installation work using the microscope device 100, the worker can check the window 301 displayed on the display device 300 to confirm the work procedure for the chip resistor installation work, the current process (stage) in that process, and the content of the work to be done in that process.

[0064] When the worker wants to leave a work record for the current work, he or she simply selects any option from the options displayed in the input area 340 and then presses the button 350. When the button 350 is pressed, the control device 200 acquires the microscope image M1, attaches tag information to the microscope image M1, and generates and records work data.

[0065] The tag information includes information corresponding to the option selected in the input area 340 as work result information. In this example, work result information indicating the success of the work is included. The tag information also includes work content information that identifies the work. In this example, work content information indicating that the work type is a chip resistor mounting work and that the work stage is the stage of pre-soldering the lands is included. Information indicating the work type and work stage may be information on the slide currently displayed on the display device 300, such as the ID or page number of the work procedure manual. Unlike the work result information, the work content information can be automatically generated based on, for example, the information displayed in the display area 305, the display area 310, and the display area 330, without relying on input operations by the worker. Furthermore, work stage information may be automatically and retroactively generated based on information on the work time corresponding to the microscope image 40, from information on what work was being performed at each time or what work should have been performed at each time. Furthermore, the work content may be inferred from not only the factual information on what work was performed but also the time.

[0066] The work result information included in the tag information may further include work time information T1 as shown in Fig. 9. The work time information T1 may be calculated from information on the work start time and information on the work end time. Furthermore, the work result information included in the tag information may also include annotation information T2 shown in Fig. 10, which the worker added to the microscope image as the basis for the work evaluation result entered in the input area 340.

[0067] Furthermore, when an input selecting NG is made in input area 340, the options in input area 340 may be switched to options 341 shown in Fig. 11. Information corresponding to one or more options selected from options 341 may be included in the tag information as work result information indicating the basis for the evaluation result of the work.

[0068] Furthermore, the control device 200 may analyze the microscope image and measure the object in the image, and may include the measurement results in the tag information as work result information. For example, as shown in FIG. 12, the control device 200 may analyze the microscope image M2 acquired after the work is completed and measure the size of the solder area, and may include the measurement results in the tag information as work result information. The control device 200 may also determine whether the measurement results are within a preset tolerance range and include the determination result in the tag information as work result information. In other words, the control device 200 may automatically generate the work result information.

[0069] Furthermore, the tag information may include work condition information. The work condition information may include information for identifying the worker, such as information entered from the keyboard 402 when launching the work support application and logging in, or information acquired by reading the barcode on the worker's name tag with the barcode reader 404. The work condition information may also include sample information read from an RFID tag attached to the sample or container. The work condition information may also include information for identifying information provided to the user during work, such as ID information of a procedure manual selected in the work support application. The work condition information may also include settings of the microscope device 100 (e.g., magnification, type of objective lens, gain, exposure time, illumination light intensity), etc.

[0070] In this way, in the microscope system 2, when an operator records a microscope image as a work record, work data is generated by automatically or manually attaching tag information to the microscope image. Furthermore, the generated work data is classified based on the work content information included in the tag information, and, for example, quality information is generated based on work result information for each work process.

[0071] The quality information may be, for example, statistical information on the options selected in the input area 340 (for example, the respective proportions of OK, NG, and ambiguous). The quality information may also be statistical information on the options selected as the basis for the NG evaluation from the options 341 (for example, ranking information on the reasons for NG). The quality information may also be information (for example, average values) obtained by statistically processing measurements such as work time and the size of a specified area. The following analyses may also be performed using work time information: a) Trend information on the quality of work at the time of work (night or morning) b) Trend information by season c) Sorting of work result information

[0072] 13 to 16 are each another example of an application screen displayed on the display device 300. Hereinafter, the quality information display processing performed by the microscope system 2 during assembly work will be specifically described with reference to FIGS.

[0073] Window 301 shown in FIG. 13 is an example of a screen displayed during the work of attaching chip resistors for assembly C1, and is similar to the example screen shown in FIG. 8 except that quality information Q1 and button 360 are displayed.

[0074] 13, the control device 200 may display quality information Q1 on an application screen corresponding to the work process. The control device 200 identifies the current work based on the information displayed in the display areas 305, 310, and 330, obtains quality information Q1 for the category corresponding to the identified work, and displays the quality information Q1 on the display device 300. In this example, the quality information Q1 is ranking information for defects in the current work (in this example, the work of attaching chip resistors).

[0075] 13, the quality information is displayed on the latest microscope image M1 (i.e., a live image) acquired by the microscope device 100, but the display position of the quality information is not limited to the microscope image M1. Also, while Fig. 13 shows an example in which quality information in a work process is displayed regardless of the number of occurrences of defects, frequency, magnitude of impact, etc., whether or not to display the quality information may be determined based on the number of occurrences of defects, frequency, impact, and the current time.

[0076] If the worker wants to check the quality information in more detail, he or she may check additional quality information by pressing button 360. Fig. 14 shows how window 302 pops up when button 360 is pressed. Window 302 contains quality information (quality information Q2 and quality information Q3) that is different from quality information Q1.

[0077] The quality information Q2 is an example of information calculated by statistically calculating the work result information, and includes the average work time, the first-pass rate, and the process capability index (Cp, Cpk). The quality information Q3 is an example of information that graphically displays the statistical results of the work result information, and is a graph showing the frequency distribution of the work time.

[0078] While FIG. 13 illustrates an example in which the microscope system 2 automatically displays quality information during assembly work, the quality information may also be displayed when an operator explicitly requests display, as shown in FIG. 15 . The control device 200 may specify an image using the input area 371 shown in FIG. 15 to display quality information Q4 for a category corresponding to the specified image in the display area 380. Furthermore, the control device 200 may input a keyword in the input area 372 shown in FIG. 15 to display quality information Q4 for a category related to the keyword in the display area 380. Here, a keyword is any word, including tag information. Furthermore, by inputting a date, time period, or the like instead of a keyword, quality information Q4 for a category related to the input date, time, or time period may be displayed.

[0079] Note that the information searched for by input into input areas 371 and 372 is not limited to quality information. The control device 200 may identify related tasks from the input into input areas 371 and 372, and display, for example, a non-defective product image M3 (also referred to as a model image or reference image) that has been recorded in advance in association with the task in the display area 380, as shown in Fig. 16 . The control device 200 may also display the non-defective product image M3 together with the quality information Q4.

[0080] As described above, the microscope system 2 can alert the worker by displaying quality information related to the current task during assembly work. This can improve the quality and efficiency of the work. Furthermore, as shown in FIG. 13 , by automatically displaying quality information on the application screen according to the task, the worker can be forced to check the quality information. Furthermore, as shown in FIG. 15 , by displaying quality information in response to the worker's request, as shown in FIGS. 14 and 15 , unnecessary displays can be reduced, preventing excessive distraction from the worker's focus on the task. Furthermore, as shown in FIG. 16 , a reference image may be displayed instead of or in addition to the quality information.

[0081] FIG. 17 shows an example of an image observed by looking through the eyepiece 106. While FIGS. 13 to 15 illustrate examples in which the microscope system 2 displays quality information on the display device 300 during assembly work, the quality information may also be displayed on the display device of the projector 113. The control device 200 displays the quality information Q1 on the display device of the projector 113, whereby the quality information Q1 is projected and superimposed on the image M4 (optical image) of the specimen, as shown in FIG. 17 . This allows the worker to check the quality information Q1 while observing the specimen through the eyepiece 106. In other words, the quality information Q1 can be checked while continuing work under the microscope device 10, thereby improving the quality of the work, etc., without sacrificing operational efficiency. Furthermore, as shown in FIG. 17 , the control device 200 may also display explanatory information U1 regarding the work procedure on the display device of the projector 113 in addition to the quality information Q1. This allows the worker to check the quality information Q1 and explanatory information U1 regarding the work procedure while observing the specimen through the eyepiece 106. Therefore, it is possible to simultaneously improve the efficiency and quality of work while continuing to work under the microscope device 10.

[0082] FIG. 18 shows another example of an image observed by looking through the eyepiece 106. While FIG. 17 illustrates an example in which quality information and other information are displayed on an optical image, such information does not necessarily have to be displayed on the optical image. As shown in FIG. 18 , the control device 200 may project an image onto the image plane using light from the display device of the projector 113, while blocking light passing through the objective lens 101 using a light-blocking means disposed on the optical path of the microscope optical system 110. The image projected onto the image plane may be an image M5 of a menu screen or an image M6 of a search screen. While these images are displayed, an image M7 containing quality information and other information may be projected onto the image plane and observed through the eyepiece 106 in response to an operation performed by the operator.

[0083] 19 is a diagram illustrating an example of the hardware configuration of a computer 200a for realizing the control device 200 according to the embodiment described above. The hardware configuration illustrated in Fig. 19 includes, for example, a processor 201, a memory 202, a storage device 203, a reading device 204, a communication interface 206, and an input / output interface 207. The processor 201, the memory 202, the storage device 203, the reading device 204, the communication interface 206, and the input / output interface 207 are connected to one another via, for example, a bus 208.

[0084] The processor 201 may be, for example, a single processor, a multiprocessor, or a multi-core processor. The processor 201 reads and executes programs stored in the storage device 203, thereby functioning as the operation data generation unit 21, the quality information generation unit 22, and the display control unit 23.

[0085] The memory 202 is, for example, a semiconductor memory and may include a RAM area and a ROM area. The storage device 203 is, for example, a semiconductor memory such as a hard disk or a flash memory, or an external storage device.

[0086] The reader 204 accesses the removable storage medium 205 in accordance with instructions from the processor 201, for example. The removable storage medium 205 is realized by, for example, a semiconductor device, a medium for inputting and outputting information by magnetic action, or a medium for inputting and outputting information by optical action. An example of a semiconductor device is a USB (Universal Serial Bus) memory. An example of a medium for inputting and outputting information by magnetic action is a magnetic disk. An example of a medium for inputting and outputting information by optical action is a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc), a Blu-ray Disc, etc. (Blu-ray is a registered trademark).

[0087] The communication interface 206 communicates with other devices (e.g., the microscope device 100, the Web camera 500, etc.) according to instructions from the processor 201, for example. The input / output interface 207 is, for example, an interface between an input device 400 and an output device. The input device 400 is, for example, a device such as a mouse 401, a keyboard 402, or a foot switch 403 that accepts instructions from a user. The output device is, for example, a display device 300 and an audio device such as a speaker.

[0088] The program executed by the processor 201 is provided to the computer in the following forms, for example: (1) pre-installed in the storage device 203; (2) provided by a removable storage medium 205; or (3) provided from a server such as a program server.

[0089] Note that the hardware configuration of the computer for realizing the control device described with reference to FIG. 19 is an example, and the embodiment is not limited to this. For example, part of the above-described configuration may be deleted, or new configuration may be added. Furthermore, in another embodiment, for example, part or all of the functions of the above-described processing device may be implemented as hardware using an FPGA (Field Programmable Gate Array), a SoC (System-on-a-Chip), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or the like.

[0090] The above-described embodiments are illustrative examples provided to facilitate understanding of the invention, and the present invention is not limited to these embodiments. Modifications of the above-described embodiments and alternatives to the above-described embodiments may be included. In other words, the components of each embodiment may be modified without departing from the spirit and scope of the embodiment. Furthermore, new embodiments can be implemented by appropriately combining multiple components disclosed in one or more embodiments. Furthermore, some components may be deleted from the components shown in each embodiment, or some components may be added to the components shown in each embodiment. Furthermore, the order of the processing steps shown in each embodiment may be reversed as long as there is no contradiction.

[0091] In this specification, the expression "based on A" does not mean "based only on A," but also means "based at least on A," and further means "based at least partially on A." That is, "based on A" may be based on B in addition to A, or may be based on a part of A.

[0092] REFERENCE SIGNS LIST 1: Work improvement support system 2: Microscope system 10, 100: Microscope device 20: Information processing device 21: Work data generation unit 22: Quality information generation unit 23: Display control unit 30, 300: Display device 40: Microscope image 50: Tag information 51: Work content information 52: Work condition information 53: Work result information 60: Work data 70, 71, 72, 73, 74: Work data set 80, Q1, Q2, Q3, Q4: Quality information 90: Assembly master information 106, 106a, 106b: Eyepiece 110: Microscope optical system 112: Imaging device 113: Projector 200: Control device 200a: Computer 201: Processor

Claims

1. A work improvement support system comprising: a work data generation unit that generates work data by attaching tag information related to the work at the time of acquisition of a microscopic image acquired by a microscope device during the work of assembling or inspecting equipment using the microscope device; a quality information generation unit that generates quality information related to the quality of the work for each classification of work data classified by the tag information based on multiple work data generated by the work data generation unit; and a display control unit that displays the quality information generated by the quality information generation unit on a display device.

2. A work improvement support system as described in claim 1, wherein the tag information includes work content information that identifies the work at the time the microscopic image was acquired, and work result information that indicates an evaluation of the result of the work at the time the microscopic image was acquired, and the quality information generation unit analyzes a plurality of pieces of work result information included in the plurality of work data, and generates the quality information for each classification classified based on the plurality of work content information included in the plurality of work data.

3. A work improvement support system as described in claim 2, wherein the tag information further includes work condition information specifying the conditions of the work at the time the microscopic image was acquired, and the quality information generation unit analyzes the multiple work result information and generates the quality information for each classification classified based on the multiple work content information and multiple work condition information contained in the multiple work data.

4. A work improvement support system as claimed in claim 1 or 2, characterized in that the display control unit causes the display device to display quality information of a classification corresponding to a current task during the assembly or inspection of the equipment.

5. A work improvement support system as claimed in claim 4, further comprising the microscope device, the microscope device comprising: an imaging device for imaging a sample; a microscope optical system including an eyepiece and forming an optical image of the sample on the object side of the eyepiece; and the display device for superimposing the quality information on the image plane on which the optical image is formed.

6. A work improvement support system as claimed in claim 1 or 2, characterized in that the display control unit causes the display device to display quality information of a classification corresponding to a task identified based on information input using an input device.

7. A work improvement support method characterized by carrying out a process in which a computer generates work data by attaching tag information related to the work at the time the microscopic image was acquired to a microscopic image acquired by a microscope device during the work of assembling or inspecting equipment using the microscope device; based on the generated multiple work data, generates quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information; and displays the generated quality information on a display device.

8. A program causing a computer to execute the following processes: a process of generating work data by attaching tag information related to the work at the time the microscopic image was acquired to a microscopic image acquired by a microscope device during the assembly or inspection of equipment using the microscope device; a process of generating quality information, which is statistical information related to the quality of the work, for each classification of work data classified by the tag information based on the multiple generated work data; and a process of displaying the generated quality information on a display device.

Images