Information processing system, and information processing method

Abstract

To shorten an operation stop time of a surface mounting machine due to a recognition error.SOLUTION: An information processing system 60 includes surface mounting machines 12, a production management device 40, and a portable terminal 50 communicable with the production management device by wireless. The surface mounting machines transmit error information, a recognition parameter used for image recognition, and camera information to the portable device via the production management device if operation stops due to an occurrence of a recognition error. The portable terminal displays an editing screen of the recognition parameter and transmits an edited recognition parameter to the surface mounting machines via the production management device when the recognition parameter is edited, and the surface mounting machines apply the edited recognition parameter to re-recognize an object image, and transmit a recognition result by the edited recognition parameter to the portable terminal via the production management device. The portable terminal transmits an instruction to restart the operation to the surface mounting machines via the production management device when image recognition is succeeded by the edited recognition parameter.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and an information processing method. [Background technology]

[0002] The surface mounter uses a camera to capture images of objects such as electronic components to be mounted on a board and marks affixed to the board, and performs image recognition. Patent Document 1 listed below is an example of a document disclosing this type of technology. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-017554 A Summary of the Invention [Problem to be solved by the invention]

[0004] When a surface mounter fails to recognize the image of an object (hereafter referred to as a recognition error), it may stop operating. To restart the surface mounter, an operator may need to go to the site to correct the data required for recovery, which can take a long time to recover. An object of the present invention is to achieve early recovery by remotely correcting data required for recovery when a surface mounting machine stops due to a recognition error. [Means for solving the problem]

[0005] (1) An information processing system according to the present invention includes a surface mounter, a production management device capable of communicating with the surface mounter, and a mobile terminal capable of wirelessly communicating with the production management device. When the surface mounting machine stops operating due to the occurrence of a recognition error, the surface mounting machine transmits error information, the recognition parameters used for image recognition of the object, and camera information to the mobile terminal via the production management device. The portable terminal displays an editing screen for the recognition parameters, and when the recognition parameters are edited, transmits the edited recognition parameters to the surface mount machine via the production management device. The surface mount machine re-recognizes the image of the target object by applying the edited recognition parameters, and transmits the recognition result based on the edited recognition parameters to the portable terminal via the production management device. When image recognition is successful using the edited recognition parameters, the portable terminal transmits an instruction to resume operation to the surface mount machine via the production management device.

[0006] Even if an operator or serviceman is located far away from the surface mount machine, he or she can understand the error occurrence situation from the error information transferred to the mobile device and edit (correct) the recognition parameters used for image recognition on the mobile device. In other words, because recognition errors can be dealt with remotely, the downtime of the surface mount machine can be shortened compared to when remote handling is not possible.

[0007] (2) In the information processing system described in (1) above, the production management device may have a storage unit, and may store an edit history of the recognition parameters as edit history information in the storage unit. By leaving the edit history information, data analysis of the recognition parameters can be performed.

[0008] (3) In the information processing system described in (1) or (2) above, the production management device may transmit warning information to the mobile terminal when the production performance after editing the recognition parameters becomes worse than the production performance before editing the recognition parameters.

[0009] In this configuration, by notifying the operator that the production performance has deteriorated due to editing of the recognition parameters, it is possible to provide the operator with an opportunity to select whether or not to restore the recognition parameters. If the recognition parameters before editing are restored, the production performance can be restored to the original performance before the deterioration.

[0010] The production performance refers to the performance of the entire process of mounting components on a board, and includes, for example, the recognition success rate in image recognition processing and the mounting position accuracy of components on a board.

[0011] (4) In the information processing system described in any one of (1) to (3) above, the mobile terminal may be capable of editing the recognition parameters not only during an operation shutdown due to a recognition error, but also before production of the board begins or after production is completed.

[0012] This configuration is not limited to when the machine is stopped due to a recognition error, but also allows the recognition parameters to be edited using a mobile terminal before production starts or after production ends.

[0013] (5) In the information processing system according to any one of (1) to (4) above, the target object may include at least one of an electronic component mounted on a board, a board mark, and a base mark.

[0014] (6) The information processing system described in any one of (2) to (5) above may include a cloud server communicatively connected to the production management device, and the production management device may transmit production history information including production performance of the board and editing history information of the recognition parameters to the cloud server, and the cloud server may use the production history information and the editing history information of the recognition parameters as training data to generate a learning model that outputs a degree of impact of editing of the recognition parameters on the production performance.

[0015] In this way, an operator or a serviceman can edit the recognition parameters while referring to the results of the machine learning (degree of impact on production results).

[0016] (7) In the information processing system described in any one of (1) to (6) above, the surface mounter may transmit an image of the mounting position of the component on the board to the mobile terminal via the production management device before resuming operation.

[0017] With this configuration, when restarting operation, it is possible to check whether or not an electronic component has already been mounted at the mounting position on the board. Effect of the Invention

[0018] According to the present invention, it is possible to reduce the downtime (restoration work time) of a surface mounter due to a recognition error. [Brief description of the drawings]

[0019] [Figure 1] Overall view of the production system (information processing system) [Diagram 2] Plan view of surface mounter [Diagram 3] Side view of surface mounter [Figure 4] Block diagram of production system (information processing system) [Diagram 5] Top view of electronic components [Figure 6] Examples of recognition parameters [Figure 7] flowchart [Figure 8] Editing screen on mobile device [Figure 9] Block diagram of an information processing system according to a second embodiment. [Figure 10] Block diagram of information processing system [Figure 11] Mobile device warning screen [Figure 12] Block diagram of information processing system [Figure 13] Diagram showing the learning model DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] <Embodiment 1> 1. Production System 10 Fig. 1 is an overall view of a production system 10. The production system 10 includes a production line Lv and a production management device 40. The production line Lv includes, in order from the upstream side (the right side in Fig. 1), a printing machine 11, a surface mounter 12, an inspection machine 13, and a reflow machine 14, and these machines 11, 12, 13, and 14 are connected in series via multiple conveyors.

[0021] The printer 11 is located upstream of the surface mounter 12 and prints solder paste on the board P to be fed into the surface mounter 12 .

[0022] The surface mounter 12 is located downstream of the printer 11. The surface mounter 12 is a device that mounts electronic components E on the surface of a substrate P. There may be one or more surface mounters 12, and in this embodiment, a line configuration is provided in which three surface mounters 12 are arranged in series. The configuration of the surface mounter 12 will be described later.

[0023] The inspection machine 13 is located downstream of the surface mounter 12. The inspection machine 13 is a device that inspects the board P on which the electronic components E are mounted before the board P is input into the reflow machine .

[0024] The reflow device 14 is located downstream of the inspection machine 13. The reflow device 14 is a device that heats the inspected board P and melts the solder paste applied to the surface of the board P, thereby soldering the board P and the electronic component E together.

[0025] 2. Surface mount machine As shown in FIG. 2, the surface mounter 12 includes a base 21, a transport conveyor 22, a driving device 23, a head unit 26, a tape feeder F1, and a tray feeder F2.

[0026] In the following description, the longitudinal direction of base 21 (left-right direction in Figure 2) will be referred to as the X-direction, the depth direction of base 21 (up-down direction in Figure 2) will be referred to as the Y-direction, and the up-down direction in Figure 3 (the lifting and lowering direction of mounting head 26A) will be referred to as the Z-direction.

[0027] The transport conveyor 22 is disposed in the center of the base 21. The transport conveyor 22 includes a pair of transport belts 22A that are driven to circulate in the X direction, and transports the substrate P on the transport belts 22A in the X direction.

[0028] In this embodiment, the right side shown in Fig. 2 is the entrance, and the substrate P is carried into the machine from the right side shown in Fig. 2 via the transport conveyor 22. The carried-in substrate P is transported by the transport conveyor 22 to a working position in the center of the base.

[0029] On the base 21, tape feeders F1 and tray feeders F2 are installed at four locations around the work position of the substrate P.

[0030] The tape feeder F1 is a device that feeds a component supply tape (not shown) to a supply position to supply a tape component E1. The tape component E1 is, for example, a small electronic component such as a chip capacitor.

[0031] The tray feeder F2 is a device that supplies the tray parts E2 by drawing out a tray that contains the tray parts E2 from a storage cabinet. The tray parts E2 are, for example, medium-sized or large-sized electronic parts such as ICs.

[0032] As shown in FIGS. 2 and 3, the driving device 23 is composed of a pair of Y beams 24, an X beam 25, a Y-axis ball screw 24A, a Y-axis motor 24B, an X-axis ball screw 25A, and an X-axis motor 25B.

[0033] 2, the pair of Y beams 24 are located on both sides in the X direction of the base 21. The pair of Y beams 24 extend parallel to the Y direction, and support the X beam 25 slidably in the Y direction.

[0034] A Y-axis ball screw 24A extending in the Y direction and a Y-axis motor 24B at its shaft end are disposed on the upper surface of the Y beam 24. The Y-axis ball screw 24A is a drive shaft in the Y-axis direction, and a ball nut (not shown) fixed to the lower surface of the X beam 25 is screwed into the Y-axis ball screw 24A.

[0035] By driving the Y-axis motor 24B to rotate the Y-axis ball screw 24A about its axis, the X-beam 25 and the head unit 26 can be moved in the Y direction (Y-axis servo mechanism).

[0036] X-beam 25 is elongated in the X-direction, and supports head unit 26 so that it can slide along the X-direction. An X-axis ball screw 25A extending in the X-direction and an X-axis motor 25B at the shaft end are disposed on the side of X-beam 25. X-axis ball screw 25A is a drive shaft in the X-axis direction, and a ball nut (not shown) fixed to head unit 26 is screwed into X-axis ball screw 25A.

[0037] By driving the X-axis motor 25B to rotate the X-axis ball screw 25A about its axis, the head unit 26 can be moved in the X direction relative to the X-beam 25 (X-axis servo mechanism).

[0038] From the above, it is possible to move head unit 26 to any position on base 21 by controlling X-axis motor 25B and Y-axis motor 24B in a composite manner.

[0039] The head unit 26 has a plurality of mounting heads 26A. The mounting heads 26A are arranged in a row in the X direction with respect to the head unit 26. Each mounting head 26A can move in the vertical direction by being driven by a Z-axis motor 26B (Z-axis servo mechanism).

[0040] The mounting head 26A is a shaft that is long in the Z direction, and has a suction nozzle at its tip (lower end). By switching the internal pressure of the suction nozzle between negative and positive pressure, it is possible to hold or release the tape component E1 and the tray component E2.

[0041] With this configuration, by operating the X-axis motor 25B, the Y-axis motor 24B, and the Z-axis motor 26B at a predetermined timing, the tape components E1 supplied by the tape feeder F1 and the tray components E2 supplied by the tray feeder F2 can be picked up by the mounting head 26A and mounted on the substrate P.

[0042] 2 denotes a component recognition camera, and in Fig. 3, reference numeral 28 denotes a board recognition camera. The component recognition camera 27 is fixed on the base 21 with its imaging surface facing upward, and captures images (bottom surface images) of the tape components E1 and tray components E2 picked up by the mounting head 26A.

[0043] The board recognition camera 28 is attached to the head unit 26 with its imaging surface facing downward, and can move together with the head unit 26 on the base 21. The board recognition camera 28 captures images of the marks provided on the board P, the correction marks on the base 21, and the like.

[0044] Next, the electrical configuration of the surface mounter 12 will be described with reference to Fig. 4. The surface mounter 12 includes an arithmetic processing unit 30, a communication unit 31, a data storage unit 32, a program storage unit 33, a motor control unit 34, a recognition unit 35, a display unit 37, an operation unit 38, and the like.

[0045] The arithmetic processing unit 30 includes a CPU, a RAM, etc. The arithmetic processing unit 30 mounts the electronic component E on the board P by controlling the motor control unit 34 in accordance with a control program.

[0046] The communication unit 31 is an interface for communicating information with the production management device 40. The communication unit 31 is communicably connected to the production management device 40 via a communication cable.

[0047] The data storage unit 32 stores recognition parameters used for image recognition, and the program storage unit 33 stores recognition algorithms for image recognition.

[0048] The recognition unit 35 is connected to the component recognition camera 27 and the board recognition camera 28, and performs image recognition of the target object based on the images output from these cameras 27, 28. Image recognition will be described in detail later.

[0049] The display unit 37 is, for example, a display. The operation unit 38 is a user interface that allows the user to perform input operations.

[0050] 3. Configuration of the production management device 40 The production management device 40 is a device that manages the production line Lv, and is, for example, a personal computer. As shown in FIG. 4, the production management device 40 includes a communication unit 41, a control unit 42, a storage unit 43, a display unit 44, and an operation unit 45.

[0051] The communication unit 41 is connected to the communication unit 31 of the surface mounter 12 via a communication cable so that they can communicate with each other. Commands (instructions) and data can be transmitted and received between the surface mounter 12 and the production management device 40. The communication unit 41 can also wirelessly communicate with a communication unit 51 of a mobile terminal 50, which will be described later. Therefore, the surface mounter 12 can transmit and receive commands and data to and from the mobile terminal 50 via the production management device 40.

[0052] The control unit 42 includes a CPU, a RAM, etc. The storage unit 43 stores information about the board P produced by the surface mounter 12 and information about the electronic components E to be mounted on the board P.

[0053] The display unit 44 is, for example, a display. The operation unit 45 is a user interface that allows the user to perform input operations.

[0054] 4. Image Recognition Processing In order to confirm the position of the substrate P transported onto the base 21 and to confirm the electronic components E to be mounted on the substrate P, the surface mounter 12 photographs and recognizes the position marks on the substrate P and the electronic components E adsorbed and held by the mounting head 26A using a component recognition camera 27 and a substrate recognition camera 28.

[0055] For example, when performing image recognition on an electronic component E2 with leads, as shown in FIG. 5A, the leads R of the electronic component E2 are identified from a captured image of the electronic component E2, and the number of leads R, lead length L, lead width d, and lead pitch P are detected.

[0056] Then, the electronic component E2 is recognized by comparing the inspection results of the leads R (number of leads, lead length L, lead width d, and lead pitch P) with the comparison data. The comparison data is design data for the inspection target portion of the leads R (number of leads, lead length L, lead width d, and lead pitch P).

[0057] In other words, if the inspection results for the leads R match the comparison data, it can be recognized as an electronic component E2 with leads. In this embodiment, if the recognition of the electronic component E2 is successful, the inclination of the electronic component E2 and the amount of deviation of the center position O are recognized and the mounting position of the electronic component E on the board P is corrected, thereby improving the mounting accuracy of the electronic component E, as shown in FIG.

[0058] In this manner, in this embodiment, the inspection target area (number of leads, lead length L, lead width d, lead pitch P) is detected from an image of the object (electronic component with leads), and the result is compared with the comparison data, thereby performing image recognition of the object (electronic component with leads).

[0059] The recognition parameters are parameters used for image recognition, and as described above, in addition to the lead R comparison data, they include information on the recognition algorithm used for image recognition, the lighting conditions when photographing the object, and the conditions for extracting the inspection target area from the image.

[0060] In this embodiment, as shown in FIG. 6A, the recognition parameters are made up of two types of parameters: a first parameter and a second parameter.

[0061] <First parameter> (1) The type of recognition algorithm used for image recognition (2) Data on the shape of electronic components E1 and E2 (external dimensions, component thickness, etc.) (3) Comparison data for the inspection target area (number of leads, lead length L, lead width d, lead pitch P)

[0062] <Second parameter> (1) Lighting conditions when capturing images (2) Conditions for extracting the inspection target area (lead R) from the captured image (detection range, threshold value) The detection range is the range to be cut out from the image as the recognition target, and the threshold is the brightness threshold value when extracting the contour or outer shape of the recognition target from the image.

[0063] The initial values ​​of these recognition parameters are transmitted from the production management device 40 to the surface mounter 12 when the operation of the surface mounter 12 starts, and are stored in the data storage unit 32 of the surface mounter 12. In this embodiment, three examples (1) to (3) are shown as the first parameters, and two examples (1) to (2) are shown as the second parameters, but these are merely examples, and one or more parameters selected arbitrarily from the examples can be used as the recognition parameters. Also, parameters other than those shown as examples can be included in the recognition parameters.

[0064] 5. Information processing system for assisting in the recovery of the surface mounter 12 If the image recognition results show that the inspection result of lead R does not match the comparison data, or if extraction of lead R from the image fails, a recognition error occurs.

[0065] When a recognition error occurs, the surface mounter 12 may stop operating, and in order to resume operation, an operator or service technician may need to go to the site and correct the data required for recovery (edit the recognition parameters). Due to these circumstances, the time it takes to recover the surface mounter 12 has been an issue.

[0066] The information processing system 60 is a system that assists in the recovery of a surface mounting machine 12 that has stopped operating due to a recognition error, and as shown in FIG. 1, is configured to include a mobile terminal 50 (FIGS. 1 and 4) in addition to the surface mounting machine 12 and production management device 40.

[0067] The mobile terminal 50 is an information communication terminal that can be carried by an operator. The mobile terminal 50 includes a communication unit 51, a control unit 52, a storage unit 53, a display unit 54, and an operation unit 57, as shown in FIG.

[0068] The communication unit 51 is connected to the communication unit 41 of the production management device 40 so as to be able to communicate wirelessly. As described above, since the production management device 40 and the surface mounter 12 are also connected so as to be able to communicate, even if the mobile terminal 50 is located away from the surface mounter 12, communication between the two is possible via the production management device 40.

[0069] In this embodiment, a smartphone equipped with a touch panel will be described as an example of the mobile terminal 50. The touch panel has the functions of both a display unit 54 and an operation unit 57 for operating the mobile terminal 50.

[0070] The control unit 52 includes a CPU, a RAM, etc. Editing application software for editing the recognition parameters is stored in the storage unit 53. The control unit 52 executes the editing application software to display the editing screen of Fig. 8, enabling editing of the recognition parameters.

[0071] 6.Remote recovery of production lines The surface mounter 12 stops operation if a recognition error occurs during production of the board P. After the operation is stopped, the remote recovery process shown in FIG.

[0072] After the remote recovery process is started, the surface mounter 12 generates error information (S11). The error information is information on the image recognition result that has resulted in a recognition error, and includes at least the recognition result information and an error image. In this embodiment, as shown in Fig. 6B, the error information includes three pieces of information: recognition failure information, recognition result information, and an error image Ie. Note that in Fig. 6B, an error image of a chip component is displayed instead of an electronic component with leads.

[0073] The recognition failure information includes an error number indicating the type of error and information regarding the mounting head 26A that held the electronic component to be recognized.

[0074] When the recognition target is an electronic component E2 with leads, the recognition result information is data on the number of leads, the lead length L, the lead width d, and the pitch P between the leads.

[0075] When the surface mounter 12 generates the error information, it transmits the data (1) to (3) to the mobile terminal 50 via the production management device 40 (S11, S21).

[0076] (1) Error information (2) The recognition parameters used for image recognition when the recognition error occurred (3) Camera information used for image recognition

[0077] The camera information (see FIG. 6C) includes the type of camera used for image recognition and the adjustment information of the camera. The type of camera is information for identifying the camera such as a model number, and information for identifying the shooting method such as scan, multi, or fixed. The camera information can also include information for identifying the shooting purpose such as FID (for shooting fiducial marks on a board) and coplanarity check (for checking the coplanarity of ball parts and IC parts). The adjustment information of the camera is information on the magnification and mounting angle of the camera. In this embodiment, before the equipment is put into operation or during maintenance, test shooting is performed, and the magnification and mounting angle of the cameras 27 and 28 are adjusted so that a desired image can be obtained, and the data at that time is transmitted to the surface mounter 12. It is to be noted that these data (camera information including the type of camera and the adjustment information of the camera) are stored in the data storage unit 32 and can be read out as necessary.

[0078] The processes of S11 and S21 disclose the following as described in claim 1 of the present application: "When the surface mount machine stops operation due to the occurrence of a recognition error, the surface mount machine transmits error information, recognition parameters used for image recognition, and camera information to the mobile terminal via the production management device."

[0079] When the mobile terminal 50 receives the error information and the recognition parameters from the production management device 40 (S31), it displays the error information on the display unit 54 and stores the recognition parameters in the storage unit 53. Thereafter, it goes into a standby state to wait for an input operation by an operator or a service person.

[0080] If an operator or a serviceman believes that restoration can be expected by editing the recognition parameters, he or she executes an editing application software (hereinafter, an editing app) registered in the mobile terminal 50.

[0081] Execution of the editing application causes a recognition parameter editing screen to be displayed on the display unit of the mobile terminal 50. In this embodiment, the display screen shown in FIG.

[0082] An operator or service person can edit the recognition parameters that were used for image recognition when the error occurred, from the editing screen of the portable terminal 50 (S32: YES, S33).

[0083] After editing the recognition parameters, the mobile terminal 50 transmits the edited recognition parameters to the surface mounter 12 via the production management device 40 (S34, S24).

[0084] The processes of S33, S34, and S24 disclose the following as described in claim 1 of the present application: "The mobile terminal displays an editing screen for the recognition parameters, and, if the recognition parameters are edited, transmits the edited recognition parameters to the surface mounter via the production management device."

[0085] When the surface mounter 12 receives the edited recognition parameters, it applies the received edited recognition parameters and performs image recognition on the recognition target image again (S14).

[0086] The image recognition results based on the edited recognition parameters are transmitted from the surface mounter 12 to the mobile terminal 50 via the production management device 40 (S25).

[0087] The processes of S14 and S25 disclose the following as described in claim 1 of the present application: "The surface mounter re-recognizes the target image by applying the edited recognition parameters, and transmits the recognition result based on the edited recognition parameters to the mobile terminal via the production management device."

[0088] When the mobile terminal 50 receives the recognition result (S35), it displays the recognition result on the display unit 54. If the image recognition is successful using the edited recognition parameters, the operator or service person determines that editing of the recognition parameters is successful (S36: YES).

[0089] Furthermore, the recognition parameter editing does not necessarily have to be successful on the first try. Even if the first try is not successful, the recognition parameter editing may be successful as a result of repeated editing.

[0090] Then, if editing of the recognition parameters is successful after one or more edits (S36: YES), the service technician or operator operates the mobile terminal 50 to send an instruction to resume operation to the surface mount machine 12 via the production management device 40 (S26).

[0091] When the surface mounter 12 receives an instruction to resume operation from the mobile terminal 50, it resumes operation, and after resuming operation, it performs image recognition using the edited recognition parameters (S15). This ends the remote recovery process.

[0092] The processing of S36 and S26 discloses claim 1 of the present application, which states that "if image recognition is successful using the edited recognition parameters, the mobile terminal transmits an instruction to resume operation to the surface mount machine via the production management device."

[0093] If operation recovery is not expected even after editing the recognition parameters, the determinations in S32 and S37 are both NO. In this case, the mobile terminal 50 transmits an operation stop command to the surface mounter 12 to stop the operation.

[0094] Moreover, if operation recovery is expected without editing the recognition parameters, the determination is NO in S32 and YES in S37. In this case, the mobile terminal 50 transmits an operation restart command to the surface mounter 12 to restart the operation.

[0095] 7. Recognition parameter editing screen 8 shows one example of a recognition parameter editing screen displayed on the mobile terminal 50. The editing screen is made up of a first display area 55 occupying the upper half of the screen, and a second display area 56 occupying the lower half of the screen.

[0096] The first display area 55 is an area for displaying an error image Ie of the electronic component E2 transmitted from the surface mounter 12. The error image Ie displayed in the first display area 55 can be moved by operating the panel of the mobile terminal 50. In addition, it is also possible to enlarge or reduce the error image Ie.

[0097] The editing application has a shape drawing function, and automatically draws a shape model of the electronic component E2 based on the recognition parameters transferred from the surface mounter 12. The shape can be drawn in actual size (actual scale) based on the camera angle and enlargement ratio included in the recognition parameters.

[0098] A first button 55A and a second button 55B for operation are displayed in the first display area 55. The first button 55A is for switching the display, and by operating the button, as shown in Fig. 8, it is possible to display the shape model Im superimposed on the error image Ie in the first display area 55, or to hide the shape model Im and display only the error image Ie.

[0099] Second button 55B is a button for measurement. By operating second button 55B, a measurement program is started in the editing application, and by specifying any two points in first display area 55, the distance between any two points can be measured.

[0100] Using this dimension measurement function, it is possible to measure each dimension of the external shape and lead R of the electronic component E2 that is the recognition target from the error image Ie. Based on the camera angle and magnification factor included in the recognition parameters, it is possible to measure each dimension of the external shape and lead R of the electronic component E2 in actual size (actual scale).

[0101] The second display area 56 is an area for displaying the recognition parameters and recognition results transferred from the surface mounter 12, and in the second display area 56, a third button 56A, a fourth button 56B, and a data display frame 56C are displayed.

[0102] By operating the third button 56A, the recognition parameters are displayed in the data display frame 56C, and by operating the fourth button 56B, the recognition results are displayed in the data display frame 56C. In this embodiment, the data on the number of leads R, the lead length L, the lead width d, and the lead pitch P are displayed as the recognition results.

[0103] When the recognition parameters are displayed in the data display frame 56C, the displayed recognition parameters can be edited by operating the panel of the mobile terminal 50.

[0104] Here, the editing screen can display the following information, which can be used to help identify the cause of a recognition error and consider ways to resolve it, thereby assisting in the editing of recognition parameters.

[0105] (1) Error image (2) Shape model (3) Recognition results (4) Recognition parameters

[0106] <Example of editing recognition parameters> For example, if an error image Ie has a chip at the tip of the lead R, causing a recognition error due to a discrepancy in the lead length L, it is possible to modify the detection range of the image so that the entire electronic component E2, including the lead R, is included in the detection range.

[0107] After completing the editing of the recognition parameters, the operator or serviceman can operate the recognition execution button 56D displayed at the bottom of the second display area 56 to instruct the surface mounter 12 from the mobile terminal 50 to send the edited recognition parameters via the production management device 40 and to perform re-recognition using the edited recognition parameters.

[0108] 8.Effects According to the present invention, the recovery work (recognition parameter editing work) of the surface mounter 12 that has been stopped due to the occurrence of a recognition error can be performed remotely using the mobile terminal 50. This allows the surface mounter 12 to be stopped for a shorter period of time and to resume operation sooner, compared to a case in which the recovery work (recognition parameter editing work) of the surface mounter 12 can only be performed on-site.

[0109] <Embodiment 2> 9 is a block diagram of an information processing system 160 according to embodiment 2. The information processing system 160 differs in that, when image recognition using post-editing recognition parameters results in a deterioration in production performance compared to before editing, the mobile terminal 50 outputs warning information to notify the operator of the deterioration in production performance.

[0110] More specifically, as shown in FIG. 9, the storage unit 43 of the production management device 40 stores production history information and editing history information.

[0111] <Production history information> The production history information is a history of information relating to the production results of the board P, and includes the equipment name, board data, part number, mounting number, and production results as shown in Fig. 10. The production results include the recognition success rate and mounting accuracy.

[0112] The recognition success rate is the ratio of the number of successful recognitions to the number of image recognition trials. The mounting accuracy is a value expressed as a standard deviation (3σ) of the amount of deviation between the mounting position of the electronic component E on the board P and the designed mounting position. The production history information is transferred from the surface mounter 12 or the inspection machine 13 to the production management device 40, regardless of whether the recognition parameters have been edited, and is cumulatively stored in the production management device 40.

[0113] <Edit history information> The editing history information is information that records the editing history of the recognition parameters by the mobile terminal 50, and includes, for example, as shown in FIG. 10(B), the date and time of editing, the device name, the type of board, the part numbers of the electronic components E1 and E2 to be installed, the user name of the operator or serviceman who performed the editing work, as well as changes to the recognition parameters.

[0114] The changes in the recognition parameters include the type of the recognition parameter that was edited and the parameter values ​​before and after editing.

[0115] The editing history information is cumulatively stored in the production management device 40 each time the recognition parameters are edited on the mobile terminal 50 and the successfully recognized recognition parameters are transferred from the mobile terminal 50 to the surface mounter 12 via the production management device 40.

[0116] After the recognition parameters are edited, the production management device 40 applies the edited recognition parameters and, after a predetermined period of time has elapsed, compares the production results before and after editing, and if the production results after editing are worse than before editing, wirelessly transmits warning information to the mobile terminal 50.

[0117] For example, by comparing the recognition success rate before and after the recognition, if the recognition success rate after the recognition is worse than the recognition success rate before the recognition, warning information is wirelessly transmitted to the mobile terminal 50.

[0118] When the mobile terminal 50 receives the warning information, as shown in FIG. 11, it displays the warning information on the display unit 54, and notifies the operator that the production performance has deteriorated after editing the recognition parameters.

[0119] The mobile terminal 50 may notify the operator by sound such as an alarm sound in addition to displaying the warning on the display unit 54 or instead of displaying the warning information.

[0120] After the warning information is transmitted, if the operator determines to return the recognition parameters to the pre-editing state, the operator transmits, via the mobile terminal 50, an instruction to the production management device 40 to change the recognition parameters to the pre-editing state.

[0121] When the production management device 40 receives an instruction to change the recognition parameters from the mobile terminal 50 , it reads out the recognition parameters before editing from the editing history information in the storage unit 43 and transfers them to the surface mounter 12 .

[0122] Thereafter, during production of the board P, the surface mounter 12 applies the recognition parameters before editing when performing image recognition.

[0123] In the configuration of the second embodiment, when the production performance deteriorates due to editing of the recognition parameters, the editing parameters are returned to the state before editing, thereby making it possible to suppress the deterioration of the production performance.

[0124] <Embodiment 3> 12 is a block diagram of an information processing system 260 according to embodiment 3. The information processing system 260 differs from the information processing system 260 according to embodiment 2 in that the information processing system 260 accumulates production history information including production results and editing history information of recognition parameters in a cloud server 70, and performs machine learning of the relationship between the production results and the recognition parameters.

[0125] The cloud server 70 is communicably connected to the production management device 40 via a wired or wireless network. The production management device 40 transmits production history information of the surface mounter 12 and editing history information of the recognition parameters to the cloud server 70.

[0126] The cloud server 70 generates a learning model M that determines the degree of influence of editing of the recognition parameters on the production performance, using the production history information of the board P and the editing history information of the recognition parameters as teaching data (FIG. 13). The degree of influence is, for example, "the production performance improves by 10%" or "decreases by 10%". The learning model M can be, for example, a learning model by deep learning using a neural network.

[0127] The results of the above machine learning can be utilized in the following ways: When an operator edits the recognition parameters, edited data of the recognition parameters is sent from the mobile terminal 50 to the cloud server 70 via the production management device 40, and the degree of influence on the production performance is evaluated using the learning model M. Then, the evaluation result by the learning model M is transferred to the mobile terminal 50 via the production management device 40 and presented to the operator or serviceman (displayed on the display screen), which can be useful for editing the recognition parameters.

[0128] <Other embodiments> (1) In the above embodiment, an example in which the recognition parameters are edited during an operation stop due to a recognition error has been described. The recognition parameters may be edited regardless of the presence or absence of a recognition error. For example, the recognition parameters may be edited during a period in which the surface mounter 12 is not operating and the production of the substrate P is not being performed, such as before the start of production of the substrate P or after the end of production. For example, when the recognition parameters are edited before the start of production, a configuration is conceivable in which the recognition parameters to be used in production are transmitted from the surface mounter 12 to the mobile terminal 50 by sending a command from the mobile terminal 50 to the surface mounter 12 via the production management device 40, and the transmitted recognition parameters are edited. Similarly, when the recognition parameters are edited after the start of production, a configuration is conceivable in which the recognition parameters used in production are transmitted from the surface mounter 12 to the mobile terminal 50 by sending a command from the mobile terminal 50 to the surface mounter 12 via the production management device 40, and the transmitted recognition parameters are edited. In this case, the camera information may be transmitted to the mobile terminal 50 together with the recognition parameters, or the transmission may be omitted. In addition, the destination of the recognition parameters is not limited to the surface mounter 12, and a form in which the recognition parameters are read from the production management device 40 may also be conceivable.

[0129] (2) In the above embodiment, a smartphone operable with a touch panel is exemplified as the mobile terminal 50. The mobile terminal 50 is not limited to a smartphone, and may be, for example, an information communication terminal that an operator can carry around, such as a notebook computer.

[0130] (3) In the above embodiment, an example was shown in which the detection range of the image was edited among the recognition parameters, but other recognition parameters may be edited. For example, the brightness threshold value used for extracting the contour may be edited, or the lead length or lead width may be edited.

[0131] (4) In the above embodiment, the object of image recognition is electronic component E, but the object of image recognition is not limited to electronic component E and may be a board mark or base mark photographed by board recognition camera 28. The board mark is fiducial mark 81 (see FIG. 2) for position confirmation attached to the top surface of the board or a butt mark for identifying the quality of the board. The base mark is correction mark 82 (see FIG. 2) for position correction placed on the base.

[0132] (5) In the above embodiment, the object of image recognition is an electronic component with leads, but the object of image recognition is not limited to an electronic component with leads, and may be a chip component such as a chip capacitor or a ball component. In the case of a chip component, it is possible to perform image recognition of the component based on the shape of the electrodes, and in the case of a ball component, it is possible to perform image recognition of the component based on the shape of the ball.

[0133] (6) In the above embodiment, when image recognition is successful with the edited recognition parameters, the mobile terminal 50 issues an instruction to resume operation to the surface mounter 12 via the production management device 40. When image recognition is successful, a board image of the intended mounting position of the electronic component E that is the subject of the recognition error may be displayed on the mobile terminal 50, and after confirming that the electronic component E is not mounted at the intended mounting position of the board P, an instruction to resume operation may be sent to the surface mounter 12. The board image of the intended mounting position may be sent from the surface mounter 12 to the mobile terminal 50 via the production management device 40 together with the error image Ie, or a transmission instruction may be sent from the mobile terminal 50 at another timing (for example, when recognition is successful with the edited recognition parameters) and transferred from the surface mounter 12 via the production management device 40.

[0134] (7) In the first embodiment, when a recognition error occurs, three pieces of information, namely, error information, recognition parameters used for image recognition, and camera information, are transmitted from the surface mounter 12 to the mobile terminal 50 (via the production management device 40), but when a recognition error occurs, only the error information and the recognition parameters may be transmitted, and transmission of the camera information may be omitted. In addition, when transmitting camera information, both the camera type and camera adjustment information may be transmitted, or only one of the camera type and camera adjustment information may be transmitted. In other words, only the camera information may be transmitted, or only the camera adjustment information may be transmitted. [Explanation of symbols]

[0135] 10: Production System 12: Surface mount machine 26: Head Unit 26A: Mounting head 27: Parts Recognition Camera 28: Circuit Board Recognition Camera 40: Production control device 50: Mobile devices 60: Information Processing Systems 70: Cloud Server Lv: Production Line P: Board E1, E2: Electronic components

Claims

1. An information processing system, A surface mounter, a production management device capable of communicating with the surface mounter; a mobile terminal capable of wirelessly communicating with the production management device; when the surface mounter stops operation due to the occurrence of a recognition error, the surface mounter transmits error information, recognition parameters used in the image recognition of the object, and camera information to the mobile terminal via the production management device, the portable terminal displays an editing screen for the recognition parameters, and when the recognition parameters are edited, transmits the edited recognition parameters to the surface mounter via the production management device; the surface mounter re-recognizes the image of the object by applying the edited recognition parameters, and transmits the recognition result based on the edited recognition parameters to the portable terminal via the production management device; When image recognition is successful using the edited recognition parameters, the mobile terminal transmits an instruction to resume operation to the surface mount machine via the production management device.

2. 2. The information processing system according to claim 1, The production management device is an information processing system having a storage unit, and stores an editing history of the recognition parameters in the storage unit as editing history information.

3. 3. The information processing system according to claim 2, The production management device transmits warning information to the mobile terminal when the production performance after editing the recognition parameters becomes worse than the production performance before editing the recognition parameters.

4. 3. The information processing system according to claim 1, An information processing system, wherein the mobile terminal is capable of editing the recognition parameters not only during an operation stop due to a recognition error, but also before production of a board starts or after production ends.

5. 3. The information processing system according to claim 1, The object includes at least one of an electronic component to be mounted on a board, a board mark, and a base mark.

6. 3. The information processing system according to claim 2, A cloud server communicably connected to the production management device, The production management device transmits production history information including a production record of the board and editing history information of the recognition parameters to the cloud server, The cloud server generates a learning model that determines the degree of impact of editing the recognition parameters on production performance using the production history information and the editing history information of the recognition parameters as training data.

7. 3. The information processing system according to claim 1, When the surface mounter stops operating due to a recognition error of an electronic component to be mounted on a board, an information processing system that transmits, before restarting operation, an image of a planned mounting position of the electronic component that is the subject of the recognition error on the board to the mobile terminal via the production management device.

8. 1. An information processing method, comprising: When the surface mount machine stops operation due to the occurrence of a recognition error, the surface mount machine transmits error information, recognition parameters used for image recognition, and camera information to the mobile terminal via the production management device; the portable terminal displays an editing screen for the recognition parameters, and when the recognition parameters are edited, transmits the edited recognition parameters to the surface mounter via the production management device; the surface mounter re-recognizes the target image by applying the edited recognition parameters, and transmits a recognition result based on the edited recognition parameters to the portable terminal via the production management device; The information processing method includes, when image recognition is successful using the edited recognition parameters, transmitting an instruction to the surface mounting machine to resume operation via the production management device.

Images