Image Processing Device

Abstract

This image processing device comprises: a process execution unit that executes an inspection process for inspecting the state of a camera that is utilized in a mounting step for mounting a component on a substrate, the inspection process including a process for causing the camera to image a predetermined reference plate different from the component and the substrate; and a notification unit that executes a predetermined notification operation if an out-of-reference region having an area greater than or equal to a predetermined area is present in an image captured by the camera in the inspection process, the out-of-reference region being a blob region in which a plurality of pixels are present, the plurality of pixels being connected and having pixel values of which a difference from a pixel value indicating the reference plate in the image is greater than or equal to a predetermined value.

Description

[Technical Field]

[0001] The present specification relates to a technology for processing images from a camera used in a mounting process for mounting components on a board. [Background technology]

[0002] Patent Document 1 discloses a control device that processes images acquired by a camera used in a component mounter. The control device identifies areas of the entire image that cannot be properly imaged as unimageable areas, and halts the holding of components using nozzles that are expected to appear in the identified unimageable areas among multiple nozzles. Even if a malfunction occurs in the camera, the mounting process of mounting components on a board continues using other nozzles. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-098280 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned patent documents, the unimageable area is a rectangular area that circumscribes the fallen part area, which occurs when a part falls onto the camera, in both the vertical and horizontal directions. Therefore, the area of ​​the unimageable area is larger than the area of ​​the fallen part area, and does not accurately evaluate the camera malfunction. Furthermore, the above-mentioned patent documents are technologies for continuing the mounting process even if a malfunction occurs in the camera, but are not technologies for evaluating the camera malfunction and notifying the user of the camera malfunction. This specification provides a technology for evaluating the camera malfunction and notifying the user of the malfunction. [Means for solving the problem]

[0005] The image processing device disclosed in this specification comprises: a processing execution unit that executes an inspection process to inspect the state of a camera used in an implementation process to implement components on a board, the inspection process including a process of having the camera capture an image of a predetermined reference board that is different from the component and the board; and an alarm unit that executes a predetermined alarm operation when an out-of-reference area, which is an area of ​​a predetermined area or larger, is present in an image captured by the camera during the inspection process, the out-of-reference area being a blob area in which multiple connected pixels exist, each having a pixel value whose difference from the pixel value indicating the reference board in the image is a predetermined value or larger.

[0006] Camera defects include, for example, dust on the camera lens or a malfunction of a camera component. Such defects may be captured in a color different from the color of the object being captured. Alternatively, the brightness of the pixel value of the defective portion and the object being captured in the captured image may differ. Furthermore, if the area of ​​the region corresponding to the defect in the image is relatively large, the image of the object may not be captured properly, which may result in a failure of the mounting process. The above-mentioned non-standard region is a region that is a different color from the reference board and has an area equal to or greater than a predetermined area. If a non-standard region exists in the image, it can be determined that a defect has occurred in the camera. According to the above configuration, a notification operation is executed when it is determined that a defect has occurred in the camera. The user can be notified of the camera defect. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a side view schematically illustrating the configuration of the mounting device. [Figure 2] FIG. 1 shows a block diagram of a mounting device. [Figure 3] 1 shows a flowchart of a first camera inspection process according to a first embodiment. [Figure 4] 10 shows a flowchart of a second camera inspection process according to the first embodiment. [Figure 5] 10 shows a flowchart of a first camera inspection process according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] The main features of the embodiments described below are listed below. Note that the technical elements described below are independent technical elements that exhibit technical utility alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing.

[0009] (Feature 1) When the non-standard area does not exist in the image, the notification action may not be executed.

[0010] According to this configuration, it is possible to prevent the notification action from being performed unnecessarily.

[0011] (Feature 2) The reference plate may be a white plate.

[0012] (Feature 3) The predetermined area may be equal to or smaller than the area of ​​the smallest of the components.

[0013] Images of dust on a camera lens or images showing malfunctioning camera components are usually smaller than images showing the components. With the above configuration, it is possible to appropriately determine that a blob area that is equal to or smaller than the area of ​​the smallest component is a non-standard area.

[0014] (Feature 4) The camera may be capable of capturing an image of a nozzle that picks up the component and transports it to the board, and the reference plate may be picked up by the nozzle during the inspection process.

[0015] With this configuration, it is possible to notify the user of any malfunction in the camera capable of capturing images of the nozzles.

[0016] (Feature 5) When the blob area of ​​the image is present in a central region that indicates the center of the camera lens and the blob area is equal to or larger than a first area, which is the predetermined area, it may be determined that the non-standard area exists in the image. When the blob area of ​​the image is present in a peripheral region that indicates the periphery of the camera lens and the blob area is equal to or larger than a second area, which is the predetermined area and is smaller than the first area, it may be determined that the non-standard area exists in the image.

[0017] With this configuration, even if the image showing the dust or malfunction on the periphery of the lens is small compared to the image showing the dust or malfunction in the center of the lens, the warning operation can be executed to prompt the user to deal with the dust or malfunction on the periphery of the lens.

[0018] (Feature 6) The camera may be capable of capturing an image of the substrate supported on a predetermined support stand, and the reference plate may be supported on the support stand during the inspection process.

[0019] With this configuration, it is possible to notify the user of a malfunction in the camera capable of capturing an image of the substrate supported on the support stand. [Example]

[0020] (First Example) (Configuration of mounting device 10; Figures 1 and 2) The mounting apparatus 10 is an apparatus capable of executing a mounting process in which components are mounted on a substrate. The components are electronic components such as resistors. The mounting apparatus 10 includes a moving device 12, a nozzle 14, a display 16, a bottom part camera 20, a side part camera 22, a mark camera 24, a control unit 30, a support stand 50, and a component feeder 60. The nozzle 14, the side part camera 22, and the mark camera 24 are attached to the moving device 12.

[0021] Each of the units 12 to 24, 60 is controlled by a control unit 30. The control unit 30 includes a CPU 32 and a memory 34. The CPU 32 is capable of executing various processes in accordance with a program 40 stored in the memory 34. The CPU 32 is capable of analyzing images captured by the cameras 20, 22, 24, for example.

[0022] The support table 50 is a table that supports a board that is transported by a transport device (for example, a conveyor, not shown). The component feeder 60 is a device that supplies components into the mounting apparatus 10.

[0023] The nozzle 14 is a device that picks up components supplied by the component feeder 60. The display 16 is a device that can display various information.

[0024] The moving device 12 is a device that can move in horizontal directions (left and right directions on the paper and directions perpendicular to the paper). The moving device 12 moves, for example, to above the component feeder 60. The nozzle 14 of the moving device 12 picks up the component supplied by the component feeder 60. After the nozzle 14 picks up the component, the moving device 12 moves from above the component feeder 60 to above the support base 50. The nozzle 14 mounts the component picked up by the nozzle 14 onto the board supported by the support base 50. This completes the mounting process.

[0025] The bottom surface part camera 20 is disposed between the component feeder 60 and the support table 50. The bottom surface part camera 20 can capture images of the nozzle 14 from below as it moves between the component feeder 60 and the support table 50. By analyzing the images captured by the bottom surface part camera 20, the position within the nozzle 14 of the component picked up by the nozzle 14 can be obtained. Based on this position, it is inspected whether the component is properly picked up by the nozzle 14.

[0026] The side part camera 22 is disposed adjacent to the nozzle 14. The side part camera 22 can capture an image of the component picked up by the nozzle 14 from the side. By analyzing the image captured by the side part camera 22, the posture of the component picked up by the nozzle 14 can be obtained. Based on this posture, it is inspected whether the component is picked up correctly by the nozzle 14.

[0027] The mark camera 24 is disposed adjacent to the nozzle 14. The mark camera 24 is capable of capturing an image of the board supported on the support stand 50 from above. By analyzing the image captured by the mark camera 24, the appearance of the board on which components are mounted is inspected. Furthermore, by analyzing the image captured by the mark camera 24, it is possible to obtain the identification number of the board from a mark (e.g., a barcode) printed on the board.

[0028] The images captured by the cameras 20 to 24 are monochrome. In a modified example, the images captured by the cameras 20 to 24 may be color.

[0029] Dust and other debris may adhere to the lens of a camera such as the lower surface parts camera 20. Debris adhering to the lens can usually be removed by a blower (not shown) of the mounting apparatus 10. However, there may be a situation where the blower is unable to completely remove all of the debris adhering to the lens. Debris that cannot be completely removed by the blower is removed manually by the user of the mounting apparatus 10, for example.

[0030] In this embodiment, the CPU 32 executes a first camera inspection process (see FIG. 3) and a second camera inspection process (see FIG. 4) in accordance with the program 40 to prompt the user to remove dust adhering to the lens.

[0031] (First camera inspection process; Figure 3) The first camera inspection process is a process for inspecting whether or not dust is attached to the lower surface part camera 20. For example, the first camera inspection process is executed before the start of the mounting process. For example, the first camera inspection process is executed when triggered by an instruction from a user. In a modified example, the first camera inspection process may be executed during the mounting process. Furthermore, the first camera inspection process may be executed automatically without an instruction from a user.

[0032] In S10, the CPU 32 causes the nozzle 14 to pick up the reference plate 100 (see FIG. 1). The reference plate 100 is a white plate. The reference plate 100 is prepared in advance at a predetermined position (for example, next to the component feeder 60) within the mounting apparatus 10. The CPU 32 moves the moving device 12 to above the predetermined position, and causes the nozzle 14 to pick up the reference plate 100 placed at the predetermined position.

[0033] In S12, the CPU 32 moves the moving device 12 to above the lower surface parts camera 20 and causes the lower surface parts camera 20 to capture an image of the reference plate 100, which is being sucked onto the nozzle 14. As a result, the CPU 32 acquires an image (hereinafter referred to as a "first processed image") showing the reference plate 100 from the lower surface parts camera 20. For example, if dust is attached to the lens of the lower surface parts camera 20, a blob area showing a color (e.g., gray) different from the white color of the reference plate 100 will be included in the first processed image, as shown in FIG. 3. That is, if dust is attached to the lens of the lower surface parts camera 20, the first processed image will contain pixels having a luminance as a pixel value indicating the white color of the reference plate 100 and pixels having a luminance as a pixel value different from the pixel value indicating the reference plate 100. The blob area made up of pixels having pixel values ​​different from the pixel value indicating the reference plate 100 indicates dust attached to the lens of the lower surface parts camera 20. On the other hand, if dust is not attached to the lens of the lower surface parts camera 20, the blob area will not be included in the first processed image. The blob region is a region where a plurality of pixels having pixel values ​​indicating a color (or brightness) different from the pixel values ​​(for example, "225 to 255") indicating the white color of the reference plate 100 are connected to each other.

[0034] In S14, the CPU 32 determines whether or not the pixel value of each of the plurality of pixels constituting the first processed image is smaller than a predetermined threshold value (e.g., "220"). A pixel value being smaller than the predetermined threshold value means that the color of the pixel indicated by the pixel value is a color different from the white color of the reference plate 100. After completing the above determination for each of the plurality of pixels constituting the first processed image, the CPU 32 proceeds to S16.

[0035] In S16, the CPU 32 uses the result of the determination in S14 (see the graph at the bottom of the page) to determine whether or not the first processed image contains one or more blob regions. Here, whether or not two pixels having pixel values ​​indicating different colors are connected is determined, for example, by using a number identifying each pixel (for example, the number written on the horizontal axis of the graph at the bottom of the page). If the CPU 32 determines that the first processed image contains one or more blob regions (YES in S16), the CPU 32 proceeds to processing from S18 onwards. On the other hand, if the CPU 32 determines that the first processed image does not contain a blob region (NO in S16), the CPU 32 skips processing from S18 onwards and ends the processing of FIG. 3.

[0036] In S18, the CPU 32 determines whether the area of ​​at least one of the one or more blob areas is greater than a predetermined value. The predetermined value is set to an area equal to or smaller than the smallest component among the various types of components supplied by the component feeder 60. Images of dust adhering to a camera lens are typically smaller than images representing the components. This configuration allows a blob area equal to or smaller than the smallest component to be appropriately determined as an area representing dust adhering to the lens. In a modified example, the predetermined value may be set regardless of the size of the various types of components supplied by the component feeder 60.

[0037] If the CPU 32 determines that the area of ​​at least one blob region is greater than the predetermined value (YES in S18), the process proceeds to S20. In S20, the CPU 32 displays on the display 16 an instruction to remove dust adhering to the lens of the lower surface parts camera 20. This allows the CPU 32 to evaluate whether dust is adhering to the lens of the lower surface parts camera 20 and notify the user of the dust.

[0038] On the other hand, if the CPU 32 determines that the area of ​​all of the one or more blob areas is equal to or less than the predetermined value (NO in S18), the CPU 32 proceeds to S22. A blob area having an area smaller than the predetermined value is likely to be noise from the lower surface parts camera 20, rather than dust adhering to the lens. In S22, the CPU 32 corrects a plurality of pixel values ​​indicating a plurality of pixels constituting the blob area in order to remove noise from the lower surface parts camera 20. For example, the CPU 32 corrects the plurality of pixel values ​​by using the average value of the plurality of pixel values. When S20 or S22 ends, the CPU 32 ends the processing of FIG. 3.

[0039] If it is determined that the area of ​​one or more blog areas is smaller than the predetermined value (NO in S18), the process of S20 is not executed. This makes it possible to prevent the process of S20 (i.e., notification to the user) from being executed unnecessarily.

[0040] (Second camera inspection process; Figure 4) The second camera inspection process is a process for inspecting whether or not dust is attached to the mark camera 24. For example, the trigger for the second camera inspection process is the same as the trigger for the first camera inspection process. The second camera inspection process is the same as the first camera inspection process except that the processes of S100 and S102 are executed instead of S10 and S12 in FIG. 3.

[0041] In S100, the CPU 32 causes the conveying device to convey the reference plate 200, which is the same white color as the reference plate 100. As a result, the reference plate 200 is supported on the support base 50. The reference plate 200 is inserted into the conveying device by, for example, a user.

[0042] In S102, the CPU 32 moves the moving device 12 to above the support base 50 and causes the mark camera 24 to capture an image of the reference plate 200 supported by the support base 50. As a result, the CPU 32 acquires an image showing the reference plate 200 from the mark camera 24 (hereinafter referred to as a "second processed image").

[0043] When S102 ends, the CPU 32 executes the processes of S14 to S22. In S14 to S22 of the second camera inspection process, the second processed image is used. For example, when the CPU 32 determines that the area of ​​at least one blob region included in the second processed image is greater than a predetermined value (YES in S18), the CPU 32 causes the display 16 to display an instruction to remove dust adhering to the lens of the mark camera 24 (S20). This enables the CPU 32 to evaluate whether dust is adhering to the lens of the mark camera 24 and notify the user of the dust adhering.

[0044] (Correspondence) The control unit 30 of the mounting apparatus 10 is an example of an "image processing device." In the first camera inspection process of FIG. 3, the lower surface part camera 20, the reference plate 100, and the predetermined value of S18 of FIG. 3 are examples of a "camera," a "predetermined reference plate," and a "predetermined area," respectively. In the first camera inspection process, the CPU 32 that executes the processes of S10 to S18 of FIG. 3 and the CPU 32 that executes the process of S20 of FIG. 3 are examples of a "processing execution unit" and a "notification unit," respectively. In the second camera inspection process of FIG. 4, the mark camera 24, the reference plate 200, and the predetermined value of S18 of FIG. 4 are examples of a "camera," a "predetermined reference plate," and a "predetermined area," respectively. In the second camera inspection process, the CPU 32 that executes the processes of S100, S102, and S14 to S18 of FIG. 4 and the CPU 32 that executes the process of S20 of FIG. 4 are examples of a "processing execution unit" and a "notification unit," respectively.

[0045] (Second Example) This embodiment is similar to the first embodiment except that the content of the first camera inspection process for inspecting whether or not dust is attached to the lower surface parts camera 20 is different.

[0046] (First camera inspection process; Figure 5) S10 to S16 are the same as in Example 1. If the CPU 32 determines that the first processed image does not contain a blob area (NO in S16), it skips the processing from S200 onwards, which will be described later, and ends the processing in FIG.

[0047] When the CPU 32 determines that one or more blob areas are included in the first processed image (YES in S16), the CPU 32 proceeds to S200. In S200, the CPU 32 selects one blob area (hereinafter referred to as the "target blob area") from the one or more blob areas.

[0048] In the next step S202, the CPU 32 determines whether the target blob region is included in a central region of the first processed image, which includes the center of the lens of the lower part camera 20. Here, the position of the target blob region is determined, for example, using numbers identifying the pixels constituting the target blob region (for example, numbers written on the horizontal axis of the graph at the bottom of the page in FIG. 3). If the CPU 32 determines that the target blob region is included in the central region (YES in S202), the CPU 32 proceeds to step S204. Whether each pixel in the first processed image belongs to the central region or the peripheral region is determined in advance and stored in the memory 34. Note that if multiple pixels constituting the target blob region straddle both the central region and the peripheral region, the region that includes the largest area of ​​the blob region or the largest number of pixels of the blob region may be determined to be the region that includes the blob region.

[0049] In S204, the CPU 32 determines whether the area of ​​the target blob region is greater than a first predetermined value. If the CPU 32 determines that the area of ​​the target blob region is greater than the first predetermined value (YES in S204), the process proceeds to S210. In S210, the CPU 32 determines whether to display an instruction to remove dust adhering to the lens of the lower surface parts camera 20. Note that the display of this instruction is not executed until the determinations of S202 and S204 have been executed for all of the one or more blob regions.

[0050] Furthermore, if the CPU 32 determines that the area of ​​the target blob region is equal to or smaller than the first predetermined value (NO in S204), the CPU 32 proceeds to S212. S212 is the same as S22 in Fig. 3. When S210 or S212 ends, the CPU 32 proceeds to S230.

[0051] Furthermore, if the CPU 32 determines that the target blob region is not included in the central region but is included in the peripheral region indicating the periphery of the lens of the lower surface parts camera 20 (NO in S202), the CPU 32 proceeds to S214. In S214, the CPU 32 determines whether the area of ​​the target blob region is larger than a second predetermined value. Here, the second predetermined value is a value smaller than the first predetermined value in S204. If the CPU 32 determines that the area of ​​the target blob region is larger than the second predetermined value (YES in S214), the CPU 32 proceeds to S220. S220 is the same as S210.

[0052] Furthermore, if the CPU 32 determines that the area of ​​the target blob region is equal to or smaller than the second predetermined value (NO in S214), the CPU 32 proceeds to S222. S222 is the same as S212. When S220 or S222 ends, the CPU 32 proceeds to S230.

[0053] In S230, CPU 32 determines whether or not an unselected blob region exists among one or more blob regions. If CPU 32 determines that an unselected blob region exists among one or more blob regions (YES in S230), CPU 32 returns to S200 and selects another blob region. On the other hand, if CPU 32 determines that an unselected blob region does not exist among one or more blob regions (NO in S230), CPU 32 proceeds to S232.

[0054] In S232, the CPU 32 determines whether or not a decision was made in S210 or S220 to display an instruction to remove dust adhering to the lens. If the CPU 32 determines that a decision was made to display the instruction (YES in S232), the process proceeds to S240. On the other hand, if the CPU 32 determines that a decision was not made to display the instruction (NO in S232), the process skips S240 and ends the process of FIG. 5.

[0055] S240 is the same as S20 in Fig. 3. When S240 ends, the CPU 32 ends the processing in Fig. 5.

[0056] In this embodiment, the second predetermined value in S214 is smaller than the first predetermined value in S204. Then, even if the blob area included in the peripheral region is smaller than the blob area included in the central region, the user is instructed to remove the dust (S220). That is, even if the dust attached to the peripheral region of the lens is smaller than the dust attached to the center of the lens, the user can be prompted to remove the dust attached to the peripheral region of the lens. For example, dust attached to the center of the lens is easily removed by the blower of the mounting apparatus 10, but dust attached to the peripheral region of the lens may not be completely removed by the blower. The user can be prompted to remove the dust that cannot be completely removed by the blower.

[0057] (Correspondence) The CPU 32 that executes the processes of S10 to S16 and S200 to S230 in Fig. 5 and the CPU 32 that executes the process of S240 in Fig. 5 are examples of a "processing execution unit" and an "alert unit", respectively. The first predetermined value in S204 and the second predetermined value in S214 in Fig. 5 are examples of a "first area" and a "second area", respectively.

[0058] The following points should be noted regarding the image processing device described in the embodiment. The "image processing device" is not limited to the control unit 30 of the mounting device 10, but may be, for example, an external device (e.g., a server) provided separately from the mounting device 10.

[0059] The "camera condition" is not limited to dust on the camera lens, but may also be, for example, a malfunction of a camera part. In this modification, the "notification action" is an action for notifying the user of a malfunction of a camera part. For example, the user can take action such as replacing the malfunctioning part.

[0060] The "notification action" is not limited to displaying the instruction in S20 of FIG. 3, but may also be, for example, outputting a warning sound or turning on a warning light.

[0061] 3 may be used to inspect the state (adhesion of dust, malfunction of parts, etc.) of the side part camera 22. In this modification, the side part camera 22 is an example of a "camera."

[0062] The color of the "reference plate" is not limited to white, and may be, for example, a color other than white (for example, gray).

[0063] In the first embodiment, either the first camera inspection process in FIG. 3 or the second camera inspection process in FIG. 4 does not necessarily have to be executed.

[0064] Furthermore, the determination of pixel values ​​in S14 of FIG. 3 is not limited to the configuration of the above embodiment. For example, the CPU 32 may calculate a difference value between each of a plurality of pixels constituting the first processed image and a pixel value representing the reference plate 100. The CPU 32 may determine whether the calculated difference value is equal to or greater than a specific value that is predetermined and stored in the memory 34. The pixel value representing the reference plate 100 is, for example, stored in advance in the memory 34. In the subsequent S16, the CPU 32 may determine, as a blob region, a region in which a plurality of connected pixels having pixel values ​​whose difference value is equal to or greater than a specific value exist. In this modification, a region in which a plurality of connected pixels having pixel values ​​whose difference value is equal to or greater than a specific value exist is an example of a "non-reference region."

[0065] The technical elements described in this specification or drawings exhibit technical utility either alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technologies illustrated in this specification or drawings simultaneously achieve multiple objectives, and achieving one of those objectives is itself technically useful. [Explanation of symbols]

[0066] 10: Mounting equipment 12: Mobile device 14: Nozzle 16: Display 20: Bottom part camera 22: Side part camera 24: Mark Camera 30: Control section 32 :CPU 34: Memory 40: Program 50: Support stand 60: Parts feeder 100, 200: Reference plate

Claims

1. a processing execution unit that executes an inspection process for inspecting a state of a camera used in a mounting process for mounting a component on a board, the camera being capable of capturing an image of a nozzle that picks up the component and transports it to the board, the inspection process including a process of picking up a predetermined reference plate different from the component and the board onto the nozzle and capturing an image of the reference plate with the camera; a notification unit that executes a predetermined notification operation when a non-standard area that is an area of ​​a predetermined area or more is present in the image captured by the camera in the inspection process, the non-standard area being a blob area in which a plurality of connected pixels exist, each having a pixel value whose difference from a pixel value indicating the reference plate in the image is a predetermined value or more; Equipped with When the blob area is present in a central area of ​​the image that indicates the center of the lens of the camera and has a first area or more, the first area being the predetermined area, it is determined that the non-reference area is present in the image; An image processing device that determines that the non-standard area exists in the image when a blob area exists in the image in a peripheral area that indicates the periphery of the camera lens, the blob area being equal to or larger than the second area, which is the specified area and smaller than the first area.

2. The image processing device according to claim 1 , wherein the notification action is not executed when the non-reference area does not exist in the image.

3. The image processing device according to claim 1 , wherein the reference plate is a white plate.

4. The image processing device according to claim 1 , wherein the predetermined area is equal to or smaller than the area of ​​the smallest of the components.

5. the camera is capable of capturing an image of the substrate supported on a predetermined support stand; The image processing apparatus according to claim 1 , wherein the reference plate is supported by the support table during the inspection process.

Images