Method for inspecting appearance and appearance inspection system

Abstract

To provide a method for inspecting an appearance which can determine the state of a plating layer more accurately, while ensuring production throughput.SOLUTION: The method for inspecting an appearance includes acquiring RGB image data; converting the RGB image data into distribution data indicating at least one of an L component and an S component of an HLS color space; calculating a Mahalanobis distance from the distribution data with respect to external terminals of an electronic component; and determining the state of a plating layer on the external terminals on the basis of the Mahalanobis distance of the external terminals of the electronic component, the RGB image data indicating the external terminals of the electronic component.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an appearance inspection method and an appearance inspection system. [Background technology]

[0002] Electronic components having plating layers on their external terminals are known (see, for example, Patent Document 1). For such electronic components, inspection of the state of the plating layers on the external terminals is anticipated. For example, it has been considered to determine the state of the plating layers on the external terminals by visual inspection. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-230514 Summary of the Invention [Problem to be solved by the invention]

[0004] When inspecting the condition of the plating layer on external terminals using X-ray film thickness measurement, it takes a long time to inspect a single electronic component, making it difficult to ensure production throughput while also ensuring the reliability of the plating layer.

[0005] It is possible to visually or mechanically inspect the condition of the plating layer by capturing the external appearance with a camera. However, it can be difficult to differentiate between different types of plating using visual inspection. For example, if a tin plating layer is placed on a nickel plating layer, it is difficult to determine the condition of the tin plating layer using an RGB image.

[0006] An object of one aspect of the present invention is to provide an appearance inspection method and an appearance inspection system that can ensure production throughput while ensuring accuracy in determining the state of a plating layer. [Means for solving the problem]

[0007] An appearance inspection method according to one aspect of the present invention includes acquiring RGB image data, converting the RGB image data into distribution data representing at least one of the L and S components of the HLS color space, calculating a Mahalanobis distance for an external terminal of an electronic component from the distribution data, and determining the state of a plating layer on the external terminal based on the Mahalanobis distance for the external terminal of the electronic component. The RGB image data represents the external terminal of the electronic component.

[0008] In this appearance inspection method, RGB image data is converted into distribution data indicating at least one of the L and S components of the HLS color space, and the condition of the plating layer is determined based on the Mahalanobis distance related to the distribution data. As a result of extensive research, the inventors have found that this appearance inspection method ensures accuracy in determining the condition of the plating layer. This appearance inspection method can ensure accuracy in determining the condition of the plating layer while ensuring production throughput.

[0009] In one aspect, the method may further include acquiring a plurality of sample RGB image data, converting the plurality of sample RGB image data into first sample distribution data and second sample distribution data, calculating a Mahalanobis distance for a first external terminal and a second external terminal from the first sample distribution data and calculating a Mahalanobis distance from the second sample distribution data, and selecting either an L component or an S component based on the Mahalanobis distance for the first sample distribution data and the Mahalanobis distance for the second sample distribution data. The plurality of sample RGB image data indicates a first external terminal having a normal plating layer state and a second external terminal having an abnormal plating layer state. The first sample distribution data indicates the L component of the HLS color space. The second sample distribution data indicates the S component of the HLS color space. In converting the distribution data, the RGB image data may be converted into distribution data indicating a selected component of the L component and the S component. In calculating the Mahalanobis distance for the distribution data, the Mahalanobis distance for the distribution data indicating the selected component may be calculated for the external terminal of the electronic component. In determining the condition of the plating layer, the condition of the plating layer on the external terminal may be determined based on the Mahalanobis distance related to the distribution data indicated by the selected component. As a result of extensive research, the inventors have found that the component between the L component and the S component for which inspection accuracy is ensured differs depending on, for example, the type of electronic component. According to the above method, the component between the L component and the S component for which inspection accuracy is high can be used for visual inspection. Therefore, the accuracy of determining the condition of the plating layer can be ensured while minimizing computational processing.

[0010] In one aspect, the state of the plating layer may be determined based on a result of comparing the Mahalanobis distance of the electronic component with a threshold value, thereby ensuring accuracy in determining the state of the plating layer while reducing the amount of calculation.

[0011] In one aspect of the present invention, the method may further include acquiring sample RGB image data, converting the sample RGB image data into sample distribution data indicating at least one of the L and S components of the HLS color space, calculating a Mahalanobis distance for each of the first and second external terminals from the sample distribution data, and determining a threshold value based on the Mahalanobis distance for each of the first and second external terminals. The sample RGB image data indicates first external terminals with normal plating layer conditions and second external terminals with abnormal plating layer conditions. In this case, the accuracy of determining the plating layer condition can be further improved.

[0012] In one aspect, the plating layer may contain tin, which can ensure the accuracy of determining the state of the plating layer while reliably ensuring the function of the plating layer.

[0013] According to another aspect of the present invention, the visual inspection system includes an acquisition unit, a conversion unit, a calculation unit, and a determination unit. The acquisition unit acquires RGB image data. The conversion unit converts the RGB image data into distribution data indicating at least one of the L component and the S component of the HLS color space. The calculation unit calculates a Mahalanobis distance from the distribution data for an external terminal of the electronic component. The determination unit determines the state of the plating layer on the external terminal based on the Mahalanobis distance for the external terminal of the electronic component. The RGB image data indicates the external terminal of the electronic component. In this visual inspection device, the RGB image data is converted into distribution data indicating at least one of the L component and the S component of the HLS color space, and the state of the plating layer is determined based on the Mahalanobis distance for the distribution data. This visual inspection system can ensure production throughput while ensuring accuracy in determining the state of the plating layer. [Effects of the Invention]

[0014] One aspect of the present invention provides an appearance inspection method and an appearance inspection system that can ensure production throughput while ensuring accuracy in determining the state of a plating layer. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram of a visual inspection system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a perspective view showing an example of an electronic component to be inspected. [Figure 3] 10(a) and 10(b) are graphs showing luminance histograms in the HLS color space. [Figure 4] 10(a) and 10(b) are graphs showing luminance histograms in the HLS color space. [Figure 5] 10 is a graph showing Mahalanobis distances between external terminals with plating layers in a normal state and external terminals with plating layers in an abnormal state. [Figure 6] 10 is a graph showing Mahalanobis distances between external terminals with plating layers in a normal state and external terminals with plating layers in an abnormal state. [Figure 7] 10 is a graph showing Mahalanobis distances between external terminals with plating layers in a normal state and external terminals with plating layers in an abnormal state. [Figure 8] FIG. 1 illustrates an example of a hardware configuration of a visual inspection system. [Figure 9] 1 is a flowchart illustrating an example of a visual inspection method. [Figure 10] 1 is a flowchart illustrating an example of a visual inspection method. [Figure 11] 10 is a graph showing the results of comparing the average saturation of an external terminal having a normal plating layer and a second external terminal having an abnormal plating layer. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and redundant description will be omitted.

[0017] First, an overview of an appearance inspection system according to an example of this embodiment will be described with reference to FIGS. 1 to 7. FIG. 1 is a block diagram of the appearance inspection system according to this embodiment. For example, electronic component C1 is a multilayer capacitor or a multilayer inductor. FIG. 2 is a perspective view showing an example of an electronic component to be inspected. The appearance inspection system 1 inspects the condition of the plating layer on the external terminals 5 of the electronic component C1. For example, the plating layer on the external terminals 5 of the electronic component C1 contains tin.

[0018] The electronic component C1 includes a rectangular parallelepiped element body 3 and a plurality of external terminals 5. The element body 3 has a pair of side surfaces 3a facing each other, a pair of side surfaces 3c facing each other, and a pair of end surfaces 3e facing each other. The pair of side surfaces 3a, the pair of side surfaces 3c, and the pair of end surfaces 3e are rectangular. The pair of side surfaces 3a face each other in the Z-axis direction. The pair of side surfaces 3c face each other in the X-axis direction. The pair of end surfaces 3e face each other in the Y-axis direction. The X-axis direction, the Y-axis direction, and the Z-axis direction are perpendicular to each other.

[0019] The electronic component C1 has a pair of external terminals 5. The pair of external terminals 5 are arranged on the outer surface of the element body 3. The pair of external terminals 5 are spaced apart from each other. The rectangular parallelepiped shape includes a rectangular parallelepiped shape with chamfered corners and ridges, and a rectangular parallelepiped shape with rounded corners and ridges.

[0020] Each external terminal 5 includes a plating layer. For example, each external terminal 5 includes a sintered electrode and multiple plating layers provided on the sintered electrode. The sintered electrode is made of, for example, copper. The multiple plating layers include, for example, nickel and tin. For example, in each external terminal 5, the sintered electrode is covered with a nickel plating layer, and the nickel plating layer is covered with a tin plating layer. In other words, the nickel plating layer is provided on the sintered electrode, and the tin plating layer is provided on the nickel plating layer. The outermost layer of each external terminal 5 includes a tin plating layer.

[0021] For example, the electronic component C1 is produced by applying and baking an electrode paste, followed by plating, which involves nickel plating followed by tin plating.

[0022] The visual inspection system 1 executes an inspection preparation phase and an inspection execution phase. The visual inspection system 1 executes a visual inspection of the condition of the plating layer based on the information stored in the inspection preparation phase. The visual inspection system 1 may further execute the inspection preparation phase based on the information acquired in the inspection execution phase.

[0023] The visual inspection system 1 includes an inspection preparation unit 10 that operates in an inspection preparation phase, and an inspection execution unit 20 that operates in an inspection execution phase. The visual inspection system 1 includes, as the inspection preparation unit 10, an acquisition unit 11, a conversion unit 12, a calculation unit 13, a storage unit 14, a selection unit 15, and a threshold determination unit 16. The visual inspection system 1 includes, as the inspection execution unit 20, the acquisition unit 11, the conversion unit 12, the calculation unit 13, the storage unit 14, a comparison unit 21, and a determination unit 22. In the example shown in this embodiment, the acquisition unit 11, the conversion unit 12, the calculation unit 13, and the storage unit 14 are included in both the inspection preparation unit 10 and the inspection execution unit 20.

[0024] As a modification of this embodiment, the inspection preparation unit 10 and the inspection execution unit 20 may be separated from each other. In this case, the inspection preparation unit 10 and the inspection execution unit 20 may each include a different acquisition unit 11, conversion unit 12, calculation unit 13, and storage unit 14. As a modification of this embodiment, the visual inspection system 1 may not include at least one of the selection unit 15 and the threshold determination unit 16, and may be composed of only the inspection execution unit 20.

[0025] Next, each functional unit of the visual inspection system 1 in the inspection preparation phase will be described in more detail. In the inspection preparation phase, the inspection preparation unit 10 operates. The inspection preparation unit 10 creates information to be used for visual inspection of the condition of the plating layer. In the example shown in this embodiment, the inspection preparation unit 10 creates component information of the HLS color space to be used for the visual inspection and threshold information to be used for the visual inspection. As a variation of this embodiment, the inspection preparation unit 10 may create a learning model trained using a learning dataset containing various information as information to be used for the visual inspection.

[0026] In the inspection preparation phase, the acquisition unit 11 acquires sample RGB image data. The sample RGB image data includes RGB image data obtained by capturing images of multiple sample electronic components. The sample electronic components have, for example, the same configuration as electronic component C1. The multiple sample electronic components include sample electronic components having first external terminals and sample electronic components having second external terminals. The first external terminals are external terminals with a normal plating layer state. The second external terminals are external terminals with an abnormal plating layer state. In the first external terminal, for example, the sintered electrode is entirely covered with a tin plating layer. In the second external terminal, for example, the nickel plating layer or the sintered electrode is exposed.

[0027] The plurality of sample RGB image data indicate the first external terminal and the second external terminal. For example, the plurality of sample RGB image data include image data captured at the first external terminal and image data captured at the second external terminal. The acquisition unit 11 may include an imaging device.

[0028] The converter 12 converts multiple sample RGB image data into sample distribution data. The sample distribution data includes data indicating the components of the image data in the HLS color space. The sample distribution data includes, for example, a luminance histogram. Figures 3(a), 3(b), 4(a), and 4(b) show the sample distribution data converted from the sample RGB images.

[0029] Figure 3(a) shows sample distribution data for an electronic component with exposed sintered electrodes on the external terminals. Figure 3(b) shows sample distribution data for an electronic component with exposed nickel-plated layers on the external terminals. Figure 4(a) shows sample distribution data for an electronic component with exposed tin-plated layers on the external terminals. Figure 4(b) shows sample distribution data for an electronic component that has passed through an inspection device after a tin-plated layer has been formed. For example, in the electronic component manufacturing process, electronic components pass through a bowl feeder for various inspections after a tin-plated layer has been formed. Note that the electronic component that has passed through the inspection device differs from the electronic component in data D3 in terms of appearance, but is equivalent to the electronic component in data D3 in terms of the performance of the tin-plated layer.

[0030] The converter 12 converts a plurality of sample RGB images into sample distribution data indicating at least one of the L component and the S component of the HLS color space. The converter 12 converts the plurality of sample RGB image data into at least one of a luminance histogram indicating the L component and a luminance histogram indicating the S component. The converter 12 converts the sample RGB image data indicating the first external terminal and the sample RGB image data indicating the second external terminal into sample distribution data, respectively.

[0031] The conversion unit 12 converts a plurality of sample RGB image data into, for example, first sample distribution data and second sample distribution data. The first sample distribution data is data obtained by converting the RGB image data into the L component of the HLS color space. The second sample distribution data is data obtained by converting the RGB image data into the S component of the HLS color space. For example, the first sample distribution data is data obtained by converting the RGB image data into a luminance histogram representing the L component of the HLS color space. The second sample distribution data is data obtained by converting the RGB image data into a luminance histogram representing the S component of the HLS color space.

[0032] The converter 12 converts the sample RGB image data representing the first external terminal into first sample distribution data representing the L component of the HLS color space and second sample distribution data representing the S component of the HLS color space. The converter 12 converts the sample RGB image data representing the second external terminal into the first sample distribution data representing the L component of the HLS color space and second sample distribution data representing the S component of the HLS color space.

[0033] The calculation unit 13 calculates the Mahalanobis distance for the sample distribution data for each of the first external terminal and the second external terminal. The calculation unit 13 calculates the Mahalanobis distance from the sample distribution data into which the sample RGB image data representing the first external terminal is converted. The calculation unit 13 calculates the Mahalanobis distance from the sample distribution data into which the sample RGB image data representing the second external terminal is converted.

[0034] For example, the calculation unit 13 calculates the Mahalanobis distance for each of the first and second external terminals from the luminance histograms acquired from the conversion unit 12. For example, the calculation unit 13 performs sampling at multiple luminance points in the luminance histograms and calculates the Mahalanobis distance based on information extracted by the sampling. The positions of the sampling points are positions where the difference between the luminance histograms of the first and second external terminals is relatively large. Information on the positions of the sampling points is stored in advance in, for example, the storage unit 14. For example, in Figures 3(a), 3(b), 4(a), and 4(b), sampling is performed at luminance points with luminances of "98," "150," "203," and "255." Figures 3(a), 3(b), 4(a), and 4(b) are graphs showing luminance histograms in the HLS color space.

[0035] In the example shown in this embodiment, the calculation unit 13 calculates a Mahalanobis distance related to the first sample distribution data and a Mahalanobis distance related to the second sample distribution data for each of the first external terminal and the second external terminal. The calculation unit 13 calculates the Mahalanobis distance from each of the first sample distribution data obtained by converting sample RGB image data representing the first external terminal and the second sample distribution data obtained by converting sample RGB image data representing the second external terminal. The calculation unit 13 calculates the Mahalanobis distance from each of the second sample distribution data obtained by converting sample RGB image data representing the first external terminal and the second sample distribution data obtained by converting sample RGB image data representing the second external terminal.

[0036] Figures 5, 6, and 7 are graphs showing the Mahalanobis distances between external terminals with normal plating layers and external terminals with abnormal plating layers. Sampling was performed at four brightness points: 98, 150, 203, and 255. The Mahalanobis distances were calculated based on the information extracted by the sampling. In Figures 5, 6, and 7, data D1 is data for external terminals with exposed sintered electrodes. Data D2 is data for external terminals with exposed nickel plating layers. Data D3 is data for external terminals covered with tin plating layers. Data D4 is data for external terminals of electronic components that passed through an inspection device after a tin plating layer was formed. Data D5 is data for external terminals with marks caused by roller pressure during the electrical characteristics inspection process. Data D3, D4, and D5 correspond to data for external terminals with normal tin plating layers. Data D1 and D2 correspond to data for external terminals with abnormal tin plating layers.

[0037] In the manufacturing process of the electronic component according to data D4, for example, after the electronic component has been provided with a tin plating layer, it passes through a bowl feeder for various inspections. The electronic component that has passed through the bowl feeder differs in appearance from the electronic component according to data D3, but the performance of the tin plating layer is equivalent to that of the electronic component according to data D3.

[0038] In the electrical characteristics inspection process for data D5, for example, a roller is brought into contact with the external terminals of the electronic component, and the electrical characteristics are measured. The marks left on the external terminals in this case tend to be deeper than the marks left on the external terminals of the electronic component that passed through the bowl feeder in data D4. Even for the external terminals that have marks left by being pressed by the roller in the electrical characteristics inspection process, the performance of the tin plating layer is maintained at the same level as that of the electronic component in data D3.

[0039] Fig. 5 shows data obtained by sampling one external terminal 10 times at four brightness points. Fig. 6 shows data obtained by sampling one external terminal 50 times at four brightness points. Fig. 7 shows data obtained by sampling one external terminal 100 times at four brightness points. In Figs. 5 to 7, Mahalanobis distance data D1 and D2 for external terminals with abnormal tin plating layers are separated from Mahalanobis distance data D3, D4, and D5 for external terminals with normal tin plating layers in the vertical direction.

[0040] In Fig. 5, the range R1 within which the Mahalanobis distance data D1 and D2 for external terminals with abnormal plating layer states are separated from the Mahalanobis distance data D3, D4, and D5 for external terminals with normal plating layer states is 5926.58. In Fig. 6, the range R2 within which the Mahalanobis distance data D1 and D2 for external terminals with abnormal plating layer states are separated from the Mahalanobis distance data D3, D4, and D5 for external terminals with normal plating layer states is 1291.38. In Fig. 7, the range R3 within which the Mahalanobis distance data D1 and D2 for external terminals with abnormal plating layer states are separated from the Mahalanobis distance data D3, D4, and D5 for external terminals with normal plating layer states is 939.00.

[0041] The storage unit 14 stores information output from each functional unit. For example, the storage unit 14 stores a plurality of sample RGB image data acquired by the acquisition unit 11, sample distribution data converted by the conversion unit 12, and the Mahalanobis distance calculated by the calculation unit 13. Furthermore, the storage unit 14 stores information selected by the selection unit 15 and a threshold determined by the threshold determination unit 16.

[0042] The selection unit 15 selects components of the HLS color space used in the visual inspection of electronic components. For example, the selection unit 15 selects either the L component or the S component based on the Mahalanobis distance related to the first sample distribution data and the Mahalanobis distance related to the second sample distribution data. The selection unit 15 selects either the L component or the S component based on a comparison result between the relationship between the Mahalanobis distance calculated from the first sample distribution data indicating the first external terminals and the Mahalanobis distance calculated from the first sample distribution data indicating the second external terminals, and the relationship between the Mahalanobis distance calculated from the second sample distribution data indicating the first external terminals and the Mahalanobis distance calculated from the second sample distribution data indicating the second external terminals.

[0043] For example, the selection unit 15 compares the range of separation between the Mahalanobis distance data D1 and D2 for external terminals with abnormal plating layer states and the Mahalanobis distance data D3, D4, and D5 for external terminals with normal plating layer states between the first sample distribution data and the second sample distribution data. If the range of the first sample distribution data is wider than the range of the second sample distribution data, the selection unit 15 selects the L component. If the range of the second sample distribution data is wider than the range of the first sample distribution data, the selection unit 15 selects the S component.

[0044] The threshold determination unit 16 determines a threshold value TH used in the visual inspection of electronic components. The threshold determination unit 16 determines the threshold value TH based on the Mahalanobis distance for each of the first external terminal and the second external terminal. The threshold determination unit 16 determines the threshold value TH based on the relationship between the Mahalanobis distance calculated from the sample distribution data representing the first external terminal and the Mahalanobis distance calculated from the sample distribution data representing the second external terminal. The threshold determination unit 16 determines the threshold value TH between the Mahalanobis distance calculated from the sample distribution data representing the first external terminal and the Mahalanobis distance calculated from the sample distribution data representing the second external terminal.

[0045] For example, the threshold determination unit 16 determines a threshold value TH between the Mahalanobis distance calculated from the first sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the first sample distribution data indicating the second external terminal. For example, the threshold determination unit 16 determines a threshold value TH between the Mahalanobis distance calculated from the second sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the second sample distribution data indicating the second external terminal. For example, the threshold determination unit 16 determines the threshold value TH within a range in which the Mahalanobis distance data D1 and D2 relating to external terminals with abnormal plating layer conditions are separated from the Mahalanobis distance data D3, D4, and D5 relating to external terminals with normal plating layer conditions.

[0046] Next, each functional unit of the visual inspection system 1 in the inspection execution phase will be described in more detail. In the inspection execution phase, the inspection execution unit 20 operates. The inspection execution unit 20 executes a visual inspection of the condition of the plating layer. For example, the inspection execution unit 20 determines whether the tin plating layer is normally configured through the visual inspection.

[0047] In the inspection execution phase, the acquisition unit 11 acquires RGB image data. The RGB image data acquired by the acquisition unit 11 includes RGB image data obtained by capturing an image of an electronic component to be inspected. The electronic component to be inspected corresponds to, for example, the electronic component C1 shown in FIG. 2. The RGB image data acquired by the acquisition unit 11 indicates the external terminals 5 of the electronic component C1. For example, the RGB image data acquired by the acquisition unit 11 includes captured image data of the external terminals 5. The acquisition unit 11 may include an imaging device.

[0048] The conversion unit 12 converts the RGB image data acquired by the acquisition unit 11 into distribution data. The conversion unit 12 converts the RGB image data into distribution data indicating components of the HLS color space. The distribution data includes data indicating the components of the image data in the HLS color space. The distribution data includes, for example, a luminance histogram.

[0049] The conversion unit 12 converts RGB image data obtained by capturing an image of the external terminals 5 of the electronic component C1 to be inspected into distribution data indicating at least one of the L component and S component of the HLS color space. The L component is a component indicating lightness. The S component is a component indicating saturation. For example, the conversion unit 12 converts the RGB image data into at least one of a luminance histogram indicating the L component and a luminance histogram indicating the S component. For example, the conversion unit 12 converts the RGB image data acquired by the acquisition unit 11 into first distribution data indicating the L component of the HLS color space and second distribution data indicating the S component of the HLS color space.

[0050] The calculation unit 13 calculates the Mahalanobis distance for the external terminal 5 of the electronic component C1 from the distribution data acquired from the conversion unit 12. The calculation unit 13 samples the distribution data acquired from the conversion unit 12 at multiple points, and calculates the Mahalanobis distance based on the information extracted by the sampling.

[0051] For example, the calculation unit 13 calculates the Mahalanobis distance for the external terminal 5 of the electronic component C1 from the luminance histogram acquired from the conversion unit 12. For example, the calculation unit 13 performs sampling at multiple luminance points in the luminance histogram and calculates the Mahalanobis distance based on the information extracted by the sampling. The positions of the sampling points include the sampling points in the inspection preparation phase. The position information of the sampling points is stored in advance in the storage unit 14, for example.

[0052] The storage unit 14 stores information output from each functional unit. For example, the storage unit 14 stores the RGB image data acquired by the acquisition unit 11, the distribution data converted by the conversion unit 12, and the Mahalanobis distance calculated by the calculation unit 13. Furthermore, the storage unit 14 stores the comparison result by the comparison unit 21 and the determination result by the determination unit 22.

[0053] In determining the state of the plating layer, the comparison unit 21 compares the Mahalanobis distance for the external terminal 5 of the electronic component C1 with a threshold value TH. For example, the comparison unit 21 determines whether the Mahalanobis distance for the external terminal 5 of the electronic component C1 is above or below the threshold value TH, and outputs the result as the comparison result.

[0054] The determination unit 22 determines the state of the plating layer on the external terminal 5 based on the Mahalanobis distance for the external terminal 5 of the electronic component C1. The determination unit 22 determines the state of the plating layer on the external terminal 5 based on the Mahalanobis distance for distribution data indicating the selected component in determining the state of the plating layer. The determination unit 22 determines the state of the plating layer based on the comparison result of the comparison unit 21. The determination unit 22 determines, for example, whether the target plating layer is properly provided on the external terminal. For example, the determination unit 22 determines whether the baked electrodes of the electronic component being inspected are covered with a tin plating layer. For example, in the example shown in FIG. 5, if the Mahalanobis distance for the external terminal 5 of the electronic component C1 is below the threshold TH, the determination unit 22 determines that the baked electrodes of the electronic component being inspected are covered with a tin plating layer. If the Mahalanobis distance for the external terminal 5 of the electronic component C1 is above the threshold TH, the determination unit 22 determines that there are portions of the baked electrodes of the electronic component being inspected that are not covered with a tin plating layer.

[0055] Next, the hardware configuration of the visual inspection system 1 will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of the hardware configuration of the visual inspection system 1.

[0056] The visual inspection system 1 includes a processor 101, a memory 102, a storage 103, a communication device 104, an output device 105, and an input device 106. The visual inspection system 1 includes one or more computers configured with this hardware and software such as a program. The visual inspection system 1 may be configured with one computer or may be configured with multiple computers. The visual inspection system 1 is realized in cooperation with the hardware.

[0057] When the visual inspection system 1 is configured by multiple computers, these computers may be connected locally or via a communication network such as the Internet or an intranet. This connection logically constructs one visual inspection system 1.

[0058] The processor 101 executes an operating system, application programs, etc. The memory 102 is composed of a read-only memory (ROM) and a random-access memory (RAM). For example, at least some of the various functional units, including the acquisition unit 11, conversion unit 12, calculation unit 13, storage unit 14, selection unit 15, threshold determination unit 16, comparison unit 21, and judgment unit 22, can be realized by the processor 101 and the memory 102.

[0059] The storage 103 is a storage medium configured by a hard disk, a flash memory, etc. The storage 103 generally stores a larger amount of data than the memory 102. For example, at least a part of the storage unit 14 can be realized by the storage 103.

[0060] The communication device 104 is configured by a network card or a wireless communication module. For example, at least a part of the acquisition unit 11 can be realized by the communication device 104. The output device 105 is configured by a printer, a display, and the like.

[0061] The input device 106 includes a keyboard, a mouse, a touch panel, etc. For example, the acquisition unit 11 and at least a part of the selection unit 15 can be realized by the input device 106. The input device 106 may further include an imaging device of the acquisition unit 11.

[0062] The storage 103 stores in advance a program and data necessary for processing. This program causes a computer to execute each functional element of the visual inspection system 1. This program causes, for example, each process in the visual inspection method described below to be executed on the computer. This program may be provided in the form of being recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. This program may also be provided as a data signal via a communication network.

[0063] Next, an example of a visual inspection method in this embodiment will be described with reference to Fig. 9 and Fig. 10. For example, the visual inspection method executes an inspection preparation phase and an inspection execution phase. First, an example of the inspection preparation phase of the visual inspection method will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the inspection preparation phase of the visual inspection method.

[0064] First, sample RGB image data is acquired (process S11). In process S11, for example, the acquiring unit 11 acquires a plurality of sample RGB image data. In the example shown in this embodiment, the plurality of sample RGB image data is RGB image data obtained by capturing images of first and second external terminals of a sample electronic component using an imaging device. The first external terminals are external terminals with a normal plating layer state. The second external terminals are external terminals with an abnormal plating layer state.

[0065] Next, the plurality of sample RGB image data are converted into the HLS color space (process S12). In process S12, for example, the conversion unit 12 converts the plurality of sample RGB image data into sample distribution data. The sample distribution data indicates at least one of the L component and the S component of the HLS color space. For example, the conversion unit 12 converts the sample RGB image data indicating the first external terminal into first sample distribution data indicating the L component of the HLS color space and second sample distribution data indicating the S component of the HLS color space. The conversion unit 12 converts the sample RGB image data indicating the second external terminal into first sample distribution data indicating the L component of the HLS color space and second sample distribution data indicating the S component of the HLS color space. The first sample distribution data includes a luminance histogram indicating the L component of the HLS color space. The second sample distribution data includes a luminance histogram indicating the S component of the HLS color space.

[0066] Next, the Mahalanobis distance is calculated (process S13). In process S13, the calculation unit 13 calculates the Mahalanobis distance for each of the first external terminal and the second external terminal from the luminance histogram acquired in process S12. For example, the calculation unit 13 performs sampling at a plurality of luminance points in the luminance histogram and calculates the Mahalanobis distance based on information extracted by the sampling. For example, the calculation unit 13 calculates the Mahalanobis distance from each of first sample distribution data obtained by converting sample RGB image data representing the first external terminal and first sample distribution data obtained by converting sample RGB image data representing the second external terminal. The calculation unit 13 calculates the Mahalanobis distance from each of second sample distribution data obtained by converting sample RGB image data representing the first external terminal and second sample distribution data obtained by converting sample RGB image data representing the second external terminal.

[0067] Next, a component of the HLS space used in the visual inspection of electronic components is selected (process S14). In process S14, the selection unit 15 selects either the L component or the S component based on the Mahalanobis distance related to the first sample distribution data and the Mahalanobis distance related to the second sample distribution data. For example, the comparison unit 21 compares the relationship between the Mahalanobis distance calculated from the first sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the first sample distribution data indicating the second external terminal, with the relationship between the Mahalanobis distance calculated from the second sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the second sample distribution data indicating the second external terminal. The selection unit 15 selects either the L component or the S component based on the comparison result of the comparison unit 21. The Mahalanobis distance calculated in process S14 is stored in the storage unit 14.

[0068] Next, a threshold value used in the visual inspection of electronic components is determined (process S15). In process S15, the threshold value determination unit 16 determines a threshold value based on the Mahalanobis distance calculated in process S13. For example, the threshold value determination unit 16 determines a threshold value between the Mahalanobis distance calculated from the first sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the first sample distribution data indicating the second external terminal. For example, the threshold value determination unit 16 determines a threshold value between the Mahalanobis distance calculated from the second sample distribution data indicating the first external terminal and the Mahalanobis distance calculated from the second sample distribution data indicating the second external terminal. The threshold value determined in process S15 is stored in the storage unit 14.

[0069] When process S15 is completed, the series of processes in the inspection preparation phase of the visual inspection method is completed. Although an example of the inspection preparation phase of the visual inspection method has been described above, the order of the processes is not limited to this. For example, process S14 and process S15 may be performed in either order, or may be performed in parallel. Either process S14 or process S15 may not be performed.

[0070] Next, an example of the inspection execution phase of the visual inspection method will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the inspection execution phase of the visual inspection method.

[0071] First, RGB image data of an electronic component to be inspected is acquired (process S21). In process S21, for example, the acquiring unit 11 acquires the RGB image data. The RGB image data acquired by the acquiring unit 11 includes RGB image data obtained by capturing an image of an external terminal of the electronic component to be inspected by an imaging device.

[0072] Next, the RGB image data of the electronic component to be inspected is converted into the HLS color space (process S22). In process S22, for example, the conversion unit 12 converts the RGB image data acquired in process S21 into distribution data indicating the components of the HLS color space. For example, the conversion unit 12 converts the RGB image data into distribution data indicating at least one of the L component and the S component of the HLS color space. For example, the conversion unit 12 converts the RGB image data acquired in process S21 into first distribution data indicating the L component of the HLS color space and second distribution data indicating the S component of the HLS color space. The first distribution data includes a luminance histogram indicating the L component of the HLS color space. The second distribution data includes a luminance histogram indicating the S component of the HLS color space.

[0073] Next, the Mahalanobis distance is calculated (process S23). In process S23, the calculation unit 13 calculates the Mahalanobis distance for the external terminal of the electronic component to be inspected from the brightness histogram acquired in process S22. For example, the calculation unit 13 performs sampling at a plurality of brightness points in the brightness histogram and calculates the Mahalanobis distance based on the information extracted by the sampling.

[0074] For example, the calculation unit 13 calculates the Mahalanobis distance from distribution data indicating components selected from the HLS color space. For example, the calculation unit 13 calculates the Mahalanobis distance from first distribution data into which RGB image data indicating external terminals has been converted. The calculation unit 13 calculates the Mahalanobis distance from second distribution data into which RGB image data indicating external terminals has been converted.

[0075] Next, the Mahalanobis distance is compared with a threshold value (process S24). In process S24, the comparison unit 21 compares the Mahalanobis distance related to the distribution data with the threshold value stored in the storage unit 14. In other words, the comparison unit 21 compares the Mahalanobis distance related to the electronic component to be inspected with the threshold value stored in the storage unit 14. The threshold value stored in the storage unit 14 is, for example, the threshold value determined in process S15.

[0076] Next, the state of the plating layer is determined (process S25). In process S25, the determination unit 22 determines the state of the plating layer on the external terminal based on the Mahalanobis distance for the electronic component to be inspected. For example, the determination unit 22 determines the state of the plating layer on the external terminal based on the comparison result in process S24. The determination unit 22 determines whether the baked electrodes of the electronic component to be inspected are covered with a tin plating layer. In process S24, the determination result of the determination unit 22 is stored in the storage unit 14.

[0077] When step S25 is completed, the series of steps in the inspection execution phase of the visual inspection method is completed. Although an example of the inspection execution phase of the visual inspection method has been described above, the order of the steps is not limited to this.

[0078] Next, the effects of the visual inspection system 1 and the visual inspection method according to this embodiment and the modified example will be described.

[0079] FIG. 11 is a graph showing the comparison results of the average saturation between external terminals with normal plating layers and external terminals with abnormal plating layers. As in FIGS. 5 to 7, data D1 is data relating to external terminals with exposed sintered electrodes. Data D2 is data relating to external terminals with exposed nickel plating layers. Data D3 is data relating to external terminals covered with tin plating layers. Data D4 is data relating to external terminals of electronic components that passed through a bowl feeder after a tin plating layer was formed. Data D5 is data relating to external terminals on which marks were formed due to pressing by rollers during the electrical characteristics inspection process. Data D3, D4, and D5 correspond to data relating to external terminals with normal tin plating layers. Data D1 and D2 correspond to data relating to external terminals with abnormal tin plating layers. In this case, the Mahalanobis distance data D2 relating to the external terminals with abnormal tin plating layers and the Mahalanobis distance data D4 relating to the external terminals with normal tin plating layers overlap in the vertical axis direction. It is considered difficult to determine whether the condition of the tin plating layer is normal or not based on the average saturation.

[0080] In the visual inspection of this embodiment, RGB image data is converted into distribution data indicating at least one of the L and S components of the HLS color space, and the plating layer condition is determined based on the Mahalanobis distance for the distribution data. In this case, as shown in FIGS. 5 to 7, the Mahalanobis distance data D1 and D2 for external terminals with abnormal tin plating layers are separated from the Mahalanobis distance data D3, D4, and D5 for external terminals with normal tin plating layers along the vertical axis. This ensures accurate determination of the plating layer condition. This visual inspection method ensures a high inspection speed per electronic component C and ensures production throughput. Therefore, this visual inspection method ensures accurate determination of the plating layer condition while maintaining high production throughput.

[0081] The appearance detection method described in this embodiment may further include acquiring a plurality of sample RGB image data, converting the plurality of sample RGB image data into first sample distribution data and second sample distribution data, calculating a Mahalanobis distance for a first external terminal and a second external terminal from the first sample distribution data and calculating a Mahalanobis distance from the second sample distribution data, and selecting either an L component or an S component based on the Mahalanobis distance for the first sample distribution data and the Mahalanobis distance for the second sample distribution data. The plurality of sample RGB image data indicates a first external terminal having a normal plating layer state and a second external terminal having an abnormal plating layer state. The first sample distribution data indicates the L component in the HLS color space. The second sample distribution data indicates the S component in the HLS color space. In converting the distribution data, the RGB image data may be converted into distribution data indicating a selected component from the L component and the S component. In calculating the Mahalanobis distance for the distribution data, the Mahalanobis distance for the selected component may be calculated for the external terminal of the electronic component. In determining the condition of the plating layer, the condition of the plating layer on the external terminal may be determined based on the Mahalanobis distance related to the distribution data indicated by the selected component. As a result of extensive research, the inventors have found that the component between the L component and the S component for which inspection accuracy is ensured differs depending on, for example, the type of electronic component. According to the above method, the component between the L component and the S component for which inspection accuracy is high can be used for visual inspection. Therefore, the accuracy of determining the condition of the plating layer can be ensured while minimizing computational processing.

[0082] In the appearance detection shown in this embodiment, the state of the plating layer is determined by comparing the Mahalanobis distance of the electronic component with a threshold value, which reduces the amount of calculation while ensuring accuracy in determining the state of the plating layer.

[0083] The appearance detection method shown in this embodiment further includes acquiring sample RGB image data, converting the sample RGB image data into sample distribution data indicating at least one of the L and S components of the HLS color space, calculating a Mahalanobis distance for the sample distribution data for each of the first and second external terminals, and determining a threshold value based on the Mahalanobis distance for each of the first and second external terminals. The sample RGB image data indicates first external terminals with normal plating layer conditions and second external terminals with abnormal plating layer conditions. In this case, the accuracy of determining the plating layer condition can be further improved.

[0084] In the appearance detection method shown in this embodiment, the plating layer contains tin, which ensures the accuracy of determining the state of the plating layer while reliably ensuring the function of the plating layer.

[0085] The above describes embodiments and modifications of the present invention, but the present invention is not necessarily limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present invention.

[0086] For example, the visual inspection system 1 may create a learning model using machine learning in the inspection preparation phase. In this case, for example, the inspection preparation unit 10 creates a learning model trained using a learning dataset containing various information as information to be used in the visual inspection. Machine learning is a technique for autonomously finding laws or rules by iteratively learning based on given information. For example, in the machine learning performed in the visual inspection system 1, model parameters such as activation functions and weighting values ​​are optimized by learning using a learning dataset. This creates a learning model.

[0087] The machine learning performed in the visual inspection system 1 is, for example, deep learning. This machine learning is supervised learning configured using a multilayer perceptron (MLP). The visual inspection system 1 uses machine learning configured to include a neural network. The machine learning performed in the visual inspection system 1 is not limited to supervised learning.

[0088] For example, the neural network used in the visual inspection system 1 is a long short-term memory (LSTM) network. The LSTM is a type of recurrent neural network. The type of neural network used in the visual inspection system 1 is not limited to this.

[0089] In this case, the inspection preparation unit 10 may acquire the training data set from the acquisition unit 11, or may acquire a training data set that has been stored in advance in the storage unit 14. The inspection preparation unit 10 may acquire the training data set from outside the visual inspection system 1 via a communication network or the like.

[0090] As can be understood from the above description of the embodiments, the present specification includes disclosure of the following aspects. (Appendix 1) obtaining RGB image data representing an external terminal of an electronic component; converting the RGB image data into distribution data indicating at least one of the L component and the S component of the HLS color space; calculating a Mahalanobis distance from the distribution data for the external terminals of the electronic component; determining a state of a plating layer on the external terminal based on the Mahalanobis distance for the external terminal of the electronic component. (Appendix 2) acquiring a plurality of sample RGB image data representing first external terminals having a normal plating layer state and second external terminals having an abnormal plating layer state; converting the plurality of sample RGB image data into first sample distribution data representing an L component of the HLS color space and second sample distribution data representing an S component of the HLS color space; calculating a Mahalanobis distance from the first sample distribution data and a Mahalanobis distance from the second sample distribution data for each of the first external terminal and the second external terminal; selecting one of the L component and the S component based on a Mahalanobis distance related to the first sample distribution data and a Mahalanobis distance related to the second sample distribution data, In the conversion of the distribution data, the RGB image data is converted into distribution data indicating a component selected from the L component and the S component, In the calculation of the Mahalanobis distance regarding the distribution data, the Mahalanobis distance regarding the distribution data indicating the selected component is calculated for an external terminal of the electronic component; 2. The visual inspection method according to claim 1, wherein, in determining the state of the plating layer, the state of the plating layer on the external terminal is determined based on a Mahalanobis distance relating to distribution data indicating the selected component. (Appendix 3) 3. The visual inspection method according to claim 1, wherein the determination of the state of the plating layer comprises determining the state of the plating layer based on a result of comparing the Mahalanobis distance of the electronic component with a threshold value. (Appendix 4) acquiring a plurality of sample RGB image data representing first external terminals having a normal plating layer state and second external terminals having an abnormal plating layer state; converting the data into sample distribution data representing at least one of the L component and the S component of the HLS color space; calculating a Mahalanobis distance from the sample distribution data for each of the first external terminal and the second external terminal; 4. The visual inspection method according to claim 3, further comprising determining the threshold value based on the Mahalanobis distance for each of the first external terminal and the second external terminal. (Appendix 5) 5. The appearance inspection method according to claim 1, wherein the plating layer contains tin. (Appendix 6) an acquisition unit that acquires RGB image data indicating an external terminal of an electronic component; a conversion unit that converts the RGB image data into distribution data that indicates at least one of an L component and an S component of an HLS color space; a calculation unit that calculates a Mahalanobis distance from the distribution data for the external terminals of the electronic component; a determination unit that determines a state of a plating layer on the external terminal based on the Mahalanobis distance related to the external terminal of the electronic component. [Explanation of symbols]

[0091] 1...visual inspection system, 5...external terminal, 11...acquisition unit, 12...conversion unit, 13...calculation unit, 22...determination unit, C1...electronic component, R1, R2, R3...range, TH...threshold value.

Claims

1. acquiring RGB image data representing an external terminal of an electronic component; converting the RGB image data into distribution data indicating at least one of the L component and the S component of the HLS color space; calculating a Mahalanobis distance from the distribution data for the external terminals of the electronic component; determining a state of a plating layer on the external terminal based on the Mahalanobis distance for the external terminal of the electronic component.

2. acquiring a plurality of sample RGB image data representing first external terminals having a normal plating layer state and second external terminals having an abnormal plating layer state; converting the plurality of sample RGB image data into first sample distribution data representing an L component of an HLS color space and second sample distribution data representing an S component of the HLS color space; calculating a Mahalanobis distance from the first sample distribution data and a Mahalanobis distance from the second sample distribution data for each of the first external terminal and the second external terminal; selecting one of the L component and the S component based on a Mahalanobis distance related to the first sample distribution data and a Mahalanobis distance related to the second sample distribution data, In the conversion of the distribution data, the RGB image data is converted into distribution data indicating a component selected from the L component and the S component, In the calculation of the Mahalanobis distance regarding the distribution data, the Mahalanobis distance regarding the distribution data indicating the selected component is calculated for an external terminal of the electronic component; 2. The visual inspection method according to claim 1, wherein in determining the state of the plating layer, the state of the plating layer on the external terminal is determined based on a Mahalanobis distance related to distribution data indicating the selected component.

3. 2. The visual inspection method according to claim 1, wherein the determination of the state of the plating layer is made based on a result of comparing the Mahalanobis distance of the electronic component with a threshold value.

4. acquiring a plurality of sample RGB image data representing first external terminals having a normal plating layer state and second external terminals having an abnormal plating layer state; converting the data into sample distribution data representing at least one of the L and S components of the HLS color space; calculating a Mahalanobis distance from the sample distribution data for each of the first external terminal and the second external terminal; 4. The visual inspection method according to claim 3, further comprising determining the threshold value based on the Mahalanobis distance between the first external terminal and the second external terminal.

5. The appearance inspection method according to claim 1 , wherein the plating layer contains tin.

6. an acquisition unit that acquires RGB image data representing external terminals of an electronic component; a conversion unit that converts the RGB image data into distribution data that indicates at least one of an L component and an S component of an HLS color space; a calculation unit that calculates a Mahalanobis distance from the distribution data for the external terminals of the electronic component; a determination unit that determines a state of a plating layer on the external terminal based on the Mahalanobis distance related to the external terminal of the electronic component.

Images