Testing system, testing method, and testing program

Abstract

To provide an inspection system, etc., which enable, by a simple device configuration, the acceptability or emission characteristic of a light-emitting element to be highly accurately inspected.SOLUTION: The inspection system comprises: a camera which is installed in such a way that the camera can image a light-emitting element which is turned on, the camera configured to image the light-emitting element and output an image signal; an image generation unit generating image data representing an image in which the light-emitting element is photographed, on the basis of the image signal output from the camera; a region extraction unit extracting the image of the light-emitting element in the image as the inspection target, and further extracting, from inside of the inspection target region, a region where there is occurrence of saturation, as an irrelevant region; and a measurement unit calculating the luminance or hue of the light-emitting element on the basis of the pixel value of an effective inspection target region, that is the inspection target region, with the irrelevant region excluded therefrom.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an inspection system, an inspection method, and an inspection program for inspecting a light-emitting element.

Background Art

[0002] In recent years, light-emitting elements such as LEDs (Light Emitting Diodes) are used in various devices such as lighting devices, displays, electronic billboards, traffic signals, pachinko machines, and game machines. When shipping or using such devices, the quality and light-emitting characteristics of the light-emitting elements mounted on the circuit board are inspected. The inspection is performed using a dedicated inspection device, but it may also be performed visually in a simple manner.

[0003] For example, Patent Document 1 discloses an inspection device that inspects the lighting state of a light-emitting element based on image data obtained by imaging the light-emitting element with a camera.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When visually inspecting a light-emitting element, the burden on the operator is large. Also, it is very difficult to visually distinguish differences in color tone.

[0006] On the other hand, when inspecting based on images of light-emitting elements, the following problems arise. Specifically, the brightness of an emitting LED is highest in the direction of the optical axis. Therefore, when imaging with a camera from a direction directly facing the LED, the center of the LED image becomes overexposed, a phenomenon known as saturation. The saturated area can no longer be said to represent the brightness or color of the LED. Therefore, it is difficult to accurately determine the brightness and color of an LED based on such an image.

[0007] To address these problems, Patent Document 1 describes a process in which the camera's shooting angle is changed to cause the camera to image the light-emitting element of the object to be inspected, the image data captured by the camera is acquired, and the brightness values ​​for each color component are extracted from the image data for each shooting angle.

[0008] However, changing the camera's shooting angle requires a mechanism to adjust the camera angle and a control unit to manage this mechanism. This can lead to a more complex device configuration and increased costs. Furthermore, multiple images may be required for a single object being inspected, potentially increasing the inspection time, and processing multiple image data points with different shooting angles may complicate the computational processing.

[0009] The present invention has been made in view of the above, and aims to provide an inspection system, inspection method, and inspection program that can accurately inspect the quality or luminescence characteristics of a light-emitting element with a simple device configuration. [Means for solving the problem]

[0010] To solve the above problems, an inspection system according to one aspect of the present invention comprises: a camera installed to capture images of a lit light-emitting element and outputting an image signal when the light-emitting element is captured; an image generation unit that generates image data representing an image of the light-emitting element based on the image signal output from the camera; an area extraction unit that extracts the image of the light-emitting element in the image as an inspection target area, and further extracts an area where saturation occurs from inside the inspection target area as an exclusion area; and a measurement unit that calculates the brightness or color of the light-emitting element based on the pixel values ​​of an effective inspection target area, which is the area obtained by subtracting the exclusion area from the inspection target area.

[0011] In the above inspection system, the image generated by the image generation unit is represented by values ​​in the RGB color space, and the region extraction unit may convert the values ​​of the pixels constituting the image to values ​​in the HLS color space and extract the excluded region based on at least the L value in the HLS color space.

[0012] In the above inspection system, the measuring unit may calculate the average R, G, or B value in the RGB color space, or the H, L, or S value in the HLS color space, of the pixels in the effective inspection target area.

[0013] The above inspection system may further include a determination unit that determines the quality or light emission characteristics of the light-emitting element by comparing the average value with a preset threshold.

[0014] In the above inspection system, the camera captures images of a plurality of light-emitting elements, and the region extraction unit further includes a determination unit that extracts the contour of the inspection target area extracted for each of the plurality of light-emitting elements, counts the number of contours within the pre-set inspection area for the image, and determines whether the counted number matches a pre-set value.

[0015] In the above inspection system, the region extraction unit may further include a determination unit that extracts the contour of the region to be inspected and determines whether or not the contour is located inside a predetermined region relative to the image.

[0016] The above inspection system may further include a determination unit that measures the area of ​​the area to be inspected and determines whether the measured area falls within a predetermined range.

[0017] In the above inspection system, the image generation unit may generate a test image based on an image signal generated by capturing a test light-emitting element with the camera, and the system may further include a display unit that displays a setting screen including the test image, an input unit that accepts operations for the inspection system, and a display control unit that controls the display on the setting screen, and the display control unit may display on the setting screen a slider that can be operated by the input unit, which is at least one slider for adjusting the range of pixel values, and may overlay a predetermined color on the region of the test image in which the pixel values ​​are within the range adjusted by the at least one slider.

[0018] The inspection system described above may further include a condition setting unit that sets the range adjusted by the at least one slider as an extraction condition used by the region extraction unit to extract the inspection target region or the exclusion region.

[0019] The inspection system described above may further include a condition setting unit that sets a range adjusted by the at least one slider as a judgment criterion used to determine the quality or light emission characteristics of the light-emitting element based on the brightness or color of the light-emitting element calculated by the measurement unit.

[0020] The inspection system may further include a drive mechanism that moves the device in a predetermined direction at a predetermined speed within a plane orthogonal to the optical axis of the camera, and a control unit that causes the camera to image the device at a predetermined period in synchronization with the movement of the device by the drive mechanism.

[0021] Another aspect of the inspection method according to the present invention is to image a light-emitting element that is installed so as to be imaged by a camera while the light-emitting element is lit, generate image data representing an image in which the light-emitting element appears based on an image signal output from the camera, extract an image of the light-emitting element in the image as an inspection target region, further extract a region where saturation has occurred as an excluded region from the inside of the inspection target region, and calculate the luminance or color tone of the light-emitting element based on the pixel values of an effective inspection target region, which is a region obtained by excluding the excluded region from the inspection target region.

[0022] Still another aspect of the inspection program according to the present invention is to cause a computer to execute imaging a light-emitting element that is installed so as to be imaged by a camera while the light-emitting element is lit, generating image data representing an image in which the light-emitting element appears based on an image signal output from the camera, extracting an image of the light-emitting element in the image as an inspection target region, further extracting a region where saturation has occurred as an excluded region from the inside of the inspection target region, and calculating the luminance or color tone of the light-emitting element based on the pixel values of an effective inspection target region, which is a region obtained by excluding the excluded region from the inspection target region.

Advantages of the Invention

[0023] According to the present invention, an effective inspection target region is a region obtained by excluding a region where saturation has occurred from an inspection target region, which is a region of an image of a light-emitting element, and the luminance or color tone of the light-emitting element is calculated based on the pixel values of this effective inspection target region. Therefore, it is possible to accurately inspect the quality or light-emitting characteristics of the light-emitting element with a simple device configuration.

Brief Description of the Drawings

[0024] [Figure 1] It is a schematic diagram showing a schematic configuration of an inspection system according to an embodiment of the present invention. [Figure 2] It is a block diagram showing a schematic configuration of an inspection system according to an embodiment of the present invention. [Figure 3] It is a flowchart showing an inspection method according to an embodiment of the present invention. [Figure 4] It is a flowchart showing setting processing in an inspection method according to an embodiment of the present invention. [Figure 5] It is a schematic diagram exemplifying a setting screen. [Figure 6] It is a schematic diagram showing a partially enlarged setting screen (in a state where the inspection target area tab is open). [Figure 7] It is a schematic diagram showing a partially enlarged setting screen (in a state where the non-target area tab is open). [Figure 8] It is a schematic diagram showing a partially enlarged setting screen (in a state where the brightness setting tab is open). [Figure 9] It is a schematic diagram for explaining a setting operation of an area used for determining the position of a light-emitting element. [Figure 10] It is a schematic diagram showing an example of display of a position determination result of a light-emitting element. [Figure 11] It is a schematic diagram exemplifying a setting screen after test execution. [Figure 12] It is a schematic diagram exemplifying an inspection screen. [Figure 13] It is a schematic diagram exemplifying an inspection screen after inspection execution. [Figure 14] It is a graph showing RGB and HLS values in an image of a blue LED. [Figure 15] It is a graph showing RGB and HLS values in an image of a red LED. [Figure 16] It is a graph showing RGB and HLS values in an image of a white LED. [Figure 17] It is a graph showing RGB and HLS values in an image of a green LED. [Figure 18] It is a schematic diagram for explaining a modification example of an embodiment of the present invention. [Modes for carrying out the invention]

[0025] The following describes an inspection system, inspection method, and inspection program according to embodiments of the present invention with reference to the drawings. However, the present invention is not limited to these embodiments. Furthermore, in each drawing, the same parts are denoted by the same reference numerals.

[0026] The drawings referenced in the following description merely provide a schematic representation of the shape, size, and positional relationships to the extent necessary to understand the content of the present invention. That is, the present invention is not limited to the shapes, sizes, and positional relationships exemplified in each drawing. Furthermore, there may be differences in the dimensional relationships and proportions between drawings.

[0027] (Configuration of the inspection system) Figure 1 is a schematic diagram showing the general configuration of an inspection system according to an embodiment of the present invention. Figure 2 is a block diagram showing the general configuration of the same inspection system. As shown in Figure 1, the inspection system 1 according to this embodiment includes a camera 3 installed to capture images of the equipment 2 to be inspected, and an information processing device 4 that processes the image signal output from the camera 3. The information processing device 4 is provided with a display device 41 and an input device 42.

[0028] Device 2 is equipped with one or more (multiple in Figure 1) light-emitting elements 21, and camera 3 is positioned so that its field of view is directed toward the surface on which the light-emitting elements 21 are mounted (the light-emitting surface).

[0029] Camera 3 is a camera equipped with a solid-state image sensor such as a CMOS or CCD, capable of capturing color images. Camera 3 only needs to have the ability to capture still images, and may also have a continuous image capture function. The specifications of Camera 3 are not particularly limited, but it is preferable to use a color camera that can be connected to a computer via USB (Universal Serial Bus: USB 2.0 or USB 3.0) or GigE (Gigabit Ethernet). Specific examples include the uEye camera series from IDS (Imaging Development Systems GmbH) or the "USB3 Vision Camera" series or "GigE Vision Camera" series from Toshiba Terry Corporation.

[0030] The information processing device 4 is a device that performs various processes for inspecting the light-emitting element 21 based on the image signal output from the camera 3. The information processing device 4 can be configured using general-purpose equipment such as a personal computer (PC), notebook PC, or tablet terminal.

[0031] The display device 41 is, for example, a liquid crystal display or an organic EL display. The input device 42 is an input device such as a keyboard, a mouse or other pointing device, or a touch panel provided on the screen of the display device 41, and accepts input operations for the information processing device 4.

[0032] Furthermore, the inspection system 1 may be provided with a stand 5 to support the equipment 2, and a slide rail 6 to slidably hold the stand 5. By providing the slide rail 6, the equipment 2 supported by the stand 5 can be moved in parallel within a plane perpendicular to the optical axis of the camera 3, while keeping the light-emitting surface facing the field of view of the camera 3. In addition, the inspection system 1 may be further provided with a drive mechanism 7 to move the stand 5 along the slide rail 6 at a predetermined speed.

[0033] As shown in Figure 2, the information processing device 4 comprises an external interface 410, a storage unit 420, and a processor 430. The external interface 410 is an interface that connects the information processing device 4 to an external device and transmits and receives signals between the information processing device 4 and the external device. Examples of external devices connected to the information processing device 4 include the camera 3 and the drive mechanism 7.

[0034] The storage unit 420 is configured using a computer-readable storage medium such as semiconductor memory like ROM or RAM, or a hard disk. The storage unit 420 includes a program storage unit 421, an image data storage unit 422, and a setting value storage unit 423.

[0035] The program storage unit 421 stores the operating system program and driver program, as well as application programs that execute various functions in the inspection system 1, and various parameters used during the execution of these programs. Specifically, the program storage unit 421 stores inspection programs for measuring the brightness or color of the light-emitting element 21 based on the image signal output from the camera 3, and for inspecting the quality or light emission characteristics of the light-emitting element 21.

[0036] The image data storage unit 422 stores the image data generated by the image generation unit 434 (described later) based on the image signal output from the camera 3. The setting value storage unit 423 stores setting values ​​for the camera 3, such as exposure time, gain, and frame rate, as well as extraction conditions such as thresholds used when extracting regions from images showing the light-emitting elements 21, and judgment criteria such as thresholds used to determine the light-emitting elements 21.

[0037] The processor 430 is configured, for example, using a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and by reading various programs stored in the program storage unit 421, it comprehensively controls each part of the inspection system 1 and performs various calculations for inspecting the light-emitting element 21. In detail, the functional units realized by the processor 430 include an imaging control unit 431, a display control unit 432, an image processing unit 433, a condition setting unit 437, a determination unit 438, and a continuous inspection control unit 439.

[0038] The imaging control unit 431 controls the imaging operation of the camera 3. Specifically, the imaging control unit 431 causes the camera 3 to perform imaging based on various setting values ​​stored in the setting value storage unit 423, and also controls the capture operation (recording image data as a still image).

[0039] The display control unit 432 controls the display on the display device 41. Specifically, while the inspection system 2 is operating, the display control unit 432 generates a predetermined screen, such as an inspection screen or a settings screen, and displays it on the display device 41.

[0040] The image processing unit 433 performs various image processing operations to inspect the light-emitting element 21 based on the image signal output from the camera 3. In detail, the image processing unit 433 includes an image generation unit 434, a region extraction unit 435, and a measurement unit 436.

[0041] The image generation unit 434 generates display image data (RGB data) by applying image processing such as demosaicing to the image signal (RAW data) output from the camera 3.

[0042] The region extraction unit 435 extracts the image of each light-emitting element 21 as an inspection target region from the image of the light-emitting elements 21 represented by the image data generated by the image generation unit 434, and further extracts the areas where saturation occurs from inside each inspection target region as an exclusion region. The region extraction unit 435 also extracts the contour and centroid position of the image of the light-emitting elements 21 (i.e., the inspection target region) from the image.

[0043] The measurement unit 436 acquires the area obtained by removing the non-target area from the inspection target area extracted by the area extraction unit 435 as the effective inspection target area, and calculates the brightness or color of each light-emitting element 21 based on the pixel values ​​of this effective inspection target area.

[0044] The condition setting unit 437 sets extraction conditions such as thresholds used for region extraction by the region extraction unit 435, and determination criteria such as thresholds used for determining the light-emitting element 21.

[0045] The determination unit 438 determines the quality or luminescence characteristics of each light-emitting element 21 by comparing the brightness or color of the light-emitting element 21 calculated by the measurement unit 436 with a preset threshold. The determination unit 438 also determines the appropriateness of the number, position, and area of ​​the light-emitting elements 21 based on the outline of the image of the light-emitting element 21 extracted by the region extraction unit 435.

[0046] The continuous inspection control unit 439 controls the operation of the drive mechanism 7 when performing continuous inspection of the device 2, and also controls the timing of the capture in synchronization with the movement of the device 2 by the drive mechanism 7.

[0047] (Testing method) Figure 3 is a flowchart showing the inspection method according to this embodiment. First, the user places the device with the test light-emitting element installed on stand 5 and performs various settings (step S10).

[0048] Figure 4 is a flowchart showing the setting process in step S10. Figure 5 is a schematic diagram illustrating the setting screen for various items displayed on the display device 41.

[0049] When the information processing device 4 is started up, it first sets the imaging conditions for the camera 3 (step S101). Specifically, the information processing device 4 displays a predetermined camera setting screen on the display device 41 and starts imaging the test light-emitting element with the camera 3. As a result, a real-time image (so-called live view) based on the image signal output from the camera 3 is displayed in a predetermined area on the camera setting screen.

[0050] The camera settings screen includes, for example, sliders for adjusting exposure time and gain. The imaging control unit 431 changes the exposure time and gain of the camera 3 in response to operations performed on these sliders. The user can set the desired settings by operating the sliders while viewing the image displayed on the camera settings screen. When the camera settings screen is closed, the condition setting unit 437 saves the exposure time and gain as imaging conditions according to the slider positions at that time.

[0051] Next, the information processing device 4 displays the setting screen M1 shown in Figure 5 on the display device 41 (step S102). The setting screen M1 is provided with an image display field m20 and radio buttons m11 to m13 for selecting the image to be displayed in this image display field m20. When the radio button m11 for "camera image" is selected, the image display field m20 displays a real-time image based on the image signal output from the camera 3. In this case, when an operation (e.g., a click) is performed on the capture button m14, the image displayed in the image display field m20 is captured. As a result, the display in the image display field m20 switches to a still image, and the radio button m12 for "input image" is selected. Also, when the radio button m13 for "reference image" is selected, an image (still image) that has been previously stored as a reference image in the storage unit 420 is read out and displayed in the image display field m20. Here, a reference image is an image obtained by imaging a light-emitting element having appropriate brightness and color, and registered as a reference image.

[0052] Immediately after the settings screen M1 is displayed, as shown in Figure 5, the "Camera Image" radio button m11 is selected by default, and the image display area m20 displays a real-time image based on the image signal output from camera 3.

[0053] The information processing device 4 registers the type of light-emitting element based on the information entered in the type input field m10 provided on the setting screen M1 (step S103). Figure 5 shows the state where red LED is registered as the type. Various conditions and criteria for inspection can be set for each type of light-emitting element.

[0054] Next, the information processing device 4 performs image capture in response to the operation performed on the capture button m14 (step S104). As a result, the image data of the image displayed in the image display field m20 is recorded, and at the same time, the display in the image display field m20 switches to a still image, and the "Input Image" radio button m12 is selected.

[0055] Next, the information processing device 4 converts the values ​​of each pixel constituting the image displayed in the image display field m20 from values ​​in the RGB color space to values ​​in the HLS color space (step S105). The color space conversion can be performed using a known formula.

[0056] Next, the information processing device 4 sets the extraction conditions for extracting the inspection target area and the non-inspection area from the image (step S106). The inspection target area is the area of ​​pixels that are to be calculated during inspection, and is set to the area in which the image m21 of the light-emitting element is captured. The non-inspection area is the area of ​​pixels that are inside the inspection target area but are excluded from the calculation during inspection, and is set to the area in the image m21 of the light-emitting element where saturation occurs. Here, in the image m21 of each light-emitting element, the peripheral part m22 is captured in its original color (red in the case of a red LED), but the central part m23 is saturated and appears whitish.

[0057] The settings screen M1 includes a tab (inspection area tab m25) for setting the inspection area m27 in the image display area m20, a tab (inspection target area tab m30) for setting the extraction conditions for the area to be inspected, and a tab (excluded area tab m40) for setting the extraction conditions for the area to be excluded. Figure 5 shows the state when the inspection area tab m25 is open.

[0058] The inspection area m27 can be set, for example, by clicking the area selection button m26 located in the inspection area tab m25, and then dragging the pointer over the image displayed in the image display area m20. By setting the inspection area m27, during the actual inspection, only the images of light-emitting elements located within the inspection area m27 can be inspected. If the inspection area m27 is not set, all images of light-emitting elements captured in the image displayed in the image display area m20 can be inspected.

[0059] Figure 6 is a schematic diagram showing a magnified portion of the settings screen M1, indicating that the inspection target area tab m30 is open. The inspection target area tab m30 has checkboxes m31 and m32 for selecting the type of value to be set as the extraction condition for the inspection target area. The types of values ​​that can be used are the L value (brightness) and H value (hue) in the HLS color space, and the selection of the L value is mandatory. For example, checkbox m31 may be checked by default and cannot be unchecked, or a warning may be displayed if checkbox m31 is unchecked. Alternatively, checkbox m31 may not be provided at all.

[0060] Furthermore, the inspection target area tab m30 is provided with an L-value setting unit m33 for setting the threshold for the L-value and an H-value setting unit m34 for setting the threshold for the H-value. The L-value setting unit m33 and the H-value setting unit m34 may be made operable only when checkboxes m31 and m32 are checked, respectively.

[0061] The L-value setting unit m33 includes two sliders for adjusting the lower and upper thresholds of the L-value. Of course, the lower and upper thresholds of the L-value can also be set by directly inputting numerical values ​​into the numerical input unit m35. Alternatively, only the lower limit may be set as the threshold for the L-value, in which case only one slider is needed.

[0062] The H-value setting unit m34 includes two sliders for setting lower and upper thresholds for the H-value. Of course, these thresholds can also be set by directly inputting numerical values ​​into the numerical input unit m36.

[0063] Near the H value setting section m34, radio buttons m37 and m38 are provided for selecting the direction of hue selection. When the "Inside" radio button m37 is selected, the range of L values ​​between the lower threshold and the upper threshold is set as the extraction condition. On the other hand, when the "Outside" radio button m38 is selected, the range of L values ​​between the lower threshold and the upper threshold is set as the extraction condition.

[0064] The region extraction unit 435 extracts pixels from the pixels that make up the image displayed in the image display field m20 that satisfy the conditions set by the L value setting unit m33. Furthermore, if the checkbox m32 is checked, the region extraction unit 435 extracts pixels that satisfy both the conditions set by the L value setting unit m33 and the H value setting unit m34.

[0065] The display control unit 432 overlays a predetermined color onto the pixel region m39 extracted by the region extraction unit 435 and displays it. The user can set appropriate conditions for extracting the region to be inspected by operating the L value setting unit m33 and the H value setting unit m34 while checking the colored region m39 in the image display area m20.

[0066] Figure 7 is a schematic diagram showing a magnified portion of the settings screen M1, with the excluded area tab m40 open. The excluded area tab m40 has checkboxes m41 and m42 for selecting the type of value to be used as the extraction condition for the excluded area. The value types that can be used are the L value (brightness) and H value (hue) in the HLS color space, and the selection of the L value is mandatory. For example, checkbox m41 may be checked by default and cannot be unchecked, or a warning may be displayed if checkbox m41 is unchecked. Alternatively, checkbox m41 may not be provided at all.

[0067] Furthermore, the excluded area tab m40 is provided with an L value setting unit m43 for setting the threshold for the L value and an H value setting unit m44 for setting the threshold for the H value. The L value setting unit m43 and the H value setting unit m44 may be made operable only when checkboxes m41 and m42 are checked, respectively.

[0068] The L-value setting unit m43 includes two sliders for adjusting the lower and upper thresholds of the L-value. Of course, the lower and upper thresholds of the L-value can also be set by directly inputting numerical values ​​into the numerical input unit m45. Alternatively, only the lower limit may be set as the threshold for the L-value, in which case only one slider is needed.

[0069] The H-value setting unit m44 includes two sliders for setting lower and upper thresholds for the H-value. Of course, these thresholds can also be set by directly inputting numerical values ​​into the numerical input unit m46. Near the H-value setting unit m44, selection buttons m47 and m48 are provided for selecting the direction of hue selection. The functions of these selection buttons m47 and m48 are the same as those of selection buttons m37 and m38 in Figure 6.

[0070] The region extraction unit 435 extracts pixels from the pixels that make up the image displayed in the image display area m20 that satisfy the conditions set by the L value setting unit m43. Furthermore, if the checkbox m42 is checked, the region extraction unit 435 extracts pixels that satisfy both the conditions set by the L value setting unit m43 and the H value setting unit m44.

[0071] The display control unit 432 overlays a predetermined color onto the pixel region m49 extracted by the region extraction unit 435 and displays it. The user can set appropriate conditions for regions to be extracted as exclusion areas by operating the L value setting unit m43 and H value setting unit m44 while checking the colored region m49 in the image display area m20.

[0072] The condition setting unit 437 sets the threshold value of the L value set by the L value setting unit m33 (and the threshold value of the H value set by the H value setting unit m34 if checkbox m32 is checked) as the extraction condition for the area to be inspected, and sets the threshold value of the L value set by the L value setting unit m43 (and the threshold value of the H value adjusted by the H value setting unit m44 if checkbox m42 is checked) as the extraction condition for the area to be excluded.

[0073] Next, the information processing device 4 sets a determination criterion for the image of the light-emitting element extracted from the image (step S107). Referring again to Figure 5, the settings screen M1 has several tabs m50 to m53 for setting judgment criteria. In the brightness setting tab m50, the criteria for judging the brightness or color of the light-emitting element are set. In the number setting tab m51, the criteria for judging the number of light-emitting elements are set. In the position setting tab m52, the criteria for judging the position of the light-emitting element are set. In the area setting tab m53, the criteria for judging the area of ​​the light-emitting region are set.

[0074] Figure 8 is a schematic diagram showing a magnified portion of the settings screen M1, indicating that the brightness setting tab m50 is open. The brightness setting tab m50 includes a group of checkboxes m54 for selecting the type of value used to determine the brightness or color of the light-emitting element. The selectable value types are the R (red), G (green), and B (blue) values ​​in the RGB color space, as well as the H (hue), L (brightness), and S (saturation) values ​​in the HLS color space.

[0075] Multiple types of values ​​can be selected for the judgment. When selecting multiple types of values, you can choose "AND" or "OR" as the judgment condition in checkbox m55.

[0076] Furthermore, the brightness setting tab m50 includes a pull-down menu m56 for selecting the type of value, and a threshold setting section m57 for setting the threshold value of the value selected in the pull-down menu m56. In Figure 8, the threshold setting section m57 includes two sliders for adjusting the lower and upper threshold limits. Of course, the lower and upper threshold limits can also be set by directly inputting numerical values ​​into the numerical input section m58.

[0077] The region extraction unit 435 extracts pixels from the pixels that make up the image displayed in the image display field m20 that are selected in the checkbox group m54 and satisfy the conditions set by the threshold setting unit m57. The display control unit 432 overlays a predetermined color onto the region m59 of pixels extracted by the region extraction unit 435 and displays it. The user can set criteria for determining the quality of the brightness and color of the light-emitting element by operating the threshold setting unit m57 while checking the colored region m59 in the image display field m20.

[0078] In the quantity setting tab m51, you can set the number of images of light-emitting elements (inspection target area) that should be present within the inspection area m27 shown in Figure 5. For example, in the image of the test light-emitting elements shown in Figure 5, four light-emitting elements are visible within the inspection area m27. Therefore, if this state is used as the basis, "4" is set as the criterion for determining the number. In this case, during the actual inspection, if there are four images of light-emitting elements within the inspection area m27, it will be judged as "OK" (good), and if there are three or fewer, or five or more, it will be judged as "NG" (bad).

[0079] In the position setting tab m52, you can set a circular or rectangular area used for determining the position of each light-emitting element. Figure 9 is a schematic diagram illustrating the operation of setting the area used for position determination. To set the area, first open the position setting tab m52 and click the operation button to select the shape of the area (circular or rectangular).

[0080] Next, on the image displayed in the image display area m20, position pointer m60 approximately at the center of the image of the light-emitting element (see Figure 9(a)), and then drag pointer m60 outwards from there (see Figure 9(b)). If a circle is selected as the shape of the region, a circular position reference m63 will be drawn with a radius from the starting position m61 to the drop position m62 (see Figure 9(a) and (b)). If a rectangle is selected as the shape of the region, a rectangular position reference will be drawn with the drop position as its vertex.

[0081] Figure 10 is a schematic diagram showing an example of the display of the position determination result of a light-emitting element. When position determination is performed during actual inspection, the position determination is "OK" when the contour m64 of the image of the light-emitting element (inspection target area) is located inside the position reference m63 (see Figure 10(a)). On the other hand, the position determination is "NG" when a part of the contour m64 overlaps with or intersects with the position reference m63 (see Figure 10(b)).

[0082] In the Area Setting tab m53, you can set lower and upper threshold values ​​for determining the area (number of pixels) of each inspection target area. Users can modify the pixel count settings after reviewing the test results described later. When area determination is performed during actual inspection, the number of pixels of each inspection target area extracted from the image is calculated. Inspection target areas whose pixel count falls within the set threshold range are judged as "OK," while inspection target areas whose pixel count falls outside the threshold range are judged as "NG."

[0083] Referring again to Figure 4, the information processing device 4 performs a test on the still image displayed in the image display field m20 (i.e., the image showing the test light-emitting element) in response to the operation of the test execution button m70 (step S108). The user may also recapture a new image for the test. In this case, the user first selects the "camera image" radio button m11 and then operates the test execution button m70. As a result, the test is performed on the still image captured at the time the test execution button m70 is operated.

[0084] When the test begins, the region extraction unit 435 extracts the areas to be inspected and the areas not to be inspected from the image according to the extraction conditions set in step S106, and sets the area obtained by subtracting the areas not to be inspected from the areas to be inspected as the effective inspection area. The region extraction unit 435 also extracts the contour and centroid position of each inspection area. For contour extraction, a general method can be used in which the image is binarized using the lower threshold used for extracting the inspection areas, and points where the pixel values ​​change by raster scanning are searched for.

[0085] The measurement unit 436 counts the number of contours of the extracted inspection target area and displays the counted number near each contour. The measurement unit 436 also calculates the R, B, and G values ​​in the RGB color space, as well as the H, L, and S values ​​in the HLS color space, for each effective inspection target area and displays them in the result display field m71. Furthermore, the measurement unit 436 calculates the area (number of pixels) of each inspection target area and displays it in the result display field m71.

[0086] By checking the values ​​displayed in the results display area m71 after the test execution, users can determine whether the extraction conditions and judgment criteria are appropriate and modify the settings as needed.

[0087] Next, the information processing device 4 registers the image obtained from the test as the reference image in response to the operation performed on the reference image registration button m72 (step S109). The information processing device 4 closes the settings screen M1 in response to the operation performed on the "close" button m73, and saves the various conditions set on the settings screen M1 as conditions for the registered product type (in Figure 11, "red LED"). After that, the process returns to the main routine.

[0088] Referring again to Figure 3, the user places the device on which the light-emitting element to be inspected is mounted on stand 5 and performs the actual inspection. Figure 12 is a schematic diagram illustrating the inspection screen displayed on the display device 41.

[0089] The information processing device 4 displays the inspection screen M3 shown in Figure 12 on the display device 41 (step S11). The inspection screen M3 is provided with an image display area m80 in which an image of the light-emitting element to be inspected is displayed. Immediately after the inspection screen M3 is displayed, a real-time image based on the image signal output from the camera 3 is displayed in the image display area m80.

[0090] Next, the information processing device 4 displays a selection of pre-registered varieties in response to the operation performed on the variety selection button m81, and selects the variety of the light-emitting element to be inspected in response to the operation performed on that selection (step S12). As a result, the extraction conditions and judgment criteria set for the selected variety are read from the setting value storage unit 423.

[0091] The information processing device 4 performs image capture in response to the operation performed on the inspection execution button m83 (step S13). As a result, the image data of the image displayed in the image display field m80 is recorded, and the display in the image display field m80 switches to a still image.

[0092] Next, the information processing device 4 converts the values ​​of each pixel constituting the image displayed in the image display field m80 from values ​​in the RGB color space to values ​​in the HLS color space (step S14).

[0093] Next, the information processing device 4 extracts the region of the image of the light-emitting element to be inspected from the image displayed in the image display field m80 (step S15). That is, the information processing device 4 extracts the region of the image of the light-emitting element as the inspection target region based on the L value threshold (or L value and H value threshold) set as the extraction condition for the inspection target region. The information processing device 4 also extracts the region where saturation is occurring from inside the inspection target region as the exclusion region based on the L value threshold (or L value and H value threshold) set as the extraction condition for the exclusion region. Then, the region obtained by subtracting the exclusion region from the inspection target region is set as the effective inspection target region.

[0094] Next, the information processing device 4 extracts the contour and centroid of each extracted inspection target area (step S16). Next, the information processing device 4 makes a determination of the light-emitting element according to the set determination criteria (step S17). Specifically, for each pixel in the effective inspection area, it calculates the average value of various values ​​(R value, G value, B value, H value, L value, S value) and determines whether the calculated average value falls within the range of numerical values ​​set as the determination criteria for brightness or color.

[0095] Furthermore, if an inspection area m27 (see Figure 5) is set, the information processing device 4 counts the number of contours of the area to be inspected within the inspection area m27 (i.e., the number of light-emitting elements) and determines whether the counted number matches the set value.

[0096] Furthermore, the information processing device 4 determines whether the contour of each inspection target area is located inside the area of ​​the position reference m63 (see Figures 9 and 10). Furthermore, the information processing device 4 calculates the area (number of pixels) of each area to be inspected and determines whether the calculated area falls within the range of the threshold set as the area determination criterion.

[0097] Next, the information processing device 4 displays the judgment result on the display device 41 (step S18). Figure 13 is a schematic diagram illustrating the inspection screen M3 after the inspection has been performed. On the inspection screen M3, the judgment criteria display field m82 displays the judgment criteria set for the selected variety.

[0098] The image displayed in the image display area m80 shows the outline and centroid of the inspection area superimposed with the number of the extracted inspection area. The results display area m84 displays the average value of various values ​​calculated based on the pixels of the effective inspection area, as well as the area of ​​the inspection area, for each numbered inspection area.

[0099] If all of these inspection areas meet the criteria set for the selected variety, the word "OK" will be displayed in the result display field m85. On the other hand, if any of the inspection areas do not meet the criteria, the word "NG" will be displayed in the result display field m85, and the result for the inspection area that does not meet the criteria will be marked in the result display field m84. This completes the inspection process.

[0100] As explained above, according to this embodiment, the effective inspection area is defined as the area obtained by excluding the area where saturation occurs from the inspection target area, which is the image area of ​​the light-emitting element. The brightness or color of the light-emitting element is calculated based on the pixel values ​​of this effective inspection target area, making it possible to accurately inspect the quality or luminescence characteristics of the light-emitting element.

[0101] Figures 14 to 17 are graphs showing the RGB and HLS values ​​in images of LEDs of each color. In Figures 14 to 17, the horizontal axis represents the coordinates on a straight line passing through the center of the LED. As shown in Figures 14 to 17, the R, G, and B values ​​(range: 0 to 255) are particularly high in the center and surrounding areas of an image of a light-emitting LED compared to the periphery. Areas where these R, G, and B values ​​exceed approximately 80% of the upper limit of 255 are saturated (blown out), and the original brightness and color information is lost. Therefore, if the judgment is made including such saturated areas, it is not possible to accurately determine the brightness and color of the LED.

[0102] In contrast, in this embodiment, the region where saturation occurs (the region not being inspected) is excluded from the LED image region (the region to be inspected), and the determination is made based only on the region that reflects the actual brightness and color of the LED. Therefore, accurate determination results can be obtained with a simple device configuration consisting of a camera 3 and an information processing device 4.

[0103] However, as shown in Figures 14 to 17, the region where saturation occurs varies depending on the color of the LED. Therefore, in images represented by values ​​in the RGB color space, saturation must be defined for each color of LED. In other words, it becomes necessary to set threshold values ​​for R, G, and B for each color of LED.

[0104] In contrast, in this embodiment, the pixel values ​​expressed in the RGB color space are converted to values ​​in the HLS color space, and the excluded areas are extracted based on at least the L value in the HLS color space. By using at least the L value, saturation can be uniquely defined regardless of the color of the LED, making it possible to accurately extract the saturation areas in the center and surrounding areas of the LED image.

[0105] Furthermore, according to this embodiment, the light-emitting elements can be inspected based on a single still image obtained by imaging the object to be inspected. Therefore, since it is not necessary to perform multiple imaging or processing on multiple images, it is possible to suppress an increase in inspection time and reduce the computational load on the information processing device 4.

[0106] Furthermore, according to this embodiment, when setting the extraction conditions for the inspection target area and the exclusion area, or when setting the judgment criteria for brightness and color, a predetermined color is superimposed and displayed on the area of ​​pixels in the image that satisfies the conditions. As a result, the user can easily set the desired conditions by operating the slider while checking the colored area.

[0107] Furthermore, according to this embodiment, the inspection system 1 can be configured with a simple hardware configuration consisting of a camera 3 and a general-purpose information processing device 4. Therefore, even users who previously performed visual inspections can easily introduce the inspection system 1 at a relatively low cost.

[0108] (modified version) Figure 18 is a schematic diagram illustrating a modified embodiment of the present invention. As shown in Figure 18, if the entire device 2A to be inspected does not fit within the field of view of the camera 3, the inspection can be performed by moving the device 2A and imaging it in multiple stages.

[0109] Specifically, the information processing device 4 (continuous inspection control unit 439) controls the drive mechanism 7 to move the stand 5 supporting the device 2A along the slide rail 6 at a predetermined speed, and, in synchronization with the movement of the device 2A, causes the camera 3 to perform imaging at a predetermined interval. As a result, the information processing device 4 sequentially acquires image signals of images (still images) in which the light-emitting surface of the device 2A (the surface on which the light-emitting element 21A is mounted) is partially captured, and performs region extraction and determination processing for each image (see steps S13 to S18 in Figure 3). The imaging period and number of images taken by the camera 3 can be appropriately set according to the size of the light-emitting surface of the device 2A relative to the field of view of the camera 3.

[0110] The present invention is not limited to the embodiments and modifications described above, and can be implemented in various other forms without departing from the spirit of the invention. For example, it may be formed by excluding some of the components shown in the embodiments and modifications, or by appropriately combining the components shown in the embodiments and modifications. [Explanation of Symbols]

[0111] 1…Inspection system, 2·2A…Equipment, 3…Camera, 4…Information processing device, 5…Stand, 6…Slide rail, 7…Drive mechanism, 21·21A…Light-emitting element, 42…Input device, 42…Display device, 410…External interface, 420…Storage unit, 421…Program storage unit, 422…Image data storage unit, 423…Setting value storage unit, 430…Processor, 431…Imaging control unit, 432…Display control unit, 433…Image processing unit, 434…Image generation unit, 435…Region extraction unit, 436…Measurement unit, 437…Condition setting unit, 438…Determination unit, 439…Continuous inspection control unit

Claims

1. A camera is installed to capture images of a light-emitting element while it is lit, and captures the light-emitting element and outputs an image signal. An image generation unit generates image data representing an image of the light-emitting element based on the image signal output from the camera, A region extraction unit extracts a two-dimensional image of the light-emitting element in the aforementioned image as the inspection target region, and further extracts a two-dimensional region where saturation occurs from inside the inspection target region as the exclusion region. A measuring unit calculates the brightness or color of the light-emitting element based on the pixel values ​​of the effective inspection target area, which is a two-dimensional area obtained by excluding the non-inspection area from the inspection target area. An inspection system equipped with the following features.

2. The image generated by the image generation unit is represented by values ​​in the RGB color space, The inspection system according to claim 1, wherein the region extraction unit converts the values ​​of the pixels constituting the image into values ​​in the HLS color space, and extracts the excluded region based on at least the L value in the HLS color space.

3. The inspection system according to claim 1 or 2, wherein the measuring unit calculates the average value of the R, G, or B values ​​in the RGB color space, or the H, L, or S values ​​in the HLS color space, of the pixels in the effective inspection target area.

4. The inspection system according to claim 3, further comprising a determination unit that determines the quality or light emission characteristics of the light-emitting element by comparing the average value with a preset threshold.

5. The camera captures images of multiple light-emitting elements, The region extraction unit further extracts the contour of the inspection target region extracted for each of the plurality of light-emitting elements, The inspection system according to any one of claims 1 to 4, further comprising a determination unit that counts the number of contours within a preset inspection area for the image and determines whether the counted number matches a preset value.

6. The region extraction unit further extracts the contour of the region to be inspected, The inspection system according to any one of claims 1 to 4, further comprising a determination unit for determining whether or not the contour is located inside a predetermined region with respect to the image.

7. The inspection system according to any one of claims 1 to 4, further comprising a determination unit that measures the area of ​​the area to be inspected and determines whether the measured area is within a predetermined range of numerical values.

8. The image generation unit generates a test image based on an image signal generated by capturing a test light-emitting element with the camera, A display unit that displays a settings screen including the aforementioned test image, An input unit that accepts operations for the inspection system, A display control unit that controls the display on the settings screen, Furthermore, The display control unit, At least one slider for adjusting the range of pixel values, wherein the slider, which can be operated by the input unit, is displayed on the setting screen. A predetermined color is superimposed and displayed on a region of pixels in the test image whose values ​​are within the range adjusted by at least one of the sliders. The inspection system according to any one of claims 1 to 7, further comprising a condition setting unit for setting the range adjusted by the at least one slider as an extraction condition used for extracting the inspection target area or the non-inspection area by the area extraction unit.

9. The image generation unit generates a test image based on an image signal generated by capturing a test light-emitting element with the camera, A display unit that displays a settings screen including the aforementioned test image, An input unit that accepts operations for the inspection system, A display control unit that controls the display on the settings screen, Furthermore, The display control unit, At least one slider for adjusting the range of pixel values, wherein the slider, which can be operated by the input unit, is displayed on the setting screen. A predetermined color is superimposed and displayed on a region of pixels in the test image whose values ​​are within the range adjusted by at least one of the sliders. The inspection system according to any one of claims 1 to 7, further comprising a condition setting unit that sets the range adjusted by the at least one slider as a judgment criterion used to determine the quality or light emission characteristics of the light-emitting element based on the brightness or color of the light-emitting element calculated by the measurement unit.

10. The inspection system is a system for inspecting equipment on which one or more of the light-emitting elements are mounted, A drive mechanism for moving the device in a predetermined direction at a predetermined speed within a plane perpendicular to the optical axis of the camera, A control unit that causes the camera to image the device at a predetermined interval in synchronization with the movement of the device by the drive mechanism, The inspection system according to any one of claims 1 to 9, further comprising:

11. A camera positioned to capture images of the illuminated light-emitting element is used to capture images of the light-emitting element, Based on the image signal output from the camera, image data representing the image of the light-emitting element is generated. The two-dimensional image of the light-emitting element in the aforementioned image is extracted as the inspection target area, and further, the two-dimensional area where saturation occurs is extracted from inside the inspection target area as the exclusion area. An inspection method comprising calculating the brightness or color of the light-emitting element based on the pixel values ​​of the effective inspection target area, which is a two-dimensional area obtained by excluding the non-inspection area from the inspection target area.

12. A camera positioned to capture images of the illuminated light-emitting element is used to capture images of the light-emitting element, Based on the image signal output from the camera, image data representing the image of the light-emitting element is generated. The two-dimensional image of the light-emitting element in the aforementioned image is extracted as the inspection target area, and further, the two-dimensional area where saturation occurs is extracted from inside the inspection target area as the exclusion area. An inspection program that causes a computer to calculate the brightness or color of the light-emitting element based on the pixel values ​​of the effective inspection target area, which is a two-dimensional area obtained by excluding the non-inspection area from the inspection target area.

Images