Image processing method and image processing system

Abstract

An image processing method and image processing system according to an embodiment of the present invention comprises: a step of calculating a difference image between a reference image and a source image; a step of grouping pixels of the difference image according to gray level; a step of binarizing the grouped pixels to generate a mask image for each pixel group; a step of generating a partial reference image in the reference image with a region corresponding to the mask image, and generating a partial source image in the source image with a region corresponding to the mask image; a step of assigning the histogram of the partial source image to the histogram of the partial reference image; and a step of combining all the partial source images generated per pixel group to calculate a combined image.

Description

Technology Field

[0001] The present invention relates to an image processing method and an image processing system. Background Technology

[0002] Manufacturing processes in various industrial settings are automated. Products produced through these processes are inspected for defects via visual inspection. However, this type of visual inspection presents a problem in that results are inconsistent depending on the inspector's mental and physical state, consequently leading to reduced product reliability and productivity. Therefore, to overcome the limitations of visual inspection, machine vision technology is being developed to automate defect inspection using mechanical devices.

[0003] Such machine vision systems can detect product defects by using product images. The aforementioned machine vision technology identifies product defects by referencing a reference image, but errors can occur even due to minute changes in brightness of the image of the product being inspected and input into the inspection system. Therefore, it is desirable for the image of the product being inspected and input into the machine vision system to have the same brightness as the reference image.

[0004] However, product images are acquired under lighting conditions, and even under the same lighting, it is difficult to maintain a constant amount of light irradiated onto the product due to factors such as changes in the surrounding environment. Consequently, even images of the same product may exhibit subtle differences in brightness; therefore, image processing technology is required to ensure that product images possess a consistent brightness.

[0005] Histogram specification can be used as an image processing method. Histogram specification is a method that adjusts the gray level distribution of a source image to be identical to the gray level distribution of a reference image.

[0006] However, unlike continuous signals such as analog images, errors may occur due to quantization errors when performing histogram specification on digital images.

[0007] Reducing the size or resolution of an image can reduce errors in histogram specification, but reducing the size or resolution of the image may result in the loss of information contained in the image.

[0008] Therefore, a technology is needed to improve the accuracy of histogram designation by reducing quantization error during the histogram designation process without such information loss. The problem to be solved

[0009] The objective of the present invention is to provide an image processing method and an image processing system that can maintain the brightness of an image constant using histogram designation and reduce quantization errors that affect the accuracy of the histogram designation.

[0010] The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0011] An image processing method according to one embodiment of the present invention comprises: a step of calculating a difference image between a reference image and a source image; a step of grouping pixels of the difference image according to gray level; a step of binarizing the grouped pixels to generate a mask image for each pixel group; a step of generating a partial reference image in the reference image with a region corresponding to the mask image, and generating a partial source image in the source image with a region corresponding to the mask image; a step of assigning the histogram of the partial source image to the histogram of the partial reference image; and a step of combining all the partial source images generated per pixel group to calculate a combined image.

[0012] Specifically, in the step of generating a partial image, the partial reference image is generated through an AND operation between the reference image and the mask image, and the partial source image can be generated through an AND operation between the source image and the mask image.

[0013] Specifically, the step of specifying the histogram may include: calculating the histogram of the partial reference image and the histogram of the partial source image; calculating the reference cumulative distribution function for the histogram of the partial reference image and the source cumulative distribution function for the histogram of the partial source image; generating a specified function by comparing the reference cumulative distribution function and the source cumulative distribution function; and converting the gray level of the pixels of the partial source image based on the specified function.

[0014] Specifically, it may include a step of performing a vision inspection based on the above composite image.

[0015] In addition, an image processing system according to one embodiment of the present invention comprises: a difference image calculation unit that calculates a difference image between a reference image and a source image; a grouping unit that groups pixels of the difference image according to gray level; a mask image generation unit that binarizes the grouped pixels to generate a mask image for each pixel group; a partial image generation unit that generates a partial reference image in the reference image as a region corresponding to the mask image and generates a partial source image in the source image as a region corresponding to the mask image; a histogram designation unit that designates the histogram of the partial source image to the histogram of the partial reference image; and a combined image calculation unit that calculates a combined image by combining all the partial source images generated per pixel group.

[0016] Specifically, the partial image generation unit can generate the partial reference image through an AND operation between the reference image and the mask image, and generate the partial source image through an AND operation between the source image and the mask image.

[0017] Specifically, the histogram designation unit calculates the histogram of the partial reference image and the histogram of the partial source image, calculates a reference cumulative distribution function for the histogram of the partial reference image and a source cumulative distribution function for the histogram of the partial source image, generates a designation function by comparing the reference cumulative distribution function and the source cumulative distribution function, and can convert the gray level of the pixels of the partial source image based on the designation function.

[0018] Specifically, it may include a vision inspection unit that performs a vision inspection based on the above composite image. Effects of the invention

[0019] An image processing method and an image processing system according to one embodiment of the present invention can improve the visual characteristics of an image.

[0020] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims. Brief explanation of the drawing

[0021] FIG. 1 is a drawing showing an image processing system according to one embodiment of the present invention. FIG. 2 is a flowchart of an image processing method according to one embodiment of the present invention. FIG. 3 is a drawing showing mask images according to gray levels, where FIG. 3(a) shows the first mask image, FIG. 3(b) shows the second mask image, and FIG. 3(c) shows the Nth mask image. FIG. 4 is a flowchart of a partial image generation step according to one embodiment of the present invention. FIG. 5 is a flowchart of the histogram designation step according to one embodiment of the present invention. FIG. 6(a) is a drawing showing a reference image, FIG. 6(b) is a drawing showing a histogram of the image in FIG. 6(a), FIG. 6(c) is a drawing showing a source image, and FIG. 6(d) is a drawing showing a histogram of the image in FIG. 6(c). FIG. 7(a) is a drawing showing a source image corrected through full histogram designation, and FIG. 7(b) is a drawing showing a source image corrected through partial histogram designation. Specific details for implementing the invention

[0022] The objects, specific advantages, and novel features of the present invention will become more apparent from the following detailed description and preferred embodiments in conjunction with the accompanying drawings. It should be noted that in assigning reference numerals to the components of each drawing in this specification, identical components are assigned the same number whenever possible, even if they are shown in different drawings. Furthermore, in describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions would unnecessarily obscure the essence of the invention.

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0024] FIG. 1 is a drawing showing an image processing system according to one embodiment of the present invention.

[0025] Referring to FIG. 1, an image processing system (100) according to one embodiment of the present invention comprises: a difference image calculation unit (120) for calculating a difference image between a reference image and a source image; a grouping unit (130) for grouping pixels of the difference image according to gray level; a mask image generation unit (140) for binarizing the grouped pixels to generate a mask image for each pixel group; a partial image generation unit (150) for generating a partial reference image in the reference image as an area corresponding to the mask image and generating a partial source image in the source image as an area corresponding to the mask image; a histogram designation unit (160) for designating the histogram of the partial source image to the histogram of the partial reference image; and a combined image calculation unit (170) for calculating a combined image by combining all the partial source images generated per pixel group.

[0026] An image processing system (100) according to one embodiment of the present invention has an image acquisition unit (110) that converts an analog image (or video signal) captured by a shooting device such as a camera into a digital image (or video signal) and outputs it. At this time, a source image to be processed and a reference image to be used as a standard for image processing may be output.

[0027] In this specification, an image may include a video in which a plurality of images appear in succession.

[0028] The difference image calculation unit (120) can calculate a difference image using a source image and a reference image. The difference image may represent the difference in brightness values ​​of pixels at the same location in the source image and the reference image. The brightness value of each pixel may be expressed as a gray level. The gray level may be calculated as an absolute value. The gray level of each pixel in the difference image may be calculated according to the following formula.

[0029]

[0030] In the above formula is Cha-yeong-sang, is the reference image, means source image.

[0031] The reference image may be an image with brightness optimized for vision inspection. The source image may be an image that is relatively brighter or darker compared to the reference image.

[0032] The reference image and the source image may be images taken of the same object. Before the difference image is calculated, the reference image and the source image may be aligned. Specifically, the individual pixels of the reference image and the source image may be aligned to the same position.

[0033] The grouping unit (130) groups the pixels according to the gray level. The grouping unit (130) may group the pixels according to the range of the gray level. The gray level may have a range of 0 to 255.

[0034] Grouping the pixels of the difference image can be expressed as follows.

[0035]

[0036]

[0037]

[0038]

[0039] In the above formula represents the gray value of the pixels composing the image, and therefore the maximum value will be 255. Also is a grouped image.

[0040] The mask image generation unit (140) can generate a mask image for each pixel group by binarizing the grouped pixels. That is, the mask image generation unit (140) can assign a value of 1 or 255 to pixels corresponding to the gray level range and assign a value of 0 to pixels not corresponding to the gray level range. Accordingly, the mask image generation unit (140) can generate a mask image according to the gray level range. That is, the mask image generation unit (140) can generate a mask image for each pixel group grouped according to the gray level.

[0041] The generation of the mask image can be expressed as follows:

[0042]

[0043]

[0044]

[0045]

[0046] In the above formula is a mask image, and it shows the process of creating a mask image using grouped images.

[0047] The partial image generation unit (150) generates a partial reference image in the area corresponding to the mask image in the reference image, and generates a partial source image in the area corresponding to the mask image in the source image. The partial image generation unit (150) can generate a partial reference image and a partial source image for each pixel group grouped according to the gray level.

[0048] In detail, the partial image generation unit (150) can generate the partial reference image through an AND operation between the reference image and the mask image, and generate the partial source image through an AND operation between the source image and the mask image.

[0049] The AND operation between the reference image and the mask image can be expressed as follows.

[0050]

[0051]

[0052]

[0053]

[0054] In the above formula is a partial reference image generated through the AND operation of the reference image and the mask image.

[0055] The AND operation between the source image and the mask image can be expressed as an equation as follows.

[0056]

[0057]

[0058]

[0059]

[0060] In the above formula represents a partial source image generated through the AND operation of the source image and the mask image.

[0062] The histogram designation unit (160) designates the histogram of the partial source image to the histogram of the partial reference image.

[0063] In detail, the histogram designation unit (160) can calculate the histogram of the partial reference image and the histogram of the partial source image, calculate the reference cumulative distribution function for the histogram of the partial reference image and the source cumulative distribution function for the histogram of the partial source image, generate a designation function by comparing the reference cumulative distribution function and the source cumulative distribution function, and convert the gray level of the partial source image pixels based on the designation function.

[0064] The histogram specification can be expressed as an equation as follows.

[0066]

[0067]

[0068]

[0069] In the above formula is the cumulative distribution function of the grouped reference image histogram, represents the cumulative distribution function of the grouped source image histogram, and is a gray level conversion function for source image pixels.

[0070] The converted gray level can be applied to the partial source image. That is, the partial source image can be corrected according to the converted gray level.

[0071] The composite image generation unit (170) calculates a composite image by combining all of the partial source images generated per pixel group. In the equation below is a conversion function It is a partial source image converted by is the final output image.

[0072] The final output image can be expressed as an equation as follows.

[0073]

[0075] An image processing system (100) according to one embodiment of the present invention can reduce quantization error compared to performing histogram designation on the entire source image by calculating a histogram for a part of a source image, designating a histogram for the part of the source image, and combining (merging) the part of the source images to which the histogram designation is applied. Therefore, the accuracy of the histogram designation can be increased.

[0076] The image processing system (100) may include a vision inspection unit (180) that performs vision inspection based on the composite image. The vision inspection unit (180) can detect defects in the product through the composite image. The vision inspection unit (180) may determine that pixels having a gray level that differs significantly from the average gray level are defective, but the present invention is not limited thereto.

[0077] The above vision inspection unit (180) can determine whether there is a defect in the product based on deep learning, AI technology, etc. That is, the above vision inspection unit (180) can store defect patterns based on various samples and identify defects based on defect patterns.

[0078] The above image processing system (100) can be applied to various products such as semiconductors, LCDs, integrated circuits, PCBs, and barcodes, and can be used for various purposes in addition to manufacturing products, such as monitoring workers. However, the present invention is not limited thereto.

[0080] FIG. 2 is a flowchart of an image processing method according to an embodiment of the present invention. FIG. 3 is a diagram showing mask images according to gray levels, where FIG. 3(a) shows a first mask image, FIG. 3(b) shows a second mask image, and FIG. 3(c) shows an Nth mask image. FIG. 4 is a flowchart of a partial image generation step according to an embodiment of the present invention. FIG. 5 is a flowchart of a histogram designation step according to an embodiment of the present invention.

[0082] Referring to FIG. 2, an image processing method according to an embodiment of the present invention may include: a step of calculating a difference image between a reference image and a source image (S120); a step of grouping pixels of the difference image according to gray level (S130); a step of binarizing the grouped pixels to generate a mask image for each pixel group (S140); a step of generating a partial reference image in the reference image with an area corresponding to the mask image, and generating a partial source image in the source image with an area corresponding to the mask image (S150); a step of assigning the histogram of the partial source image to the histogram of the partial reference image (S160); and a step of calculating a combined image by combining all the partial source images generated per pixel group (S170).

[0083] An image processing method according to one embodiment of the present invention may include a step of converting an analog image (or video signal) captured by a capturing device such as a camera into a digital image (or video signal) and outputting it. At this time, a source image and a reference image may be output.

[0084] Based on the above source image and reference image, a difference image is calculated, a mask image is generated, a partial image is generated, and a histogram can be specified based on the partial image.

[0085] For example, in the step of grouping pixels (S130), the first pixel group may be composed of pixels having a gray level of 0 to 20, the second pixel group may be composed of pixels having a gray level of 21 to 40, and the Nth pixel group may be composed of pixels having a gray level of 64 to 255.

[0086] For example, in the step (S140) of generating a mask image, the first mask image may be composed of pixels of a first pixel group, the second mask image may be composed of pixels of a second pixel group, and the Nth mask image may be composed of pixels of an Nth pixel group. FIG. 3(a) is a first mask image, FIG. 3(b) is a second mask image, and FIG. 3(c) is a Nth mask image, respectively.

[0087] For example, in the step of generating partial images (S150), a first partial reference image is generated as an area corresponding to the first mask image in the reference image, a second partial reference image is generated as an area corresponding to the second mask image in the reference image, and an Nth partial reference image is generated as an area corresponding to the Nth mask image in the reference image.

[0088] For example, in the step (S150) of generating partial images, the first partial source image is generated as an area in the source image corresponding to the first mask image, the second partial source image is generated as an area in the source image corresponding to the second mask image, and the Nth partial source image can be generated as an area in the source image corresponding to the Nth mask image.

[0089] Referring to FIG. 4, in the step of generating a partial image (S150), the partial reference image is generated through an AND operation between the reference image and the mask image (S151), and the partial source image can be generated through an AND operation between the source image and the mask image (S152).

[0090] A step (S160) of specifying a histogram based on the above partial image may be performed. Accordingly, quantization errors occurring during the process of specifying the histogram may be reduced.

[0091] Referring to FIG. 5, the step of specifying the histogram (S160) may include: a step of calculating the histogram of the partial reference image and the histogram of the partial source image (S161); a step of calculating the reference cumulative distribution function for the histogram of the partial reference image and the source cumulative distribution function for the histogram of the partial source image (S162); a step of generating a specified function by comparing the reference cumulative distribution function and the source cumulative distribution function (S163); and a step of converting the gray level of the pixels of the partial source image based on the specified function (S164).

[0092] An image processing method according to one embodiment of the present invention may include a step of performing a vision inspection based on the combined image.

[0094] As shown in Table 1 below, the similarity between the reference image, the basic source image, the source image corrected through full histogram designation, and the source image corrected through partial histogram designation was evaluated, respectively. Similarity was measured according to known methods using RMSE (Root mean square error) and PSNR (Peak signal-to-noise ratio).

[0095] Source image Source image corrected through full histogram specification Source image corrected through partial histogram specification RMSE 0.0156 0.0028 0.0006 PSNR 36.109 50.955 63.454

[0097] The smaller the RMSE, the more similar the reference image and source image are, and the larger the PSNR, the more similar the reference image and source image are.

[0098] According to Table 1 above, the RMSE of the partial histogram-specified image is 0.0006, which is smaller than the RMSE of the full histogram-specified image, and the PSNR of the partial histogram-specified image is 63.454, which is larger than the PSNR of the full histogram-specified image.

[0099] Therefore, it was confirmed that the image with the partial histogram designation is most similar to the reference image. In other words, it is confirmed that dividing the source image and performing histogram designation improves the accuracy of the histogram designation.

[0100] FIG. 6(a) is a drawing showing a reference image, FIG. 6(b) is a drawing showing a histogram of the image in FIG. 6(a), FIG. 6(c) is a drawing showing a source image, and FIG. 6(d) is a drawing showing a histogram of the image in FIG. 6(c).

[0101] FIG. 7(a) is a drawing showing a source image corrected through full histogram designation, and FIG. 7(b) is a drawing showing a source image corrected through partial histogram designation.

[0102] Referring to FIGS. 6(a) and FIGS. 6(c), the source image has a higher brightness compared to the reference image. Also, referring to FIGS. 6(b) and FIGS. 6(d), the histogram of the source image has a different shape from the histogram of the reference image and has a shape that is shifted relatively to the right compared to the histogram of the reference image.

[0103] Referring to FIGS. 7(a) and FIGS. 7(b), when comparing the source image corrected through full histogram designation and the source image corrected through partial histogram designation with the reference image of FIG. 6(a), it is confirmed that the source image corrected through partial histogram designation is more similar to the reference image.

[0104] Specifically, in the partial histogram image, the hat area has fewer bright (white) regions compared to the full histogram image. In the reference image, the hat area is not rendered bright. Therefore, it is visually confirmed that the partial histogram image is more similar to the reference image than the full histogram image.

[0105] As described above, the image processing method and image processing system according to the present embodiment divide a source image into multiple regions based on gray levels and apply histogram designation to each of the divided regions to reduce quantization errors that negatively affect the accuracy of histogram designation. Accordingly, the source image can have visual characteristics similar to a reference image.

[0106] By correcting the source image to be similar to a reference image, the accuracy of the vision inspection performed on the source image can be increased.

[0108] Although the present invention has been described above with reference to embodiments, this is merely illustrative and does not limit the invention. Those skilled in the art will understand that various combinations, modifications, and applications not exemplified in the embodiments are possible without departing from the essential technical content of the embodiments. Therefore, technical details regarding modifications and applications that can be easily derived from the embodiments of the present invention should be interpreted as being included in the present invention. Explanation of the symbols

[0109] 100: Image Processing System 110: Image acquisition unit 120: Difference image calculation unit 130: Grouping unit 140: Mask image generation unit 150: Partial image generation unit 150: Histogram designation unit 160: Addition calculation unit 170: Addition calculation unit 180: Vision Inspection Execution Unit

Claims

Claim 1 An image processing method performed by an image processing system, comprising: a step of generating a difference image between a reference image and a source image; a step of grouping pixels of the difference image according to gray level; a step of generating a mask image for each pixel group by binarizing the grouped pixels; a step of generating a partial reference image from the reference image into an area corresponding to the mask image, and generating a partial source image from the source image into an area corresponding to the mask image; a step of converting the gray level of pixels of the partial source image so that the histogram distribution of the partial source image corresponds to the histogram distribution of the partial reference image; and a step of generating a combined image by combining all the partial source images generated per pixel group. Claim 2 An image processing method according to claim 1, wherein, in the step of generating a partial image, the partial reference image is generated through an AND operation between the reference image and the mask image, and the partial source image is generated through an AND operation between the source image and the mask image. Claim 3 An image processing method according to claim 1, wherein the step of specifying the histogram comprises: a step of calculating the histogram of the partial reference image and the histogram of the partial source image; a step of calculating the reference cumulative distribution function for the histogram of the partial reference image and the source cumulative distribution function for the histogram of the partial source image; a step of generating a specified function by comparing the reference cumulative distribution function and the source cumulative distribution function; and a step of converting the gray level of the pixels of the partial source image based on the specified function. Claim 4 An image processing method comprising, in claim 1, a step of performing a vision inspection based on the combined image. Claim 5 An image processing system comprising: a difference image generation unit for generating a difference image between a reference image and a source image; a grouping unit for grouping pixels of the difference image according to gray level; a mask image generation unit for generating a mask image for each pixel group by binarizing the grouped pixels; a partial image generation unit for generating a partial reference image in a region corresponding to the mask image in the reference image and generating a partial source image in a region corresponding to the mask image in the source image; a histogram designation unit for converting the gray level of pixels of the partial source image so that the histogram distribution of the partial source image corresponds to the histogram distribution of the partial reference image; and a combined image generation unit for generating a combined image by combining all the partial source images generated per pixel group. Claim 6 In claim 5, the partial image generation unit generates the partial reference image through an AND operation between the reference image and the mask image, and generates the partial source image through an AND operation between the source image and the mask image, an image processing system. Claim 7 An image processing system according to claim 5, wherein the histogram designation unit calculates a histogram of the partial reference image and a histogram of the partial source image, calculates a reference cumulative distribution function for the histogram of the partial reference image and a source cumulative distribution function for the histogram of the partial source image, generates a designation function by comparing the reference cumulative distribution function and the source cumulative distribution function, and converts the gray level of the pixels of the partial source image based on the designation function. Claim 8 An image processing system comprising, in paragraph 5, a vision inspection unit that performs a vision inspection based on the above combined image.

Images