Chip appearance inspection system based on photometric stereo vision

Abstract

The present invention relates to the field of visual inspection technology, and in particular to a chip appearance inspection system based on photometric stereo vision, comprising an appearance recognition module, a detection and analysis module and a visual inspection module. By using a standard image of a chip to be tested, a plurality of independent inspection areas are set on the surface of the chip to be tested, and then abnormal points in each independent inspection area are determined, and for the abnormal points and pixel values, a pixel separation value of the independent inspection area is calculated, and then a region state corresponding to the independent inspection area is determined according to the pixel separation value, and finally, a light intensity quantity of photometric stereo vision detection of each independent inspection area is determined according to the region state. When the visual inspection module detects the chip to be tested, a corresponding light intensity quantity is set during photometric stereo vision detection according to a detection position, and an appearance inspection of the chip to be tested is performed, thereby realizing intelligent adjustment in the detection process, and can better adapt to the detection requirements of different regions of the chip.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and in particular to a chip appearance inspection system based on photometric stereo vision. Background Art

[0002] Chip appearance inspection is an important part of ensuring chip quality and performance. It involves visual inspection of the chip surface and structure to identify whether there is physical damage, defects or irregular shapes.

[0003] Prior art CN118483237A discloses a chip surface defect detection system and method based on photometric stereo vision, including a graphics acquisition device and a data processing system, wherein the data processing system includes: a light source module: used to provide uniformly distributed lighting; a data processing module: used to receive image data and perform preprocessing; an image enhancement module: used to enhance the preprocessed image and improve the key features in the image; a defect detection module: used to analyze the enhanced image and identify existing defects; a display control module: used to display the detection results; the graphics acquisition device includes: the light source module, a computer control and processing system.

[0004] However, when using photometric stereo vision to detect the appearance of chips, the number of lights is usually set according to the work experience of the staff. Since the production equipment will be affected by various factors when the chip is manufactured, the surface of the chip is not completely consistent. At this time, if the number of lights cannot be adjusted according to the appearance of the chip, the number of lights will not match the chip characteristics. Furthermore, if the number of lights is too small, the surface information of the chip will be incomplete, which is prone to missed detection or misjudgment. If the number of lights is too large, the amount of data that needs to be processed increases, thereby increasing the calculation cost and time complexity, resulting in reduced detection efficiency. Summary of the invention

[0005] The purpose of the present invention is to solve the problems in the background technology and to propose a chip appearance detection system based on photometric stereo vision.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: The chip appearance inspection system based on photometric stereo vision includes: The appearance recognition module is used to collect the appearance image of the chip to be tested, mark it as a target analysis image, and transmit it to the detection and analysis module; The detection and analysis module is used to obtain a standard image of the chip to be tested according to the model of the chip to be tested, identify the pixels in the standard image, compare adjacent pixels, and determine the edge points based on the comparison results, and divide the standard image into several regions with the same value according to the edge points; Obtain a target analysis image of the chip to be tested, place the target analysis image according to the image direction and position of the standard image, and then mark the same value area in the standard image in the target analysis image to obtain an independent detection area; Mark the pixel values ​​of the same value area in the standard image as the reference value, then traverse the pixel points in the independent detection area, obtain the pixel value of each pixel point, compare the pixel value with the reference value, determine the abnormal points in the independent detection area, and then calculate the pixel separation value of the independent detection area based on the number of abnormal points and the pixel values ​​of the pixels in the independent detection area; Based on the pixel separation value, determine the region state of the independent detection region, and transmit the region state of the independent detection region to the visual detection module by the detection and analysis module, wherein the region state includes a normal region, a normal abnormal region, and an extreme abnormal region; The visual inspection module is used to obtain the regional status of the independent inspection area, and determine the light intensity quantity of the photometric stereoscopic vision inspection of each independent inspection area according to the regional status. When the visual inspection module inspects the chip to be tested, the corresponding light intensity quantity is set during the photometric stereoscopic vision inspection according to the inspection position, and the appearance inspection of the chip to be tested is performed.

[0007] As a further solution of the present invention, a method for dividing regions of equal value includes: S1: extracting the model of the chip to be tested, and acquiring a standard image of the chip to be tested based on the model of the chip to be tested; A plane coordinate system is set in the standard image, wherein the lower left corner of the standard image is set as the origin of the plane coordinate system, and the pixel size of the standard image is set as the unit length of the plane coordinate system; S2: Set a starting pixel point in the standard image, and start from the starting pixel point, compare the pixel values ​​of adjacent pixels in turn. If the pixel values ​​between two points are the same, then merge the two pixels and mark the area after the pixel merger as the same value area. Otherwise, if the pixel values ​​between the two points are not the same, then mark the pixel point with the same pixel value as the starting pixel point as the edge point. Identify the edge points of the same-value region, and connect the edge points in order according to their position coordinates. When the edge points enclose the same-value region into a closed area, the identification of the edge points of the same-value region is completed. Then, the initial pixel points are reselected in the remaining area of ​​the standard image, and are processed according to the method in the above step S2, so as to divide the standard image into a plurality of regions with the same value.

[0008] As a further solution of the present invention, when the edge points are connected in sequence, the connected edge points are adjacent pixel points, that is, there are no other pixel points between the connected edge points.

[0009] As a further solution of the present invention, a method for determining an independent detection area includes: Identify the image direction of the standard image, obtain the target analysis image at the same time, and place the target analysis image according to the image direction and position of the standard image; Then, the same-value regions and edge points of the same-value regions in the standard image are obtained, and the corresponding points are marked in the target analysis image according to the position coordinates of the edge points. At this time, multiple closed regions are also obtained in the target analysis image, and then each closed region is marked as an independent detection region; Furthermore, there is a one-to-one correspondence between the regions with the same value in the standard image and the independent detection regions in the target analysis image.

[0010] As a further solution of the present invention, when capturing the target analysis image, the image capturing angle of the target analysis image is consistent with the capturing angle of the standard image.

[0011] As a further solution of the present invention, a method for calculating the pixel separation value of the independent detection area includes: An independent detection area is randomly selected in the target analysis image, and the pixels in the independent detection area are traversed first to obtain the pixel value of each pixel and mark it as Pi, where i represents different pixels, i=1, 2, 3, ..., I, indicating that there are I pixels in the independent detection area; Obtain the same-value region corresponding to the independent detection region in the standard image, identify the pixel value in the same-value region, and mark it as the reference value, then obtain the pixel value of the pixel point in the independent detection region, subtract the reference value from the pixel value to obtain the data difference, identify the pixel points in the data difference that are not equal to 0, and mark the pixel points with the data difference that are not equal to 0 as abnormal points; Identify the number of abnormal points in the independent detection area and mark them as DF. Then use the formula The pixel separation value LS is calculated, where Pa is the average of the pixel values ​​Pi, a1 is the base factor, and a1>1, is the weight coefficient, and AL is the total number of pixels in the independent detection area.

[0012] As a further solution of the present invention, a method for determining the regional state of an independent detection region includes: The pixel separation value LS is compared with the discrete threshold Ly1. If LS<Ly1, the independent detection area is marked as a normal area. If Ly1≤LS<Ly2, the corresponding independent detection area is marked as a normal abnormal area. If LS≥Ly2, the corresponding independent detection area is marked as an extreme abnormal area, where Ly1<Ly2.

[0013] As a further solution of the present invention, a method for determining the light intensity quantity includes: If the area state is a normal area, the light intensity quantity of the corresponding independent detection area is set to b1; if the area state is a normal abnormal area, the light intensity quantity of the corresponding independent detection area is set to b2; if the area state is an extreme abnormal area, the light intensity quantity of the corresponding independent detection area is set to b3, wherein b1<b2<b3.

[0014] As a further solution of the present invention, it also includes a chip information acquisition module, which is used to identify the model of the chip to be tested, and based on the model of the chip to be tested, collect basic information of the chip to be tested, and transmit the basic information to the detection and analysis module, wherein the basic information includes the chip size.

[0015] As a further solution of the present invention, it also includes a terminal display module, which is used to receive the detection result of the appearance detection of the chip to be tested transmitted by the visual detection module and display it in the terminal display device.

[0016] Compared with the prior art, the advantages of the present invention are: The present invention sets a number of independent detection areas on the surface of the chip to be tested by utilizing a standard image of the chip to be tested, then determines the abnormal points in each independent detection area, and calculates the pixel separation value of the independent detection area for the abnormal points and pixel values, then determines the area state of the corresponding independent detection area according to the pixel separation value, and finally determines the light intensity quantity of the photometric stereoscopic vision detection of each independent detection area according to the area state. When the visual detection module detects the chip to be tested, the corresponding light intensity quantity is set according to the detection position during the photometric stereoscopic vision detection, and the appearance detection of the chip to be tested is performed, thereby realizing intelligent adjustment in the detection process, which can better adapt to the detection requirements of different areas of the chip, avoids the problem of detection blind spots or over-detection that may be caused by uniform light intensity, and effectively improves the detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Reference Figure 1 , a chip appearance inspection system based on photometric stereo vision, including a chip information acquisition module, an appearance recognition module, a detection and analysis module, a visual inspection module and a terminal display module; The chip information acquisition module is used to identify the model of the chip to be tested, and based on the model of the chip to be tested, collect the basic information of the chip to be tested, where the basic information includes chip size, packaging form and pin information, etc., and then the chip information acquisition module transmits the basic information of the chip to be tested to the detection and analysis module; The appearance recognition module is used to visually recognize and collect the appearance of the chip to be tested in the detection area to obtain the appearance image of the chip to be tested; When visually identifying the appearance of the chip to be tested, it is necessary to use a visual recognition device to collect the appearance image of the chip to be tested and mark it as a target analysis image. The visual recognition device in this embodiment refers to an industrial camera; Afterwards, the appearance recognition module transmits the target analysis image to the detection and analysis module; The detection and analysis module is used to analyze the pixel points of the target analysis image and determine the regional status of the chip to be tested based on the analysis results. The specific method for determining the regional status includes: S1: extracting the model of the chip to be tested, and obtaining a standard image of the chip to be tested based on the model of the chip to be tested, wherein the standard image is set by a technician in this field based on big data experience; A plane coordinate system is set in the standard image. In this embodiment, the lower left corner of the standard image is set as the origin of the plane coordinate system, and the pixel size of the standard image is set as the unit length of the plane coordinate system. At this time, each pixel point in the standard image has a corresponding position coordinate; S2: First, set a starting pixel point in the standard image, and start from the starting pixel point, compare the pixel values ​​of adjacent pixels in turn. If the pixel values ​​between two points are the same, then merge the two pixels and mark the area after the pixel merger as the same value area. Otherwise, if the pixel values ​​between the two points are not the same, then mark the pixel point with the same pixel value as the starting pixel point as the edge point. After that, the edge points of the same-value region are identified, and the edge points are connected in order according to the position coordinates of the edge points of the same-value region. When the edge points enclose the same-value region into a closed area, the edge point identification of the same-value region is completed. In this embodiment, the pixel point at the origin position coordinate in the standard image is set as the starting pixel point, and when the edge points are connected in sequence, the connected edge points must be adjacent pixels, that is, there are no other pixels between the connected edge points; Reselecting initial pixels in the remaining area of ​​the standard image, and processing them according to the method in step S2, thereby dividing the standard image into a number of regions with the same value; S3: re-identify the image direction of the standard image, obtain the target analysis image at the same time, and place the target analysis image according to the image direction and position of the standard image; It should be further explained that when the target analysis image is captured, the image capture angle of the target analysis image is consistent with the capture angle of the standard image; Then, the same-value area and edge points of the same-value area in the standard image are obtained, and the corresponding points are marked in the target analysis image according to the position coordinates of the edge points. At this time, multiple closed areas are also set in the target analysis image, and each closed area is marked as an independent detection area; It should be further explained that there is a one-to-one correspondence between the regions with the same value in the standard image and the independent detection regions in the target analysis image; S4: randomly select an independent detection area in the target analysis image, and take this independent detection area as an example, first traverse the pixels in this independent detection area, obtain the pixel value of each pixel, and mark it as Pi, where i represents different pixels, i=1, 2, 3, ..., I, indicating that there are I pixels in the independent detection area; Obtain the region with the same value corresponding to the independent detection region in the standard image, identify the pixel value in the region with the same value, and mark it as the reference value, then obtain the pixel value of the pixel point in the independent detection region, subtract the reference value from the pixel value to obtain the data difference, identify the pixel points in the data difference that are not equal to 0, and mark the pixel points with the data difference that are not equal to 0 as abnormal points; Identify the number of abnormal points in the independent detection area and mark them as DF. Then use the formula The pixel separation value LS is calculated, where Pa is the average of the pixel values ​​Pi, a1 is the base factor, and a1>1, is the weight coefficient, AL is the total number of pixels in the independent detection area, a1 and The specific values ​​of are obtained by technicians in this field after big data calculation; Compare the pixel separation value LS with the discrete threshold Ly1. If LS<Ly1, the independent detection area is marked as a normal area. If Ly1≤LS<Ly2, the corresponding independent detection area is marked as a normal abnormal area. If LS≥Ly2, the corresponding independent detection area is marked as an extreme abnormal area. Ly1<Ly2, and the specific values ​​of Ly1 and Ly2 are obtained by technicians in this field after big data calculation. Then, according to the processing method in the above steps S3 to S4, the regional status of all independent detection areas in the target analysis image is marked, wherein the regional status is a normal area, a normal abnormal area, and an extreme abnormal area, and the detection and analysis module transmits the regional status of the independent detection area to the visual detection module; The visual detection module is used to obtain the regional status of the independent detection area, and determine the light intensity quantity of the photometric stereoscopic vision detection of each independent detection area according to the regional status. Specifically, the method for determining the light intensity quantity includes: If the regional state is a normal area, the light intensity quantity of the corresponding independent detection area is set to b1; if the regional state is a normal abnormal area, the light intensity quantity of the corresponding independent detection area is set to b2; if the regional state is an extreme abnormal area, the light intensity quantity of the corresponding independent detection area is set to b3. Furthermore, b1<b2<b3, and the specific values ​​of b1, b2 and b3 are set by technicians in this field according to big data experience; Then the visual inspection module obtains the position of the chip to be tested, and overlaps the chip to be tested with the target analysis image to determine the light intensity quantity of each inspection position in the chip to be tested; Afterwards, when the visual inspection module inspects the chip to be tested, the corresponding light intensity is set according to the inspection position during the photometric stereoscopic vision inspection, and the appearance inspection of the chip to be tested is performed. At the same time, the visual inspection module transmits the inspection results to the terminal display module; The terminal display module is used to receive the test results of the appearance test of the chip to be tested and display them in the terminal display device.

[0020] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A chip appearance inspection system based on photometric stereo vision, characterized in that: include: The appearance recognition module is used to collect the appearance image of the chip to be tested, mark it as a target analysis image, and transmit it to the detection and analysis module; The detection and analysis module is used to obtain a standard image of the chip to be tested according to the model of the chip to be tested, identify the pixels in the standard image, compare adjacent pixels, and determine the edge points based on the comparison results, and divide the standard image into several regions with the same value according to the edge points; Obtain a target analysis image of the chip to be tested, place the target analysis image according to the image direction and position of the standard image, and then mark the same value area in the standard image in the target analysis image to obtain an independent detection area; Mark the pixel values ​​of the same value area in the standard image as the reference value, then traverse the pixel points in the independent detection area, obtain the pixel value of each pixel point, compare the pixel value with the reference value, determine the abnormal points in the independent detection area, and then calculate the pixel separation value of the independent detection area based on the number of abnormal points and the pixel values ​​of the pixels in the independent detection area; Based on the pixel separation value, determine the region state of the independent detection region, and transmit the region state of the independent detection region to the visual detection module by the detection and analysis module, wherein the region state includes a normal region, a normal abnormal region, and an extreme abnormal region; The visual inspection module is used to obtain the regional status of the independent inspection area, and determine the light intensity quantity of the photometric stereoscopic vision inspection of each independent inspection area according to the regional status. When the visual inspection module inspects the chip to be tested, the corresponding light intensity quantity is set during the photometric stereoscopic vision inspection according to the inspection position, and the appearance inspection of the chip to be tested is performed.

2. The chip appearance detection system based on photometric stereo vision according to claim 1, characterized in that: The methods for dividing the same value area include: S1: extracting the model of the chip to be tested, and acquiring a standard image of the chip to be tested based on the model of the chip to be tested; A plane coordinate system is set in the standard image, wherein the lower left corner of the standard image is set as the origin of the plane coordinate system, and the pixel size of the standard image is set as the unit length of the plane coordinate system; S2: Set a starting pixel point in the standard image, and start from the starting pixel point, compare the pixel values ​​of adjacent pixels in turn. If the pixel values ​​between two points are the same, then merge the two pixels and mark the area after the pixel merger as the same value area. Otherwise, if the pixel values ​​between the two points are not the same, then mark the pixel point with the same pixel value as the starting pixel point as the edge point. Identify the edge points of the same-value region, and connect the edge points in order according to their position coordinates. When the edge points enclose the same-value region into a closed area, the identification of the edge points of the same-value region is completed. Then, the initial pixel points are reselected in the remaining area of ​​the standard image, and processed according to the method in the above step S2, so as to divide the standard image into a plurality of regions with the same value.

3. The chip appearance detection system based on photometric stereo vision according to claim 2, characterized in that: When the edge points are connected in sequence, the connected edge points are adjacent pixel points, that is, there are no other pixel points between the connected edge points.

4. The chip appearance detection system based on photometric stereo vision according to claim 2, characterized in that: Methods for determining independent detection areas include: Identify the image direction of the standard image, obtain the target analysis image at the same time, and place the target analysis image according to the image direction and position of the standard image; Then, the same-value regions and edge points of the same-value regions in the standard image are obtained, and the corresponding points are marked in the target analysis image according to the position coordinates of the edge points. At this time, multiple closed regions are also obtained in the target analysis image, and then each closed region is marked as an independent detection region; Furthermore, there is a one-to-one correspondence between the regions with the same value in the standard image and the independent detection regions in the target analysis image.

5. The chip appearance detection system based on photometric stereo vision according to claim 4, characterized in that: When the target analysis image is captured, the image capture angle of the target analysis image is consistent with the capture angle of the standard image.

6. The chip appearance detection system based on photometric stereo vision according to claim 4, characterized in that: The calculation method of the pixel separation value of the independent detection area includes: An independent detection area is randomly selected in the target analysis image, and the pixels in the independent detection area are traversed first to obtain the pixel value of each pixel and mark it as Pi, where i represents different pixels, i=1, 2, 3, ..., I, indicating that there are I pixels in the independent detection area; Obtain the same-value region corresponding to the independent detection region in the standard image, identify the pixel value in the same-value region, and mark it as the reference value, then obtain the pixel value of the pixel point in the independent detection region, subtract the reference value from the pixel value to obtain the data difference, identify the pixel points in the data difference that are not equal to 0, and mark the pixel points with the data difference that are not equal to 0 as abnormal points; Identify the number of abnormal points in the independent detection area and mark them as DF. Then use the formula The pixel separation value LS is calculated, where Pa is the average of the pixel values ​​Pi, a1 is the base factor, and a1>1, is the weight coefficient, and AL is the total number of pixels in the independent detection area.

7. The chip appearance detection system based on photometric stereo vision according to claim 6, characterized in that: The method for determining the area status of the independent detection area includes: The pixel separation value LS is compared with the discrete threshold Ly1. If LS<Ly1, the independent detection area is marked as a normal area. If Ly1≤LS<Ly2, the corresponding independent detection area is marked as a normal abnormal area. If LS≥Ly2, the corresponding independent detection area is marked as an extreme abnormal area, where Ly1<Ly2.

8. The chip appearance detection system based on photometric stereo vision according to claim 1, characterized in that: Methods for determining the light intensity quantity include: If the area state is a normal area, the light intensity quantity of the corresponding independent detection area is set to b1; if the area state is a normal abnormal area, the light intensity quantity of the corresponding independent detection area is set to b2; if the area state is an extreme abnormal area, the light intensity quantity of the corresponding independent detection area is set to b3, wherein b1<b2<b3.

9. The chip appearance detection system based on photometric stereo vision according to claim 1, characterized in that: It also includes a chip information acquisition module, which is used to identify the model of the chip to be tested, and based on the model of the chip to be tested, collect basic information of the chip to be tested, and transmit the basic information to the detection and analysis module, wherein the basic information includes the chip size.

10. The chip appearance detection system based on photometric stereo vision according to claim 1, characterized in that: It also includes a terminal display module, which is used to receive the detection results of the appearance detection of the chip to be tested transmitted by the visual detection module and display them in the terminal display device.

Images