Image acquisition method, image acquisition device and defect detection method

CN118901000APending Publication Date: 2024-11-05BOE TECHNOLOGY GROUP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380008001.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-03-03
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

During the image acquisition process, the surface heights of at least two areas on the target object are different, resulting in shadows, blurs, or uneven brightness, which affects the quality of image processing.

Method used

Use a shadowless light source to illuminate, calculate the characteristic values ​​of the image, such as shadow width and brightness difference, determine the displacement correction value of the shadowless light source based on these characteristic values, and adjust the position of the light source until the characteristic values ​​meet the preset conditions.

Benefits of technology

By automatically correcting the position of the light source, we can achieve good image acquisition effects, avoid shadows, blurs or uneven brightness, and improve the quality of subsequent image processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118901000A_ABST
    Figure CN118901000A_ABST
Patent Text Reader

Abstract

The invention provides an image acquisition method, and relates to the field of image acquisition and processing. The method comprises the steps that under the condition of lighting of a shadowless light source, an image of a target object is shot, the image comprises a first area and a second area of the target object, and the surface height of the first area is different from the surface height of the second area; calculating N feature values of the image according to the relative position relationship between the first region and the second region, N being greater than or equal to 1; when the N characteristic values do not meet preset conditions, determining a displacement correction value of the shadowless light source according to the N characteristic values; and adjusting the position of the shadowless light source according to the displacement correction value so as to shoot the image again and calculate the N feature values until the N feature values meet the preset condition. The invention also provides an image acquisition device and a defect detection method.
Need to check novelty before this filing date? Find Prior Art

Description

Image acquisition method, image acquisition device and defect detection method Technical Field

[0001] The present disclosure relates to the field of image acquisition and processing, and in particular to an image acquisition method, an image acquisition device, and a defect detection method. Background Art

[0002] Image processing technology is widely used in scenarios such as target classification, defect detection, target recognition, and machine vision. Before image processing begins, the target object is captured to obtain a qualified image, such as by photographing the target object with a camera. This image capture process is easily affected by the target object's surface shape and external light sources. If at least two areas of the target object have different surface heights, it is difficult to maintain image quality across all areas. The captured image may exhibit shadows, blur, or uneven brightness, hindering subsequent image processing.

[0003] The above information disclosed in this section is only for understanding the background of the inventive concept of the present disclosure and therefore the above information may contain information that does not constitute prior art.

[0004] Summary of the Invention

[0005] In order to solve at least one aspect of the above problems, the embodiments of the present disclosure provide an image acquisition method, an image acquisition device, and a defect detection method.

[0006] In one aspect, an image acquisition method is provided, comprising: capturing an image of a target object under illumination by a shadowless light source, wherein the image includes a first area and a second area of ​​the target object, and the surface height of the first area is different from the surface height of the second area; calculating N eigenvalues ​​of the image based on the relative positional relationship between the first area and the second area, where N is greater than or equal to 1; determining a displacement correction value of the shadowless light source based on the N eigenvalues ​​when the N eigenvalues ​​do not meet a preset condition; and adjusting the position of the shadowless light source based on the displacement correction value to recapture the image and calculate the N eigenvalues ​​until the N eigenvalues ​​meet the preset condition.

[0007] In some embodiments, it further includes: pre-establishing a functional relationship between the N characteristic values ​​and the displacement correction value; wherein, determining the displacement correction value of the shadowless light source includes: determining the displacement correction value according to the functional relationship.

[0008] In some embodiments, the target object is any one of M types of objects, the relative positional relationship between the first area and the second area on the M types of objects is different, and M is greater than or equal to 2; wherein, pre-establishing a functional relationship between the N eigenvalues ​​and the displacement correction value includes: pre-establishing M functional relationships corresponding one-to-one to the M types of objects.

[0009] In some embodiments, before determining the displacement correction value according to the functional relationship, the method further includes: determining the type of the target object; and determining the functional relationship corresponding to the type of the target object.

[0010] In some embodiments, the first side of the target object faces the shadowless light source, the first area and the second area are located on the first side, and the surface height includes the distance between the area surface and the second side of the target object, and the second side is opposite to the first side; wherein, the surface height of the first area is different from the surface height of the second area including: a first distance between at least a portion of the surface of the first area and the second side is different from a second distance between at least a portion of the surface of the second area and the second side.

[0011] In some embodiments, the N characteristic values ​​include at least one of the following: shadow width, including the maximum length of the shadow area in the image in the first direction; shadow area, including the area of ​​the shadow area; brightness difference value, obtained based on the difference between the average grayscale value of at least part of the first area in the image and the average grayscale value of at least part of the second area.

[0012] In some embodiments, the first area is connected to the second area, and the calculation of the N eigenvalues ​​of the image includes: determining the position of the reference point on the target object in the image; determining the first area of ​​interest on the image based on the position of the reference point, wherein the first area of ​​interest includes the connecting part between the first area and the second area; and calculating the shadow width in the first area of ​​interest.

[0013] In some embodiments, calculating the shadow width in the first region of interest includes: calculating the average grayscale value in the first region of interest; multiplying the average grayscale value in the first region of interest by a binarization coefficient to obtain a binarization threshold; binarizing the first region of interest according to the binarization threshold; determining the shadow area and its maximum length in the first direction according to the binarization result, wherein the average grayscale value of the shadow area is greater than the average grayscale value of the remaining areas in the first region of interest.

[0014] In some embodiments, the preset condition includes a first threshold value of the shadow width, and determining the displacement correction value of the shadowless light source based on the N characteristic values ​​includes: when the shadow width is greater than the first threshold value, determining the displacement correction value based on a pre-established functional relationship between the shadow width and the displacement correction value.

[0015] In some embodiments, before the image is captured, pre-establishing a functional relationship between the shadow width and the displacement correction value includes: when the shadow width is greater than or less than the first threshold, adjusting the position of the shadowless light source S times, where S is greater than or equal to 2; recording the position of the shadowless light source each time adjusted and the shadow width after the position is adjusted; fitting S shadowless light source positions and the shadow width after the position is adjusted to obtain a functional relationship between the shadow width and the displacement correction value.

[0016] In some embodiments, calculating the N eigenvalues ​​of the image includes: obtaining the shadow area according to the number of pixels in the shadow area.

[0017] In some embodiments, calculating the N eigenvalues ​​of the image includes: determining the position of a reference point on the target object in the image; determining a second region of interest in the first area on the image and a third region of interest in the second area based on the position of the reference point; and calculating the brightness difference between the second region of interest and the third region of interest.

[0018] In some embodiments, the calculation of the brightness difference between the second region of interest and the third region of interest includes: determining a first brightness-sensitive region in the second region of interest, and a second brightness-sensitive region in the third region of interest, wherein the reflectivity of the brightness-sensitive region is greater than that of the non-brightness-sensitive region; and calculating the brightness difference between the first brightness-sensitive region and the second brightness-sensitive region.

[0019] In some embodiments, the preset condition includes a second threshold value of the brightness difference value, and determining the displacement correction value of the shadowless light source based on the N characteristic values ​​includes: when the brightness difference value is greater than the second threshold value, determining the displacement correction value based on a pre-established functional relationship between the brightness difference value and the displacement correction value.

[0020] In some embodiments, before the image acquisition, pre-establishing the functional relationship between the brightness difference and the displacement correction value includes: adjusting the position of the shadowless light source K times when the brightness difference is greater than or less than the second threshold, K is greater than or equal to 2; recording the adjusted shadowless light source position and the adjusted brightness difference each time; fitting K shadowless light source positions and the adjusted brightness difference to obtain the functional relationship between the brightness difference and the displacement correction value.

[0021] In some embodiments, before taking an image of the target object, it also includes: obtaining the current coordinates of a reference point on the target object, wherein the reference point is used to determine the position of at least one of the first area and the second area; when the current coordinates are inconsistent with the preset coordinates, moving the target object so that the current coordinates coincide with the preset coordinates.

[0022] In some embodiments, the surface shape of the first region is a flat surface, and the surface shape of the second region is a curved surface, and at least a portion of the curved surface is higher than the flat surface.

[0023] In some embodiments, the target object includes a display device, and capturing an image of the target object includes capturing at least a portion of a pad area of ​​the display device, wherein the at least a portion of the pad area includes a planar area and a bending area of ​​the chip-on-film, wherein the first area includes the planar area, and the second area includes the bending area.

[0024] In some embodiments, the shadowless light source includes a bowl-shaped light source.

[0025] On the other hand, an embodiment of the present disclosure provides an image acquisition device for executing the image acquisition method as described above, including: a shadowless light source located on one side of a target object; an image acquisition device for capturing an image of the target object when the shadowless light source is used for lighting, wherein the image includes a first area and a second area of ​​the target object, and the surface height of the first area is different from the surface height of the second area; an image processing device for calculating N eigenvalues ​​of the image based on the relative position relationship between the first area and the second area, where N is greater than or equal to 1; when at least one eigenvalue among the N eigenvalues ​​does not meet a preset condition, determining a displacement correction value of the shadowless light source based on the at least one eigenvalue; a moving mechanism connected to the shadowless light source, for adjusting the position of the shadowless light source according to the displacement correction value, so that the image acquisition device retakes the image and the image processing device recalculates the N eigenvalues ​​until the N eigenvalues ​​meet the preset condition.

[0026] On the other hand, an embodiment of the present disclosure provides a defect detection method, comprising: obtaining an image of a target object according to the image acquisition method as described above; processing the image using a defect detection model to obtain a defect detection result output by the defect detection model.

[0027] In some embodiments, the target object includes a display device, the image includes at least a portion of the pad area of ​​the display device, and the at least portion of the pad area includes a planar area and a bending area of ​​the chip-on-chip film. Before using the defect detection model to process the image, the image is also preprocessed, specifically including: determining a reference point position on the at least portion of the pad area in the image; determining a fourth region of interest based on the reference point position, the fourth region of interest including a routing area on the at least portion of the pad area; processing the fourth region of interest to extract the boundary of the routing, wherein the defect detection model is configured to perform defect detection on the routing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other objects and advantages of the present disclosure will become apparent from the following description of the present disclosure with reference to the accompanying drawings, which will help to provide a comprehensive understanding of the present disclosure.

[0029] FIG1 is a schematic diagram of a display device according to an embodiment of the present disclosure;

[0030] FIG2 is a schematic diagram of a display panel according to an embodiment of the present disclosure;

[0031] FIG3 is a schematic plan view of a chip-on-film according to some embodiments of the present disclosure;

[0032] FIG4 is a schematic plan view of a chip-on-film according to some other embodiments of the present disclosure;

[0033] FIG5 schematically shows a schematic diagram of bending a chip-on-film according to some embodiments of the present disclosure;

[0034] FIG6a and FIG6b schematically show images captured by a conventional shooting method in the related art;

[0035] FIG7 schematically illustrates a non-ideal image captured under illumination by a shadowless light source according to some embodiments of the present disclosure;

[0036] FIG8 schematically shows a flow chart of an image acquisition method according to some embodiments of the present disclosure;

[0037] FIG9 schematically illustrates an ideal image illuminated by a shadowless light source according to some embodiments of the present disclosure;

[0038] FIG10 schematically shows a flow chart for determining a displacement correction value according to some embodiments of the present disclosure;

[0039] FIG11 schematically shows a flow chart of calculating shadow width according to some embodiments of the present disclosure;

[0040] FIG12 schematically illustrates a visualization diagram of calculated shadow width according to some embodiments of the present disclosure;

[0041] FIG13 schematically illustrates a flow chart for establishing a functional relationship between shadow width and displacement correction value according to some embodiments of the present disclosure;

[0042] FIG14 schematically shows a flow chart of calculating a brightness difference value according to some embodiments of the present disclosure;

[0043] FIG15 schematically shows a visualization diagram of calculated brightness difference values ​​according to some embodiments of the present disclosure;

[0044] FIG16 schematically shows a structural diagram of an image acquisition device according to some embodiments of the present disclosure;

[0045] FIG17 schematically shows a flow chart of a defect detection method according to some embodiments of the present disclosure;

[0046] FIG18 schematically shows a flowchart of training and deploying a defect detection model according to some embodiments of the present disclosure; and

[0047] FIG19 schematically shows a flowchart of image acquisition and defect detection according to some embodiments of the present disclosure.

[0048] It should be noted that, for the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the sizes of layers, structures or regions may be enlarged or reduced, that is, these drawings are not drawn according to the actual scale.

[0049] The reference numerals of some of the drawings are as follows:

[0050] 100, Chip on Film; 200, Display Panel; 300, Circuit Board;

[0051] 110, plane area; 120, bending area; 201, gate signal input terminal; 202, data signal input terminal; 500, PAD detection area; 501, routing area; 1210, T-shaped reference point; 1220, first area of ​​interest; 1510, cross-shaped reference point; 1520, second area of ​​interest; 1530, third area of ​​interest. DETAILED DESCRIPTION

[0052] In the following description, for the purpose of explanation, many specific details are set forth to provide a comprehensive understanding of the various exemplary embodiments. However, it is apparent that the various exemplary embodiments can be implemented without these specific details or with one or more equivalent arrangements. In other cases, well-known structures and devices are shown in block diagram form to avoid unnecessarily obscuring the various exemplary embodiments. In addition, the various exemplary embodiments can be different, but not necessarily exclusive. For example, the specific shape, configuration, and characteristics of the exemplary embodiment can be used or implemented in another exemplary embodiment without departing from the inventive concept.

[0053] In the accompanying drawings, the sizes and relative sizes of the elements may be exaggerated for clarity and / or descriptive purposes. Thus, the sizes and relative sizes of the individual elements are not necessarily limited to those shown in the drawings. When the exemplary embodiments can be implemented differently, the specific process sequence may be performed differently from the described sequence. For example, two processes described in succession may be performed substantially simultaneously or in an order opposite to the described sequence. In addition, the same reference numerals represent the same elements.

[0054] When an element is described as being "on" another element, "connected to" another element, or "bound to" another element, the element may be directly on, directly connected to, or directly bound to another element, or there may be an intermediate element. However, when an element is described as being "directly on" another element, "directly connected to," or "directly bound to," there are no intermediate elements. Other terms and / or expressions used to describe the relationship between elements should be interpreted in a similar manner, for example, "between" versus "directly between," "adjacent" versus "directly adjacent," or "on" versus "directly on," etc. In addition, the term "connected" may refer to a physical connection, an electrical connection, a communication connection, and / or a fluid connection. In addition, the X-axis, Y-axis, and Z-axis are not limited to the three axes of a rectangular coordinate system, and may be interpreted in a broader sense. For example, the X-axis, Y-axis, and Z-axis may be perpendicular to each other, or may represent different directions that are not perpendicular to each other.

[0055] It should be understood that although the terms first, second, etc. may be used herein to describe different elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first element may be named a second element, and similarly, a second element may be named a first element without departing from the scope of the exemplary embodiments.

[0056] In the embodiments of the present application, "multiple" means two or more, "multiple" means two or more, and "at least one" means one or more, unless otherwise clearly defined.

[0057] Herein, the expression "PAD" means a pad area.

[0058] As used herein, the term "pin" refers to a portion of a COF that is electrically connected to other leads, traces, electrodes, etc., including but not limited to a PAD on the COF.

[0059] Herein, the expression "trace" refers to a signal line for transmitting a signal.

[0060] In this article, the expression "bowl-shaped light source" refers to a diffuse reflection shadowless light source based on a bowl-shaped structure.

[0061] Herein, the expression "brightness-sensitive area" means having a greater reflectivity than a non-brightness-sensitive area. Reflectivity refers to the ability to reflect light, and the greater the reflectivity, the higher the surface brightness.

[0062] Some embodiments of the present disclosure provide an image acquisition method, comprising: capturing an image of a target object under illumination by a shadowless light source, wherein the image includes a first area and a second area of ​​the target object, and the surface height of the first area is different from the surface height of the second area. Based on the relative positional relationship between the first area and the second area, N eigenvalues ​​of the image are calculated. When the N eigenvalues ​​do not meet preset conditions, a displacement correction value of the shadowless light source is determined based on the N eigenvalues. The position of the shadowless light source is adjusted based on the displacement correction value to re-capture the image and calculate the N eigenvalues ​​until the N eigenvalues ​​meet the preset conditions.

[0063] In some embodiments of the present disclosure, shadows can be reduced by lighting with a shadowless light source. N eigenvalues ​​of the captured image are calculated to determine whether the image meets the requirements. If not, a displacement correction value for the shadowless light source is determined based on the N eigenvalues. The position of the shadowless light source is then adjusted in real time, the image is retaken, and the N eigenvalues ​​are again calculated and determined to determine whether they meet the preset conditions. This process repeats until the image meets the requirements. Therefore, by automatically correcting the position of the shadowless light source and determining the optimal lighting position relative to the target object, a good image acquisition effect is achieved, shadows, blur, or uneven brightness in the image are avoided, and subsequent image processing operations are facilitated.

[0064] For ease of description and understanding, the following uses image acquisition and defect detection on a display device as an example to illustrate the image acquisition method, image acquisition device, and defect detection method in some embodiments of the present disclosure. It should be noted that the present disclosure is not intended to be limited to image acquisition and processing scenarios involving display devices; for example, it can also be used for image acquisition and defect detection of irregularly shaped workpieces, image acquisition and target recognition in multi-object stacking scenarios, or object processing, assembly, or sorting by industrial robots based on machine vision.

[0065] Figure 1 is a schematic diagram of a display device according to an embodiment of the present disclosure, Figure 2 is a schematic diagram of a display panel according to an embodiment of the present disclosure, Figure 3 is a schematic plan view of a chip-on-film according to some embodiments of the present disclosure, and Figure 4 is a schematic plan view of a chip-on-film according to other embodiments of the present disclosure.

[0066] 1 to 4 , a display device may include a display panel 200, a chip-on-film (COF) 100, and a circuit board 300. The display panel 200 is electrically connected to the circuit board 300 via the COF 100. For example, the display panel 200 may include a display area AA and a peripheral area NA, where the peripheral area NA is, for example, disposed around the AA area. In some exemplary embodiments, a plurality of sub-pixels SP may be disposed in the display area.

[0067] FIG2 illustrates an example of an arrangement of the above-mentioned multiple sub-pixels SP in an array. In this case, the sub-pixels SP arranged in a row along the horizontal direction can be referred to as sub-pixels in the same row, and the sub-pixels SP arranged in a row along the vertical direction can be referred to as sub-pixels in the same column. Optionally, the sub-pixels in the same row can be connected to a gate line GL, and the sub-pixels in the same column can be connected to a data line DL. On this basis, in some embodiments of the present disclosure, as shown in FIG2 , the display panel 200 may further include multiple gate signal input terminals 201 and multiple data signal input terminals 202.

[0068] As shown in FIG2 , for a single-side driven display panel 200 , the data signal input terminal 202 and the gate signal input terminal 201 are arranged on the same side of the display panel 200 , and the data signal input terminal 202 is arranged in the middle position, and the gate signal input terminal 201 is arranged at the edge position.

[0069] In some embodiments of the present disclosure, the data signal input terminal 202 is electrically connected to the data line DL on the display panel 200 , and the gate signal input terminal 201 is electrically connected to the gate line GL.

[0070] In this case, the gate signal input terminal 201 and the data signal input terminal 202 on the display panel 200 can be bound to the circuit board 300 through the COF 100 to transmit the electrical signal on the circuit board 300 to the display panel 200 .

[0071] It should be noted that the embodiments of the present disclosure do not specifically limit the type of the display panel 200. The type can be a TN (Twisted Nematic) type, VA (Vertical Alignment) type, IPS (In-Plane Switching) type, or ADS (Advanced Super Dimension Switch) type liquid crystal display panel 200, or an OLED (Organic Light-Emitting Diode) display panel 200.

[0072] For example, the circuit board 300 may be an FPC (abbreviation of Flexible Printed Circuit, also known as a flexible circuit board) or a PCB (abbreviation of Printed Circuit Board, also known as a printed circuit board).

[0073] In an embodiment of the present disclosure, a chip-on-film (COF) 100 is provided. As shown in FIG. 3 and FIG. 4 , the COF 100 may include a flexible substrate 1 .

[0074] For example, the material of the flexible substrate 1 may include PI (Polyimide), PA (Polyamide), or PBO (Poly-p-phenylene benzobisoxazole).

[0075] In an embodiment of the present disclosure, a COF 100 may include multiple bonding areas on a flexible substrate 1. For example, referring to FIG3 , the multiple bonding areas may include at least one chip bonding area B2, which is used to bond to a chip. Alternatively, the multiple bonding areas may include multiple chip bonding areas B2, each of which is bonded to multiple chip ICs. In other words, in this embodiment, multiple chip ICs may be provided on a COF 100, and the multiple chip bonding areas B2 correspond one-to-one to the multiple chip ICs.

[0076] For example, referring to FIG4 , the plurality of binding areas may include a panel binding area B1, a chip binding area B2, a circuit board binding area B3, and other non-binding areas (other areas not framed by dotted lines in FIG4 ). For example, the panel binding area B1 is used for binding to the display panel 200, the chip binding area B2 is used for binding to the chip, and the circuit board binding area B3 is used for binding to the circuit board 300. In this embodiment, there is no particular limitation on the number of panel binding areas B1, chip binding areas B2, and circuit board binding areas B3. For example, multiple chip binding areas B2 may be provided to respectively bind to multiple chip ICs.

[0077] In the embodiments of the present disclosure, multiple pins may be provided in each binding area of ​​the COF 100. For example, in the panel binding area B1, multiple pins P1 may be provided to bind to corresponding pins on the display panel 200. In the chip binding area B2, multiple pins P2 may be provided to bind to corresponding pins on the chip IC. In the circuit board binding area B3, multiple pins P3 may be provided to bind to corresponding pins on the circuit board.

[0078] Figure 5 schematically illustrates a schematic diagram of a bend of a COF 100 according to some embodiments of the present disclosure. Figures 6a and 6b schematically illustrate images captured using conventional shooting methods in related art. Figure 7 schematically illustrates a non-ideal image captured using a shadowless light source according to some embodiments of the present disclosure.

[0079] In an embodiment of the present disclosure, as shown in FIG5 , a chip-on-film (COF) 100 is folded back on the display panel 200 and attached to a wiring area 501 on the back of the display panel 200, forming a flat area 110 and a bent area 120 that adhere to the display panel 200. The flat area 110 has a flat surface, while the bent area 120 has a curved surface. A distance d is defined between the top of the bent area 120 and the display panel 200, for example, 0.3 mm to 0.5 mm (for example only).

[0080] Continuing with FIG5 , because the chip-on-film 100 has a planar region 110, a bent region 120, and the junction between the two, it is prone to defects. Therefore, a PAD inspection area 500 is defined for image acquisition and defect detection. The PAD inspection area 500 includes at least a portion of the planar region 110, at least a portion of the bent region 120, and the junction.

[0081] As shown in Figures 6a and 6b, due to the presence of bend region 120, conventional imaging methods cannot simultaneously capture images of both planar region 110 and bend region 120. Using a light source with shadows (such as a point light source, ring light, or strip light) makes it difficult to clearly capture bend region 120, making it very easy to miss defects.

[0082] While shadowless light sources can reduce shadows, image quality can vary depending on the lighting position. Even with the same lighting position, image quality can vary depending on the product (e.g., with different bending directions, display device sizes, or bending angles). For example, the image captured in Figure 7 shows large shadows at the bend, a significant brightness difference between the bend and flat surfaces, and completely black or uneven brightness across flat surfaces. The following further describes image capture methods for automatically adjusting the position of a shadowless light source.

[0083] Figure 8 schematically illustrates a flow chart of an image acquisition method according to some embodiments of the present disclosure. Figure 9 schematically illustrates an ideal image illuminated by a shadowless light source according to some embodiments of the present disclosure.

[0084] As shown in FIG. 8 , the image acquisition method of this embodiment includes operations S810 to S840 .

[0085] In operation S810 , an image of a target object is captured under illumination by a shadowless light source, wherein the image includes a first region and a second region of the target object, and a surface height of the first region is different from a surface height of the second region.

[0086] For example, the first area and the second area can be connected and both have flat surfaces, similar to steps. Due to the height difference between the surface height of the first area and the surface height of the second area, shadows will be formed in some areas under the light emitted by the shadowless light source.

[0087] For example, the first and second regions may be close together, with one region being flat and the other being uneven. Under light, at least a portion of the uneven surface's shadow is cast onto the flat surface. In some embodiments, the first region has a flat surface, while the second region has a curved surface, with at least a portion of the curved surface being higher than the flat surface.

[0088] In some embodiments, the target object includes a display device, and capturing an image of the target object includes: capturing at least a portion of a pad area of ​​the display device, wherein at least a portion of the pad area includes a planar area 110 and a bending area 120 of the chip cover film 100, wherein the first area includes the planar area 110 and the second area includes the bending area 120.

[0089] It is understandable that target objects with flat surfaces and curved surfaces are not limited to the PAD detection area 500 of the display screen, but may also include other special-shaped workpieces or other products, such as semiconductor substrates, electronic parts, rubber parts or mechanical parts.

[0090] In some embodiments, the shadowless light source includes a bowl-shaped light source. For example, the bowl-shaped light source can use LED particles, which form smooth and uniform illumination after spherical diffuse reflection. When the bowl-shaped light source is used, image acquisition can be performed in both the first area and the second area. It is understood that the present disclosure is not limited to bowl-shaped light sources, and can also include dome-shaped, annular, rectangular, flat, or multi-faceted shadowless light sources.

[0091] In some embodiments, the first side of the target object faces the shadowless light source, the first area and the second area are located on the first side, and the surface height includes the distance between the area surface and the second side of the target object, and the second side is opposite to the first side; wherein, the surface height of the first area is different from the surface height of the second area including: a first distance between at least a portion of the surface of the first area and the second side is different from a second distance between at least a portion of the surface of the second area and the second side.

[0092] Taking the target object as a display device as an example, referring to FIG5 , the PAD detection area 500 is located on the first side, and the bottom plane of the display panel 200 (e.g., the bottom of the base substrate) is the second side. In a direction perpendicular to the display panel 200, the surface height of the first area can be measured as a first distance d1, and the maximum surface height of the second area can be measured as a second distance d2, where d1 and d2 are different. As shown in FIG5 , d2 is greater than d1. In other embodiments, such as when the target object is of other categories, d2 can be less than d1.

[0093] In operation S820, N feature values ​​of the image are calculated based on the relative position relationship between the first region and the second region, where N is greater than or equal to 1.

[0094] For example, the feature values ​​can be used to represent image features and can be used as image attributes to determine whether the image meets the requirements. The N feature values ​​can include color feature values, texture feature values ​​(such as texture shape, texture area, and the degree of match between the texture in the image and the texture on the actual object), shadow feature values, brightness feature values, specific area shape feature values, and area relative position feature values.

[0095] In some embodiments, the relative position relationship includes the orientation of the position of the first area relative to the position of the second area and / or the relationship between the surface heights of the two areas. Since there is a height difference between at least a portion of the surface of the first area and at least a portion of the surface of the second area, after receiving illumination from a shadowless light source, the image properties (such as the above-mentioned eigenvalues) of the first area, the second area and the junction of the two areas may be inconsistent. Referring to Figure 5, the chip-on-film 100 is bent in the left and right directions to form a plane area 110 and a bent area 120 with a left-right relative position relationship. Problems such as shadows, uneven brightness or blurring formed on the captured image are related to the left-right relative position, and the right curved surface is higher than the left plane. Therefore, the N eigenvalues ​​to be calculated can be determined based on the image problem generated by the left-right relative position.

[0096] In other embodiments, for example, taking the orientation shown in FIG5 as a reference, unlike the left and right bends shown in FIG5 , the first area and the second area can be bent up and down, and are considered to have an up-down relative position relationship. They can also be bent along a diagonal line, and are considered to have an oblique relative position relationship. Other bends can also be achieved, and correspondingly have other positional relationships. Due to different relative positional relationships, the brightness of each area may be different, and the angle of light reflection also leads to different shadow shapes or areas. Even the shape, color, and texture of a specific area may change. Therefore, based on the specific relative positional relationship, the image problems that may occur during acquisition can be considered, and N eigenvalues ​​can be determined in a targeted manner. In other words, different relative positional relationships can have N eigenvalues ​​that are at least partially different.

[0097] In other embodiments, not only the N eigenvalues ​​are determined by relying on a specific relative position relationship, but also the shape of the shadowless light source, the distance and relative position between the shadowless light source and any area (the first area or the second area) on the image quality can be considered to determine the N eigenvalues. For example, the field of view of the uniform light emitted by the shadowless light source changes with its shape, distance and relative position, causing the light irradiating a certain area to be affected.

[0098] In other embodiments, the target object may have a third area, and any one of the first, second, and third areas may have a different surface height than at least one of the other areas. The area with the higher plane may cast a shadow on the other two areas. N feature values ​​can be determined based on the relative positional relationship between each of the first, second, and third areas. It is understood that, depending on the detection range on the target object, more areas may be present, such as a fourth area and a fifth area, and N feature values ​​can be determined based on the relative positional relationship between each of the multiple areas.

[0099] In operation S830 , it is determined whether the N characteristic values ​​meet a preset condition. If so, the image acquisition is terminated. If not, operation S840 is executed.

[0100] In operation S840 , when the N characteristic values ​​do not meet a preset condition, a displacement correction value of the shadowless light source is determined according to the N characteristic values.

[0101] Exemplarily, the displacement correction value includes the distance the shadowless light source moves from its current position to its target position. The preset condition may include each feature value being within a preset numerical range. When N feature values ​​meet the preset condition, the captured image of the target object is deemed to meet the requirements. For example, if any one of the N feature values ​​is outside the preset numerical range, or if more than a certain number of feature values ​​are outside the preset numerical range, then the N feature values ​​are deemed to fail to meet the preset condition.

[0102] If one or more eigenvalues ​​are not within the corresponding value range, the position of the shadowless light source is adjusted by determining a displacement correction value and then re-collected.

[0103] In some embodiments, historical data may be obtained, and the moving direction and displacement correction value may be determined based on the moving trajectory of the shadowless light source in the historical data when a current characteristic value that does not meet a preset condition occurs.

[0104] In other embodiments, a three-dimensional lighting simulation space can be established based on the shadowless light source, the target object, and their positions and shapes. The image captured in operation S810 is used as a lighting reference in the three-dimensional lighting simulation space. Light changes and captured images under multiple movement trajectories are simulated, and characteristic values ​​are simulated and calculated to determine the optimal movement trajectory. Finally, the displacement correction value is determined based on the optimal movement trajectory.

[0105] In operation S850 , the position of the shadowless light source is adjusted according to the displacement correction value to retake an image and calculate N eigenvalues ​​until the N eigenvalues ​​meet a preset condition.

[0106] Referring to FIG8 , if the captured image still does not meet the requirements after adjusting the position, operations S810 to S850 are executed in a loop. Referring to FIG9 , an ideal image that takes into account both the planar area 110 and the curved area 120 in the PAD detection area 500 is shown, i.e., an image that meets the requirements. It is understood that the position of the shadowless light source can be adjusted within the same horizontal plane, or the height of the shadowless light source can be adjusted, and this disclosure is not limited thereto.

[0107] According to the embodiments of the present disclosure, by realizing automatic correction of the position of the shadowless light source and determining the optimal lighting position relative to the target object, a good image acquisition effect is achieved, thereby avoiding the appearance of shadows, blur or uneven brightness on the image, and facilitating subsequent image processing operations.

[0108] In some embodiments, before capturing an image of a target object, the current coordinates of a reference point on the target object may be obtained, where the reference point is used to determine the position of at least one of the first region and the second region. If the current coordinates are inconsistent with preset coordinates, the target object may be moved so that the current coordinates coincide with the preset coordinates.

[0109] For example, the target object is placed on a stage, and image alignment is performed first. The reference points include fixed coordinate points on the target object, which can be manually marked or formed during the preparation of the target object. The reference points have a fixed relative position with the first and second regions. Moving the target object during image alignment can involve moving the stage or the position of the target object at the inspection station. Since the various parameters of the camera may have already been adjusted before shooting, moving the target object can avoid re-adjusting the parameters, saving time.

[0110] According to the embodiments of the present disclosure, performing image alignment before shooting is beneficial in that the reference points can be used to accurately determine the shooting area, such as PAD detection area 500, thereby minimizing or eliminating errors in the shooting area. Furthermore, the reference point coordinates facilitate subsequent determination of the positions of the first and second areas, calculation of characteristic values, and adjustment of the position of the shadowless light source, thereby optimizing image acquisition time and quality.

[0111] FIG10 schematically shows a flow chart of determining a displacement correction value according to some embodiments of the present disclosure.

[0112] As shown in Fig. 10, determining the displacement correction value of the shadowless light source in this embodiment includes operations S1010 to S1020. Operation S1020 is one embodiment of operation S840.

[0113] In operation S1010 , a functional relationship between N eigenvalues ​​and displacement correction values ​​is pre-established.

[0114] Exemplarily, the functional relationship includes a corresponding relationship between the displacement change (i.e., the displacement correction value) and the N eigenvalue changes. The functional relationship may include a relationship between each eigenvalue and the displacement correction value, a total of N relationships. Each relationship may include one or more independent variables, such as the angle and distance of the shadowless light source relative to the target image, the relative position between the first area and the second area (such as relative angle, height difference), etc., on the basis of including the displacement correction value, and the dependent variable is an eigenvalue.

[0115] In operation S1020, a displacement correction value is determined according to the functional relationship.

[0116] For example, for an eigenvalue that doesn't meet the preset conditions, its corresponding equation is determined. Given the change in that eigenvalue that meets the preset conditions, this change is substituted into the corresponding equation to solve for the displacement correction value. Furthermore, after solving for the displacement correction value, the corresponding eigenvalue can be substituted into other equations to calculate the corresponding eigenvalue and compared with the respective preset conditions.

[0117] In other embodiments, the functional relationship may include a mathematical model constructed and trained based on a machine learning algorithm. For example, a classification model may be trained using historical data on the process of adjusting the shadowless light source and the corresponding changes in eigenvalues. The calculated N eigenvalues ​​are input into the model to output a classification of the movement direction of the shadowless light source, such as forward, backward, left, and right. Based on the classification results, the shadowless light source is then moved in multiple steps according to a fixed movement distance.

[0118] According to the embodiments of the present disclosure, the displacement correction value can be automatically determined through a pre-established functional relationship, thereby improving the speed and accuracy of position adjustment.

[0119] In some embodiments, the target object is any one of M types of objects, the relative positional relationships between the first region and the second region on the M types of objects are different, and M is greater than or equal to 2. Operation S1010 further includes pre-establishing M functional relationships corresponding one-to-one to the M types of objects.

[0120] For example, the M types of objects may be M types of display devices, such as those with a front-to-back folded chip-on-film 100, a left-to-right folded chip-on-film 100, a diagonally folded chip-on-film 100, or a multi-position folded chip-on-film 100. Thus, the relative positional relationship between the planar region 110 and the folded region 120 on the chip-on-film 100 of each type of display device is different. It will be appreciated that the M types of objects may include different types of products, such as display devices and irregularly shaped workpieces. For each type of object, a functional relationship is established between the N feature values ​​of the captured image and the folded region 120.

[0121] When capturing images of objects of different types (e.g., shapes) using a shadowless light source, the optimal lighting position varies. For display devices, for example, this is due to differences in the bending radius, curvature, and height differences of the flat surface area in the bending region 120 of different products. When switching between different types of objects for image capture, the difficulty in quickly adjusting the position of the shadowless light source limits image capture efficiency.

[0122] According to the embodiments of the present disclosure, pre-establishing M functional relationships corresponding one-to-one to M types of objects can adapt to image acquisition of multiple types of objects, expand the applicable scenarios, and improve image acquisition efficiency.

[0123] In some embodiments, before determining the displacement correction value based on the functional relationship in operation S1020, the method further includes determining the type of the target object and determining a functional relationship corresponding to the type of the target object. This is to determine an appropriate functional relationship to accurately determine the displacement correction value and achieve better image acquisition quality.

[0124] In some embodiments, the N feature values ​​include at least one of the following:

[0125] Shadow width, including the maximum length of the shadow area in the image in the first direction.

[0126] Shaded area, including the area of ​​the shaded region.

[0127] The brightness difference value is obtained according to the difference between the average grayscale value of at least a portion of the first area and the average grayscale value of at least a portion of the second area in the image.

[0128] The shadow width, shadow area and brightness difference are further described below with reference to FIG. 11 to FIG. 15 and multiple embodiments.

[0129] Figure 11 schematically shows a flow chart of calculating shadow width according to some embodiments of the present disclosure. Figure 12 schematically shows a visualization diagram of calculating shadow width according to some embodiments of the present disclosure.

[0130] 11 and 12 , the first region and the second region are connected, and calculating N feature values ​​of the image includes operations S1110 to S1130 .

[0131] In operation S1110 , locations of reference points on a target object in an image are determined.

[0132] Referring to Figure 12, the bent area is on the top and the flat area is on the bottom, so the original image is an image of at least part of the PAD detection area 500 in Figure 5 rotated 90°. The T-shaped reference point 1210 is in the shape of a "T". Usually, the circuit area will be pre-set with a metal "cross" or "T" mark point (i.e., reference point) for the convenience of detection. The mark point is the same as the metal line (i.e., the trace). The grayscale value of the mark point is lower than that of other gray metal areas in a dark field light environment. The difference in grayscale value can be used to capture the mark point to determine the coordinate value of the T-shaped reference point 1210.

[0133] Mark point capture example: After applying a Gaussian filter to the original image to remove noise, the image is binarized using a threshold of 1. The binarized image is filtered based on morphological features (mark point size, aspect ratio, etc.), and the center point (x4, y4) of the mark point is calculated. For example, the center point is the intersection of the horizontal and vertical lines in the "T".

[0134] In operation S1120 , a first region of interest 1220 on the image is determined according to the reference point position, wherein the first region of interest 1220 includes a connecting portion between the first region and the second region.

[0135] After determining the T-shaped reference point 1210, the first region of interest 1220 is determined. Referring to the ROI (Region of Interest) in Figure 12, the ROI is divided based on the center point of the Mark (because the relative values ​​of the Mark and the bend area 120 for the same product are essentially constant). This ROI includes the imaging area at the intersection of the planar area 110 and the bend area 120.

[0136] According to the size of the area where the trace (such as the metal line) is located, set the ROI size to (width, height). Then, based on the center point (x3, y3) of the Mark as the reference point, cut out the ROI area image on the original image to generate the ROI map. The enlarged image of the ROI area is shown in Figure 12.

[0137] In operation S1130 , a shadow width in the first region of interest 1220 is calculated.

[0138] According to the embodiment of the present disclosure, due to the height difference, shadows are easily formed in the connecting portion. Therefore, the first region of interest 1220 is determined by the reference point position and the size of the trace area in the flip chip 100, and the shadow width therein can be accurately calculated.

[0139] In some embodiments, calculating the shadow width in operation S1130 includes the following steps:

[0140] First, the average grayscale value in the first region of interest 1220 is calculated. After performing noise reduction processing on the ROI region image shown in FIG12 , the average grayscale value m1 of the pixels in the ROI region is calculated.

[0141] Next, the average grayscale value within the first region of interest 1220 is multiplied by the binarization coefficient to obtain a binarization threshold. The first region of interest 1220 is binarized based on the binarization threshold. Using m1 multiplied by the binarization coefficient km (set based on image characteristics and typically less than 1) as the binarization threshold, the ROI image shown in FIG12 is binarized. Pixels with grayscale values ​​lower than m1 are set to 255, and pixels with grayscale values ​​higher than m1 are set to 0.

[0142] Finally, the shadow area and its maximum length in the first direction are determined according to the binarization result, wherein the average grayscale value of the shadow area is greater than the average grayscale value of the remaining areas in the first region of interest 1220 .

[0143] If a shadow exists at the intersection of the ROI region, the grayscale value of the pixels in the shadow area is 255, and the grayscale values ​​of all other pixels are 0. Referring to Figures 5 and 12 , the first direction is the bending direction of the chip-on-film 100. Find the circumscribed rectangle of the region with a grayscale value of 255 (the shadowed region at the intersection shown in Figure 12 ). The height of this circumscribed rectangle is the shadow width K1 of the shadowed region at the intersection.

[0144] In some embodiments, the preset condition includes a first threshold value for shadow width. Determining the displacement correction value for the shadowless light source based on the N characteristic values ​​in operation S840 includes determining the displacement correction value based on a pre-established functional relationship between the shadow width and the displacement correction value when the shadow width is greater than the first threshold value. By pre-establishing a functional relationship between the shadow width and the displacement correction value, automated, rapid, and efficient correction of the light source position can be achieved, reducing the shadow width or even completely eliminating the shadow.

[0145] FIG13 schematically illustrates a flow chart for establishing a functional relationship between shadow width and displacement correction value according to some embodiments of the present disclosure.

[0146] In some embodiments, as shown in FIG. 13 , before performing operations S810 to S840 to perform image acquisition, pre-establishing a functional relationship between the shadow width and the displacement correction value includes operations S1310 to S1330 .

[0147] In operation S1310, when the shadow width is greater than or less than a first threshold, the position of the shadowless light source is adjusted S times, where S is greater than or equal to 2.

[0148] In operation S1320, each adjusted position of the shadowless light source and the shadow width after the position adjustment are recorded.

[0149] In operation S1330, the positions of the S shadowless light sources and the shadow widths after the positions are adjusted are fitted to obtain a functional relationship between the shadow widths and the displacement correction values.

[0150] In some embodiments, when K1 is greater than its first threshold, for example, greater than 7 pixels, it indicates that the shadowless light source is biased toward the bend area 120 in FIG5 and should be moved a certain distance toward the plane area 110. Taking the current position of the shadowless light source as the origin, the light source is moved a certain distance toward the plane area 110 multiple times, and the change in K1 value is recorded. The relationship between the displacement x1 (obtained based on the position of the shadowless light source before and after adjustment) and the change in K1 is fitted, thereby establishing a functional relationship between the eigenvalue K1 and the displacement correction value. In other embodiments, when K1 is less than its first threshold, the shadowless light source can also be gradually moved toward the bend area 120, and the change in K1 value is recorded. The relationship between the displacement x1 and the change in K1 is fitted, thereby establishing a functional relationship between the eigenvalue K1 and the displacement correction value.

[0151] In some embodiments, the functional relationship between the coordinate point (single coordinate axis) of the shadowless light source position and K1 can be fitted. For example, it can be moved along any axis of the XYZ coordinate axis, and the coordinate point position and K1 can be recorded, and finally three functional relationships corresponding one to one to the XYZ coordinate axis can be fitted.

[0152] It is particularly noted that although operations S1310 to S1320 are illustrated and described in text in sequence, the two can also be performed simultaneously. For example, each time an adjustment is made in operation S1310, operation S1320 is performed to record the adjusted shadowless light source position and calculate the shadow width in the image captured at that position.

[0153] In some embodiments, calculating the N eigenvalues ​​of the image in operation S820 includes obtaining the shadow area based on the number of pixels in the shadow region (i.e., the intersection shadow region) shown in Figure 12. In other embodiments, the contour of the shadow region may be fitted and the area of ​​the contour may be calculated.

[0154] According to an embodiment of the present disclosure, if the shadow area is too large, it is determined that the acquired image does not meet the requirements, so judging the shadow area can obtain a better image.

[0155] Figure 14 schematically shows a flow chart of calculating a brightness difference value according to some embodiments of the present disclosure. Figure 15 schematically shows a visualization diagram of calculating a brightness difference value according to some embodiments of the present disclosure.

[0156] In some embodiments, calculating N feature values ​​of the image in operation S820 includes operations S1410 to S1430.

[0157] In operation S1410, the position of the reference point on the target object in the image is determined. Please refer to operation S1110 for details, which will not be described in detail here.

[0158] In operation S1420, a second region of interest 1520 in the first region and a third region of interest 1530 in the second region are determined on the image according to the reference point positions.

[0159] After determining the cross-shaped reference point 1510, the second region of interest 1520 and the third region of interest 1530 are determined. Referring to the ROI region acquisition in FIG15 , the second region of interest 1520 and the third region of interest 1530 are divided according to the center point of the Mark (such as the center of the "cross" in FIG15 ). The second region of interest 1520 and the third region of interest 1530 respectively include the imaging areas on both sides of the junction of the first region and the second region.

[0160] In operation S1430 , a brightness difference value between the second region of interest 1520 and the third region of interest 1530 is calculated.

[0161] According to the embodiments of the present disclosure, when a shadowless light source is used for lighting, the light in different areas may be uneven due to positional reasons, resulting in brightness differences, especially height differences starting to occur at the junction. Therefore, determining the second area of ​​interest 1520 and the third area of ​​interest 1530 on both sides of the junction can accurately calculate the brightness difference value.

[0162] In some embodiments, calculating the brightness difference between the second ROI 1520 and the third ROI 1530 in operation S1430 includes determining a first brightness-sensitive region in the second ROI 1520 and a second brightness-sensitive region in the third ROI 1530, wherein the brightness-sensitive region has a greater reflectivity than the non-brightness-sensitive region, and calculating the brightness difference between the first brightness-sensitive region and the second brightness-sensitive region.

[0163] For example, referring to the metal wire area extraction part of the ROI area shown in Figure 15, because the brightness difference of the metal wire area is large and the reflectivity is high, first, the metal wire area in the second area of ​​interest 1520 is found as the first brightness sensitive area, and the metal wire area in the third area of ​​interest 1530 is found as the second brightness sensitive area.

[0164] Next, the average pixel values ​​m21 and m22 are calculated for the second ROI 1520 and the third ROI 1530, respectively. Using m21 and m22 as the binarization thresholds, the images of the second ROI 1520 and the third ROI 1530 are binarized. The grayscale values ​​of pixels with grayscale values ​​lower than m21 and m22 are set to 255, and the grayscale values ​​of pixels with grayscale values ​​higher than m1 are set to 0.

[0165] Next, because the brightness of the metal line area is higher than that of the non-metal line area, the grayscale values ​​of the pixels in the metal line area in the second ROI 1520 and the third ROI 1530 are now set to 0. To eliminate interference from the metal line edges, the images are subjected to opening and closing operations for noise reduction. The original images of the second ROI 1520 and the third ROI 1530 are subtracted from the binarized images to obtain image21 and image22. At this point, the grayscale values ​​of the pixels in the metal line area remain the original values, while the pixels in the non-metal line area are set to 0.

[0166] Finally, the average grayscale values ​​of pixels at the metal lines of the second and third regions of interest 1520 and 1530 are calculated and the difference is obtained. The average grayscale values ​​of image21 and image22 are calculated, and the absolute value of the difference is the brightness difference K2.

[0167] In some embodiments, the preset condition includes a second threshold for the brightness difference, and determining the displacement correction value for the shadowless light source based on the N characteristic values ​​includes: when the brightness difference is greater than the second threshold, determining the displacement correction value based on a pre-established functional relationship between the brightness difference and the displacement correction value. By pre-establishing this functional relationship between the brightness difference and the displacement correction value, timely feedback can be obtained based on the brightness difference, and the displacement correction value can be output to adjust the light source position so that the brightness of the first area and the second area are close to the same.

[0168] In some embodiments, before performing operations S810 to S840 to capture an image, pre-establishing a functional relationship between the brightness difference value and the displacement correction value includes:

[0169] When the brightness difference is greater than or less than the second threshold, adjust the position of the shadowless light source K times, where K is greater than or equal to 2. Record the adjusted shadowless light source position and the brightness difference after each adjustment. Fit the K shadowless light source positions and the adjusted brightness difference to obtain a functional relationship between the brightness difference and the displacement correction value. For example, after each adjustment, record the adjusted shadowless light source position and the brightness difference after each adjustment.

[0170] In some embodiments, when K2 is greater than its second threshold, such as the difference is 10, and the brightness of the plane area 110 is large, it means that the shadowless light source is biased toward the plane area 110 in Figure 5 and should be moved a certain distance toward the bending area 120. Taking the current shadowless light source position as the origin, move a certain distance toward the bending area 120 multiple times, record the change in K2 value, fit the displacement x2 (obtained based on the position of the shadowless light source before and after adjustment) and the change in K2, and thus establish a functional relationship between the eigenvalue K2 and the displacement correction value. In other embodiments, when K2 is less than its second threshold, the shadowless light source can be moved toward the plane area 110 multiple times, and the change in K2 value is recorded each time. Finally, the relationship between the displacement x2 and the change in K2 is fitted, and thus establish a functional relationship between the eigenvalue K2 and the displacement correction value.

[0171] In some embodiments, similar to the fitting of shadow width, the functional relationship between the coordinate point (single coordinate axis) of the shadowless light source position and K2 can be fitted. For example, it can be moved along any axis of the XYZ coordinate axis, and the coordinate point position and K1 can be recorded. Finally, three functional relationships corresponding one to one to the XYZ coordinate axis are fitted.

[0172] In some embodiments, if the N eigenvalues ​​include K1 and K2, when fitting the function, one of the eigenvalues ​​of the captured image can be adjusted to a value below a threshold before fitting the function for the other eigenvalue above the threshold. During image acquisition, when both K1 and K2 are greater than their respective thresholds, the displacement correction value can be determined using the relationship between the two functions, while both being less than their respective thresholds.

[0173] FIG16 schematically shows a structural diagram of an image acquisition device according to some embodiments of the present disclosure, but the present disclosure is not limited thereto.

[0174] In some embodiments, an image acquisition device is provided to perform the image acquisition method of some embodiments of the present disclosure. The image acquisition device may include a shadowless light source, an image acquisition device, an image processing device, and a moving mechanism.

[0175] The shadowless light source is located on one side of the target object. The image acquisition device is used to capture an image of the target object when the shadowless light source is used, wherein the image includes a first area and a second area of ​​the target object, and the surface height of the first area is different from the surface height of the second area. The image processing device is used to calculate N eigenvalues ​​of the image based on the relative positional relationship between the first area and the second area, where N is greater than or equal to 1. When at least one eigenvalue among the N eigenvalues ​​does not meet the preset conditions, the displacement correction value of the shadowless light source is determined based on the at least one eigenvalue. The moving mechanism is connected to the shadowless light source, and is used to adjust the position of the shadowless light source according to the displacement correction value, so that the image acquisition device retakes the image and the image processing device recalculates the N eigenvalues ​​until the N eigenvalues ​​meet the preset conditions.

[0176] 16 , the light source may include a bowl-shaped light source. For example, the target object (i.e., the workpiece) is a display device, which is placed on a stage (not shown in the figure), with the side having the flat area 110 and the bent area 120 of the chip-on-film 100 facing upward. The bowl-shaped light source is located above the display device to illuminate and achieve a dark field light environment. The image acquisition device can be placed above the bowl-shaped light source, photograph the target object in a vertical direction, and digitally transmit the captured image to an image processing device, such as a computer, to calculate the characteristic value and displacement correction value.

[0177] Exemplarily, the image acquisition device mainly includes a camera and a lens. The camera can be an industrial area array camera, and the lens can be an industrial telecentric lens. The mobile device may include a motor and a guide rail. The slider on the guide rail is connected to the bowl-shaped light source, which can drive the bowl-shaped light source to adjust its position to achieve the optimal lighting angle, so that the image feature values ​​of the flat area 110 and the bent area 120 are similar, thereby reducing detection interference. The mobile device can use a screw motor / UVW alignment device to drive the light source to move quantitatively. If the shadowless light source only needs to move in one axis, the motor guide rail can be used to adjust the position of the shadowless light source. If position correction between multiple axes is required, a UVW alignment platform or other alignment device can be used.

[0178] Taking the PAD detection area 500 of the OLED product shown in Figure 5 as an example, the inability to determine the optimal lighting position of the shadowless light source relative to the PAD detection area 500 leads to various situations when capturing the image, as shown in Figure 7. This causes interference such as shadow areas and light-dark intersection lines, which seriously affects image quality and also affects the efficiency and applicability of automated image acquisition. In addition, the bending radius and flatness of the chip-on-film 100 after bending vary from product to product. When capturing images of different products, the lighting position of the previous product may not be applied to the next product, resulting in image acquisition quality that does not meet requirements.

[0179] According to the image acquisition device of the embodiment of the present disclosure, the position of the light source can be automatically adjusted based on the characteristic values ​​of the first area and the second area, so that the image characteristic values ​​of the first area and the second area acquired are ultimately similar, thereby eliminating interference and taking into account the image quality of different areas.

[0180] Figure 17 schematically illustrates a flow chart of a defect detection method according to some embodiments of the present disclosure. Figure 18 schematically illustrates a flow chart of training and deploying a defect detection model according to some embodiments of the present disclosure.

[0181] As shown in FIG. 17 , the defect detection method of this embodiment includes operations S1710 to S1720 .

[0182] In operation S1710 , an image of a target object is obtained according to the image acquisition method of some embodiments of the present disclosure.

[0183] In operation S1720, the image is processed using the defect detection model to obtain a defect detection result output by the defect detection model.

[0184] Exemplarily, a defect detection model can be constructed and trained based on a deep learning algorithm. Referring to Figure 18, a training sample is obtained as a defect sample of the same type of target object as that during detection (operation S1801), the Mark points in the training sample are captured (operation S1802), the region of interest is selected based on the Mark points (operation S1803), and the image of the region of interest is subjected to Gaussian filtering and denoising (operation S1804), and X-Sobel filtering and Y-Sobel filtering are continued to eliminate noise and sharpen edges (operation S1805). The image opening operation is continued to remove the edge information in the region of interest (operation S1806). Then the image closing operation is performed to eliminate noise points (operation S1807). The pre-processed region of interest is cut into a size that can be processed by the model (operation S1808). Considering issues such as limited sample size or uneven defect distribution, data augmentation and mosaic and mixup data augmentation are performed (operation S1809). The data is then fed into the defect detection model for training (operation S1810). For example, the data is forward propagated, the loss function value is calculated based on the defect detection results and sample defect labels, and the model parameters are then backpropagated until the loss function value is less than a certain value. The trained defect detection model is deployed to the production environment via TensorRT (operation S1811).

[0185] Referring to Figure 5, after the flip-chip film 100 is folded back, it mainly includes a fitted flat area 110 and a bent area 120. This area is prone to producing more defects, such as cracks, scratches, and bubbles. Taking cracks as an example, if a crack passes through the PAD detection area 500, it will cause a short circuit, causing abnormal screen lighting. Cracks that do not pass through the wiring may also gradually develop and expand to the wiring area, causing abnormal lighting. AOI (Automated Optical Inspection) is usually used, and then the corresponding visual algorithm is used for defect detection. However, due to the quality of image acquisition not meeting the requirements, missed detection often occurs.

[0186] In some embodiments, referring to FIG. 16 , since the widths of the trace intersections and major defects (such as crack widths) in the PAD inspection area 500 are mostly between 1 and 5 μm, the accuracy of the camera and lens combination can be preferably between 0.2 and 1.5 μm / pixel, which can better present the defects.

[0187] In some embodiments, the first threshold for shadow width can be determined based on the primary defect width. For example, if the first threshold is smaller than the primary defect width, this prevents the shadow area from being too large and obscuring the defect. The second threshold for brightness difference can be determined based on the average brightness difference between the defect and the surrounding area. For example, if the second threshold is smaller than the average zero-degree difference, this can increase defect conspicuity and avoid missed detection.

[0188] It is understandable that the present disclosure is not intended to limit the target objects of defect detection to display devices, and can also be used for other special-shaped workpieces or other products, such as semiconductor substrates, electronic parts, rubber parts or mechanical parts.

[0189] According to the embodiments of the present disclosure, images that meet the requirements can be obtained, the efficiency and accuracy of defect detection can be improved, and missed detection can be avoided.

[0190] FIG19 schematically shows a flowchart of image acquisition and defect detection according to some embodiments of the present disclosure.

[0191] As shown in Figure 19, the image acquisition and defect detection of this embodiment may include operations S1901 to S1908. The following description will be made using a display device as an example.

[0192] In operation S1901, the mark point is aligned, and a cross or T-shaped mark point of metal on the chip-on-film 100 in the camera field of view is captured, and the current coordinate of the mark point is made to coincide with the preset coordinate.

[0193] In operation S1902 , a camera captures a picture of the PAD detection area 500 as shown in FIG. 5 .

[0194] In operation S1903 , a feature value, such as at least one of a shadow width, a shadow area, or a brightness difference value, is calculated.

[0195] In operation S1904, it is determined whether each feature value meets the preset condition. If so, operation S1906 is executed; if not, operation S1905 is executed.

[0196] In operation S1905, the light source is moved, the position of the shadowless light source is adjusted, and operation S1902 is re-performed.

[0197] In operation S1906 , the image is input to the defect detection model.

[0198] In some embodiments, before processing the image using the defect detection model, the image is also pre-processed. Referring to operations S1802 to S1808, at least part of the pre-processing process is as follows:

[0199] First, the position of the reference point on at least a portion of the pad area in the image is determined, that is, the cross or T-shaped mark point of the metal on the chip-on-film 100 in the image.

[0200] Next, a fourth region of interest (ROI) is determined based on the location of the reference point, which identifies at least a portion of the trace within the pad area. The ROI size is set based on the size of the metal trace area. Then, an image of the fourth ROI is captured from the original image based on the center point of the mark to generate a fourth ROI image. Noise reduction can be performed on the fourth ROI image, such as by pre-processing the image using Gaussian filtering, X-Sobel filtering, and Y-Sobel filtering to remove noise and sharpen edges.

[0201] Next, the fourth region of interest is processed to extract the edges of the wires, where the defect detection model is configured to perform defect detection on the wires. For example, the fourth region of interest image is binarized using a set threshold and then subjected to dilation and erosion processing to eliminate small bright spots within the metal wire area, smooth the metal wire edges, and break adhesions between adjacent wires. The fourth region of interest image is then subjected to dilation and erosion processing again to fill in small dark spots and broken outlines within the metal wires, and smooth the metal wire edges again without changing the metal wire area.

[0202] Finally, the fourth region of interest image is cut and input into the defect detection model. The fourth region of interest image is cut into images of specific sizes for processing by the defect detection model, for example, the cut sizes are 640*640, 1024*1024, etc.

[0203] In operation S1907 , a detection result output by the defect detection model is acquired.

[0204] In operation S1908, defect mapping is performed by mapping the defect coordinates in the inspection result back to the acquired original image and marking them.

[0205] The disclosed embodiments calculate light source position correction values ​​by calculating image eigenvalues. This correction is then applied via a corresponding automated displacement mechanism to achieve optimal image acquisition, thereby achieving automated image acquisition. Finally, based on the acquired images, a processing method based on machine vision and deep learning enables efficient defect detection.

[0206] It should be noted that some steps of the above method can be executed individually or in combination, and can be executed in parallel or sequentially, and are not limited to the specific operation sequence shown in the figure.

[0207] It should also be noted that, in some embodiments, the display device provided in the embodiments of the present disclosure may specifically be a liquid crystal display device, or include an electroluminescent display panel 200 (Organic Light Emitting Diodes, OLED), or include a quantum dot display panel 200 (Quantum Dot Light Emitting Diodes, QLED), which is not limited here.

[0208] In some embodiments, the display device provided by the embodiments of the present disclosure may be a 3D display device or other display device, and may be any product or component with a display function, such as a mobile phone, a tablet computer, a television, a monitor, a laptop computer, a digital photo frame, a navigator, a smart watch, a fitness wristband, a personal digital assistant, etc. Optionally, the above-mentioned display device provided by the embodiments of the present disclosure includes but is not limited to components such as a radio frequency unit, a network module, an audio output & input unit, a sensor, a display unit, a user input unit, an interface unit, and a control chip. Optionally, the control chip is a central processing unit, a digital signal processor, a system-on-chip (SoC), etc. For example, the control chip may also include a memory, a power module, etc., and realize power supply and signal input and output functions through additionally provided wires, signal lines, etc. For example, the control chip may also include hardware circuits and computer executable codes, etc. In addition, it will be understood by those skilled in the art that the above-mentioned structure does not constitute a limitation on the above-mentioned display device provided by the embodiments of the present disclosure. In other words, the above-mentioned display device provided by the embodiments of the present disclosure may include more or fewer of the above-mentioned components, or a combination of certain components, or different component arrangements.

[0209] As used herein, the terms "substantially," "about," "approximately," and other similar terms are used as terms of approximation rather than as terms of degree, and are intended to account for the inherent deviations in measured or calculated values ​​that would be recognized by one of ordinary skill in the art. To account for factors such as process fluctuations, measurement problems, and errors associated with the measurement of a particular quantity (i.e., limitations of the measurement system), "about" or "approximately," as used herein, are inclusive of the stated value and mean within an acceptable range of deviation for the particular value as determined by one of ordinary skill in the art. For example, "approximately" can mean within one or more standard deviations, or within ±10% or ±5% of the stated value.

[0210] Although some embodiments according to the general inventive concept of the present disclosure have been illustrated and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the general inventive concept of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. An image acquisition method, comprising: Under the condition of lighting by a shadowless light source, capturing an image of a target object, wherein the image includes a first area and a second area of ​​the target object, and a surface height of the first area is different from a surface height of the second area; Calculating N feature values ​​of the image according to a relative position relationship between the first area and the second area, where N is greater than or equal to 1; When the N characteristic values ​​do not meet the preset conditions, determining the displacement correction value of the shadowless light source according to the N characteristic values; The position of the shadowless light source is adjusted according to the displacement correction value to retake the image and calculate the N eigenvalues ​​until the N eigenvalues ​​meet the preset conditions.

2. The method according to claim 1, wherein: Also includes: Pre-establishing a functional relationship between the N characteristic values ​​and the displacement correction value; Wherein, determining the displacement correction value of the shadowless light source includes: The displacement correction value is determined according to the functional relationship.

3. The method according to claim 2, wherein: The target object is any one of M types of objects, the relative positional relationship between the first area and the second area on the M types of objects is different, and M is greater than or equal to 2; Wherein, the pre-establishing the functional relationship between the N characteristic values ​​and the displacement correction value comprises: M functional relationships corresponding one to one with the M types of objects are pre-established.

4. The method according to claim 3, wherein: Before determining the displacement correction value according to the functional relationship, the method further includes: determining the type of the target object; and The functional relationship corresponding to the type of the target object is determined.

5. The method according to claim 1, wherein: A first side of the target object faces the shadowless light source, the first area and the second area are located on the first side, the surface height includes a distance between an area surface and a second side of the target object, and the second side is opposite to the first side; Wherein, the surface height of the first region is different from the surface height of the second region including: A first distance between at least a portion of the surface of the first region and the second side is different from a second distance between at least a portion of the surface of the second region and the second side.

6. The method according to claim 1, wherein: The N eigenvalues ​​include at least one of the following: Shadow width, comprising a maximum length of a shadow area in the image in a first direction; The shaded area includes the area of ​​the shaded region; The brightness difference value is obtained according to the difference between the average grayscale value of at least part of the first area and the average grayscale value of at least part of the second area in the image.

7. The method according to claim 6, wherein: The first area is connected to the second area, and the calculating of N eigenvalues ​​of the image includes: Determining a location of a reference point on the target object in the image; Determining a first region of interest on the image according to the reference point position, wherein the first region of interest includes a connecting portion between the first region and the second region; The shadow width in the first region of interest is calculated.

8. The method according to claim 7, wherein: The calculating the shadow width in the first region of interest comprises: Calculate the average grayscale value in the first region of interest; Multiplying the average grayscale value in the first region of interest by a binarization coefficient to obtain a binarization threshold; Binarization processing is performed on the first region of interest according to the binarization threshold; The shadow area and its maximum length in the first direction are determined according to the binarization result, wherein an average grayscale value of the shadow area is greater than an average grayscale value of the remaining areas in the first region of interest.

9. The method according to claim 6, wherein: The preset condition includes a first threshold value of the shadow width, and determining the displacement correction value of the shadowless light source according to the N characteristic values ​​includes: When the shadow width is greater than the first threshold, the displacement correction value is determined according to a pre-established functional relationship between the shadow width and the displacement correction value.

10. The method according to claim 9, wherein: Before the image acquisition, the functional relationship between the shadow width and the displacement correction value is pre-established, including: When the shadow width is greater than or less than the first threshold, adjusting the position of the shadowless light source S times, where S is greater than or equal to 2; Recording each adjusted position of the shadowless light source and the width of the shadow after the position is adjusted; Fit the S positions of the shadowless light source and the shadow width after adjusting the position to obtain the shadow The functional relationship between the shadow width and the displacement correction value.

11. The method according to claim 8, wherein: The calculating of the N eigenvalues ​​of the image comprises: The shadow area is obtained according to the number of pixels in the shadow area.

12. The method according to claim 6, wherein: The calculating of the N eigenvalues ​​of the image comprises: Determining a location of a reference point on the target object in the image; Determine a second region of interest in the first region and a third region of interest in the second region on the image according to the reference point position; The brightness difference between the second region of interest and the third region of interest is calculated.

13. The method according to claim 12, wherein: The calculating the brightness difference between the second region of interest and the third region of interest comprises: Determine a first brightness sensitive region in the second region of interest and a second brightness sensitive region in the third region of interest, wherein the brightness sensitive region has a greater reflectivity than the non-brightness sensitive region; The brightness difference between the first brightness sensitive area and the second brightness sensitive area is calculated.

14. The method according to claim 13, wherein: The preset condition includes a second threshold of the brightness difference value, and determining the displacement correction value of the shadowless light source according to the N characteristic values ​​includes: When the brightness difference is greater than the second threshold, the displacement correction value is determined according to a pre-established functional relationship between the brightness difference and the displacement correction value.

15. The method according to claim 14, wherein: Before the image acquisition, pre-establishing the functional relationship between the brightness difference value and the displacement correction value includes: When the brightness difference is greater than or less than the second threshold, adjusting the position of the shadowless light source K times, where K is greater than or equal to 2; Recording the position of the shadowless light source adjusted each time and the brightness difference after adjustment; Fit the K positions of the shadowless light sources and the adjusted brightness difference values ​​to obtain a functional relationship between the brightness difference values ​​and the displacement correction values.

16. The method according to claim 1, wherein: Before taking an image of the target object, it also includes: Acquire current coordinates of a reference point on the target object, wherein the reference point is used to determine a position of at least one of the first area and the second area; When the current coordinates are inconsistent with the preset coordinates, the target object is moved so that the current The coordinates coincide with the preset coordinates.

17. The method according to any one of claims 1 to 16, wherein: The surface shape of the first area is a flat surface, and the surface shape of the second area is an arc-shaped surface, and at least a portion of the arc-shaped surface is higher than the flat surface.

18. The method according to any one of claims 1 to 16, wherein: The target object includes a display device, and the image of the target object is captured including: At least a portion of the pad area of ​​the display device is photographed, wherein at least a portion of the pad area includes a planar area and a bending area of ​​the COF, wherein the first area includes the planar area, and the second area includes the bending area.

19. The method according to any one of claims 1 to 16, wherein: The shadowless light source includes a bowl-shaped light source.

20. An image acquisition device, used to execute the image acquisition method according to any one of claims 1 to 18, comprising: A shadowless light source, located to one side of the target object; An image acquisition device, used for capturing an image of the target object when illuminated by the shadowless light source, wherein the image includes a first area and a second area of ​​the target object, and a surface height of the first area is different from a surface height of the second area; An image processing device, configured to calculate N eigenvalues ​​of the image according to a relative positional relationship between the first area and the second area, where N is greater than or equal to 1; and when at least one eigenvalue among the N eigenvalues ​​does not meet a preset condition, determine a displacement correction value of the shadowless light source according to the at least one eigenvalue; A moving mechanism is connected to the shadowless light source and is used to adjust the position of the shadowless light source according to the displacement correction value, so that the image acquisition device retakes the image and the image processing device recalculates the N eigenvalues ​​until the N eigenvalues ​​meet the preset conditions.

21. A defect detection method, comprising: Obtaining an image of a target object according to the image acquisition method according to any one of claims 1 to 19; The image is processed using a defect detection model to obtain a defect detection result output by the defect detection model.

22. The method according to claim 21, wherein: The target object includes a display device, the image includes at least a portion of a pad area of ​​the display device, the at least a portion of the pad area includes a plane area and a bending area of ​​a chip-on-film, and before processing the image using a defect detection model, preprocessing the image is further included, specifically including: determining a location of a reference point on at least a portion of the pad area in the image; Determine a fourth region of interest according to the reference point position, wherein the fourth region of interest includes a routing area on at least part of the pad area; The fourth region of interest is processed to extract a boundary of the routing line, wherein the defect detection model is configured to perform defect detection on the routing line.