Production line assembly real-time detection method and system using computer vision

Abstract

The application discloses a production line assembly real-time detection method and system applying computer vision, and relates to the technical field of computer vision, and comprises the following steps: generating a lifting saliency map of an FPC assembly part; obtaining an outer contour line based on a bright field image of the FPC assembly part, and performing bilateral extension on the outer contour line to obtain a welt band area; generating a welt enhancement map according to a welt band saliency map corresponding to the welt band area; establishing a local coordinate system based on the outer contour line corresponding to the bright field image; performing binarization and skeletonization processing on the welt enhancement map to obtain a bright ridge center line map, and generating a welt unwound ridge line map according to the local coordinate system corresponding to the bright ridge center line map; obtaining an ordered center line point set according to an unwound connected area corresponding to the welt unwound ridge line map, and calculating a fitting deviation of the ordered center line point set based on a fitting arc corresponding to the ordered center line point set; and the application improves the recognition accuracy of a local edge abnormal area.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for real-time inspection of assembly line using computer vision. Background Technology

[0002] With the continuous development of consumer electronics, small smart terminals, and high-density electronic modules, the application of flexible printed circuit boards (FPCBs) in connector assembly, module interconnection, and precision electronic component assembly is constantly increasing. In actual production, FPCBs usually need to be inserted into ZIF connectors or pressed and positioned with module housings. In order to improve the structural stability of the insertion area, meet the thickness requirements of the connection area, and reduce edge damage caused by repeated insertion and removal, a reinforcing plate is usually attached to the tail end of the FPCB. The edge of the reinforcing plate is usually provided with small chamfers, avoidance notches, or boundary guide structures to limit the insertion direction, avoid interference with the internal structure of the connector, and improve the assembly guidance accuracy. In the process of high-speed automated assembly, when the FPCB enters the connector in the wrong direction, local compression and edge stress are easily generated between the edge of the reinforcing plate and the connector entrance guide surface, resulting in the formation of local outward flipping, edge lifting, or edge curling areas near the edge of the reinforcing plate. Under low-angle lighting conditions, such areas usually appear as local arc-shaped bright ridge areas continuously distributed along the edge of the reinforcing plate.

[0003] In existing technologies, the detection methods for the assembly status of flexible printed circuit boards typically employ image detection methods based on overall shape position, insertion depth, or edge contour matching to judge the assembly results. However, when the flexible printed circuit board has an incorrect orientation but can still enter the connector, the local edge overturning area often only manifests as a local brightness change area close to the edge of the reinforcing plate. Traditional detection methods struggle to effectively distinguish such local continuous arc-shaped bright ridge areas. At the same time, non-directional error factors such as adhesive residue, burrs, indentations, random reflections, and edge shadows can also form local bright areas in the image, leading to misjudgment or missed judgment in existing detection methods, making it difficult to stably output the detection results corresponding to the edge overturning state. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies in that it is difficult to stably output the detection results corresponding to the edge outward state, and to propose a real-time detection method and system for production line assembly using computer vision.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution: Real-time inspection methods for production line assembly using computer vision include: S1. Generate a salient image of the warped FPC assembly; S2. Obtain the outer contour line based on the bright field image of the FPC assembly, and expand the outer contour line on both sides to obtain the edge banding area; generate the edge enhancement map according to the edge banding saliency map corresponding to the edge banding area. S3. Based on the outer contour line corresponding to the bright field image, establish a local coordinate system; perform binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generate the edge unfolded ridge line image according to the local coordinate system corresponding to the bright ridge centerline image. S4. Obtain the ordered centerline point set based on the expanded connected region corresponding to the edge-attached ridge line diagram, and calculate the fitting deviation of the ordered centerline point set based on the fitted arc corresponding to the ordered centerline point set; sort the ordered centerline point set according to the fitting deviation to determine the candidate edge-attached short arc segments. S5. Based on the normal distance coordinate range of the candidate edge-attaching short arc segment, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; and obtain the detection result based on the normal offset direction.

[0006] Preferably, generating a salience map includes: Bright-field images were obtained by performing bright-field illumination acquisition on the FPC assembly. Low-angle dark-field illumination was used to acquire dark-field images of the FPC assembly. A salience map of warping is generated based on the bright field image and the dark field image.

[0007] Preferably, the edge-fitting area includes: Edge extraction is performed on the bright field image to obtain the outer contour edge image; The outer contour line is obtained by extracting the closed contour from the outer contour edge image; The width of the first pixel is determined based on the normal direction of the outer contour line; The width of the second pixel is determined based on the normal direction of the outer contour line; Based on the first pixel width and the second pixel width, the outer contour line is extended on both sides to obtain the edge band area.

[0008] Preferably, generating the edge-fitting enhancement map includes: Based on the edge band area, the raised salience image is cropped to obtain the edge band salience image; A morphological opening operation is performed on the saliency map of the edge band to obtain the opening operation image; An edge enhancement map is generated based on the edge saliency map and the opening operation image.

[0009] Preferably, establishing a local coordinate system includes: Arc length sampling is performed on the outer contour line to obtain multiple outer contour sampling points; Calculate the tangent direction of the outer contour sampling points; Calculate the normal direction of the sampling points on the outer contour; A local coordinate system is established with each outer contour sampling point as the origin, the tangent direction of the outer contour sampling point as the horizontal direction, and the normal direction of the outer contour sampling point as the vertical direction.

[0010] Preferably, generating the edge-unfolded ridge line diagram includes: Binarize the edge enhancement image to obtain an edge enhancement binary image; Connectivity regions were extracted from the edge-enhanced binary image to obtain multiple bright ridge connected regions. The bright ridge connected regions are subjected to skeletonization processing to obtain the bright ridge centerline map; Determine the local coordinate system corresponding to the centerline diagram of the bright ridge; Project each centerline pixel in the bright ridge centerline map onto the local coordinate system corresponding to the bright ridge centerline map to obtain the arc length coordinates and normal distance coordinates of the centerline pixel; Based on the arc length coordinates and normal distance coordinates of the centerline pixel, generate an edge-fitting unfolded ridge map.

[0011] Preferably, calculating the fitting deviation of the ordered centerline point set includes: The connected pixel regions of the unfolded ridge map are extracted to obtain multiple unfolded connected regions. The centerlines of the expanded connected regions are sorted to obtain an ordered set of centerline points. By fitting a circular arc to an ordered set of centerline points, a fitted circular arc is obtained. The fitting deviation of the ordered centerline point set is calculated based on the distance from the centerline pixel point in the ordered centerline point set to the fitted arc.

[0012] Preferably, determining the candidate short arc segment for edge contact includes: Based on the fitting deviation, the ordered centerline point set is sorted to obtain the target centerline point set; The expanded connected region corresponding to the target centerline point set is used as the candidate short arc segment for edge contact.

[0013] Preferably, the detection results include: The range of normal distance coordinates for the candidate short arc segments is determined based on their pixel coordinates in the edge unfolded ridge map. Based on the range of normal distance coordinates, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; Based on the normal offset direction, the candidate edge-fitting short arc segments are judged by reverse bending to obtain the detection result; the detection result is either that there is an edge reverse bending or that there is no edge reverse bending.

[0014] To address the above problems, the present invention also provides a real-time inspection system for production line assembly using computer vision, the system comprising: The salient generation module generates salient images of FPC assemblies; The edge-attaching enhancement module obtains the outer contour line based on the bright-field image of the FPC assembly, and expands the outer contour line on both sides to obtain the edge-attaching strip area; based on the edge-attaching strip saliency map corresponding to the edge-attaching strip area, it generates an edge-attaching enhancement map; The ridge unfolding module establishes a local coordinate system based on the outer contour line corresponding to the bright field image; it performs binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generates the edge unfolded ridge image according to the local coordinate system corresponding to the bright ridge centerline image. The arc segment filtering module obtains an ordered set of centerline points based on the expanded connected regions corresponding to the edge-attached ridge map, and calculates the fitting deviation of the ordered centerline point set based on the fitted arcs corresponding to the ordered centerline point set; according to the fitting deviation, the ordered centerline point set is sorted to determine candidate edge-attached short arc segments. The reverse folding determination module determines the normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line based on the normal distance coordinate range of the candidate edge-attached short arc segment; and obtains the detection result based on the normal offset direction.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. In this invention, by jointly processing the bright field image and the low-angle dark field image, and combining the edge-attaching area corresponding to the edge of the reinforcing plate to generate an edge-attaching enhancement map, the brightness change information corresponding to the local raised area of ​​the edge can be enhanced, while reducing the grayscale interference caused by the flat area, random reflection and background area, thereby improving the recognition stability of the local abnormal area of ​​the reinforcing plate edge and solving the problem that the existing detection method is difficult to accurately locate the edge outward area.

[0016] 2. In this invention, by establishing a local coordinate system corresponding to the outer contour line and converting the bright ridge center line into an edge-attached unfolded ridge line diagram corresponding to the arc length coordinate and normal distance coordinate, the local bright ridge distribution near complex boundaries can be converted into a continuous unfolded trajectory for analysis. At the same time, by combining arc fitting and fitting deviation calculation, candidate edge-attached short arc segments consistent with the continuous arc bright ridge structure can be effectively screened, thereby improving the accuracy of identifying local edge abnormal areas.

[0017] 3. In this invention, by analyzing the normal offset direction of the candidate edge short arc segment relative to the outer contour line, and combining the spatial distribution relationship of the edge local bright ridge area relative to the reinforcing plate boundary for reverse folding determination, it is possible to distinguish between the true edge outward folding state and the local bright area caused by glue marks, burrs, indentations and shadows, thereby improving the reliability of direction error detection and being able to output clear edge reverse folding and warping detection results. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a real-time inspection method for production line assembly using computer vision, provided as an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] This embodiment provides a real-time inspection method for production line assembly using computer vision. See [link to relevant documentation]. Figure 1 Specifically, this includes: S1, generating a salient image of the FPC assembly warping; In an embodiment of the present invention, generating a salience map of warping includes: Bright-field images were obtained by performing bright-field illumination acquisition on the FPC assembly. FPC refers to flexible printed circuit boards; FPC assemblies refer to objects to be inspected that have entered the assembly station and include FPC flexible circuit boards, FPC reinforcing boards, and related connection areas; bright field lighting refers to a lighting method that allows the surface contours and boundaries of FPC assemblies to be uniformly presented; bright field images refer to images of FPC assemblies acquired under bright field lighting conditions. The FPC assembly is fixed at the image acquisition station, ensuring that the edge of the FPC reinforcing plate, the boundary of the FPC flexible substrate, and the interface area between the FPC reinforcing plate and the FPC flexible substrate are within the camera's imaging field of view. The surface of the FPC assembly is illuminated by a bright field illumination source, and the illumination light is reflected off the surface of the FPC assembly and enters the camera imaging unit. The camera focal length is adjusted so that the outer contour edge of the FPC reinforcing plate is clearly presented in the imaging image, triggering the camera to acquire an image of the FPC assembly under the current bright field illumination conditions, and this image is used as the bright field image.

[0021] Low-angle dark-field illumination was used to acquire dark-field images of the FPC assembly. Low-angle dark field illumination refers to a lighting method that illuminates the surface of FPC components at a low incident angle, causing brightness changes in protrusions, warps, or small edge structures; dark field images refer to images of FPC components acquired under low-angle dark field illumination conditions. Keeping the position of the FPC assembly, the position of the camera, the camera focal length, and the imaging field of view unchanged, turn off the bright field illumination source and turn on the low-angle dark field illumination source, so that the illumination light shines on the edge of the FPC reinforcement plate and its adjacent area in an oblique direction, so that the raised, warped, and edge undulation areas on the surface of the FPC assembly generate scattered or reflected light that can be received by the camera, triggering the camera to acquire an image of the FPC assembly under the current low-angle dark field illumination conditions, and using this image as the dark field image.

[0022] A salience map of warping is generated based on the bright field image and the dark field image.

[0023] A salience map is an image generated based on the pixel grayscale difference between a bright field image and a dark field image. It is used to characterize the imaging difference between local salience areas and flat areas on the surface of an FPC assembly.

[0024] Pixel coordinate registration is performed on the bright field image and the dark field image to make the outer contour edge position of the FPC reinforcement plate in the bright field image correspond to the outer contour edge position of the FPC reinforcement plate in the dark field image. The gray value is read pixel by pixel in the registered bright field image and the registered dark field image. The gray value of each pixel position in the dark field image is subtracted from the gray value of the same pixel position in the bright field image and the absolute value is taken. The absolute value of the difference is written to the same pixel position to form a saliency map.

[0025] It should be noted that under bright field lighting conditions, the surface of FPC components mainly forms specular reflection and uniform diffuse reflection. The grayscale difference between flat areas and locally raised areas is small. Under low-angle dark field lighting conditions, the illumination light is incident on the surface of FPC components at an oblique direction. The reflected light in the flat areas deviates from the camera's receiving direction, while the propagation direction of reflected and scattered light changes in locally raised areas, edge undulation areas, and areas with changes in surface height. This allows the corresponding areas to return more reflected energy to the camera, thus forming a grayscale value that is significantly higher than the surrounding areas in the dark field image. Therefore, after calculating the absolute value of the grayscale difference at the same pixel position in the dark field image and the bright field image, the background grayscale influence of the uniformly flat areas can be weakened, and the grayscale change area corresponding to the locally raised areas can be enhanced. This allows the brightness change caused by the locally raised areas to be concentrated and retained in the result image, thus forming a saliency map for characterizing the locally raised areas of the FPC components.

[0026] S2. Obtain the outer contour line based on the bright field image of the FPC assembly, and expand the outer contour line on both sides to obtain the edge banding area; generate the edge enhancement map according to the edge banding saliency map corresponding to the edge banding area. In an embodiment of the present invention, the edge-fitting area is obtained, including: Edge extraction is performed on the bright field image to obtain the outer contour edge image; The bright-field image is converted to a grayscale image. The grayscale image is then smoothed to reduce the impact of isolated noise on edge positions. The horizontal and vertical grayscale changes are calculated on the smoothed grayscale image. The gradient magnitude of each pixel position is calculated based on the horizontal and vertical grayscale changes. The edge pixels corresponding to the outer boundary of the FPC reinforcement plate are determined based on the gradient magnitude. The edge pixels are written into a blank image of the same size as the bright-field image to obtain the outer contour edge image.

[0027] The outer contour line is obtained by extracting the closed contour from the outer contour edge image; The outer contour edge image refers to the image obtained after edge extraction of the bright field image, which is used to present the pixel distribution of the outer boundary of the FPC reinforcement plate in the image; closed contour extraction refers to the process of connecting the edge pixels of the same FPC reinforcement plate boundary in the outer contour edge image to form a continuous closed boundary; the outer contour line refers to the outer boundary line of the FPC reinforcement plate obtained by closed contour extraction; the normal direction refers to the direction perpendicular to the tangent of the outer contour line at a certain position. Connectivity analysis is performed on edge pixels in the outer contour edge image. Adjacent edge pixels that belong to the same boundary direction are grouped into edge connected segments. Endpoints are connected at the break points in the edge connected segments. Hole filling and contour closure are performed on the connected edge connected segments. A closed contour is selected from the closed contour to form the boundary of the FPC reinforcement plate solid area. The pixel positions of the closed contour are arranged according to the boundary direction to obtain the outer contour line.

[0028] The width of the first pixel is determined based on the normal direction of the outer contour line; The tangent direction of each outer contour line pixel is determined based on the positional changes of adjacent pixels in the outer contour line. The normal direction perpendicular to the tangent direction is determined based on the tangent direction. The side of the normal direction facing the FPC reinforcing plate solid area is determined based on the grayscale distribution position of the FPC reinforcing plate solid area and the FPC flexible substrate area in the bright field image. The search is performed pixel by pixel along the side facing the FPC reinforcing plate solid area until the grayscale of the search position belongs to the grayscale distribution inside the FPC reinforcing plate solid area. The pixel distance between the outer contour line pixel and the search position is taken as the first local width corresponding to the outer contour line pixel. The first pixel width is generated based on the first local width corresponding to each outer contour line pixel.

[0029] The width of the second pixel is determined based on the normal direction of the outer contour line; The first pixel width refers to the pixel distance that the outer contour line extends towards the solid area of ​​the FPC reinforcing plate; the second pixel width refers to the pixel distance that the outer contour line extends towards the flexible substrate area of ​​the FPC. The tangent direction of each outer contour pixel is determined based on the positional changes of adjacent pixels in the outer contour. The normal direction perpendicular to the tangent direction is then determined based on the tangent direction. The side of the normal direction facing the FPC flexible substrate region is determined based on the grayscale distribution of the FPC reinforcing plate solid region and the FPC flexible substrate region in the bright field image. The search proceeds pixel by pixel along the side facing the FPC flexible substrate region until the grayscale of the search position belongs to the grayscale distribution inside the FPC flexible substrate region. The pixel distance between the outer contour pixel and the search position is taken as the second local width corresponding to the outer contour pixel. The second pixel width is generated based on the second local width corresponding to each outer contour pixel.

[0030] Based on the first pixel width and the second pixel width, the outer contour line is extended on both sides to obtain the edge band area.

[0031] Double-sided expansion refers to the process of generating extended areas centered on the outer contour line, extending to both the solid area of ​​the FPC reinforcing plate and the flexible substrate area of ​​the FPC. The edge-attached area refers to the image area defined by the first pixel width and the second pixel width and continuously distributed along the outer contour line.

[0032] Using each outer contour pixel as the starting point for expansion, inner expansion pixels are marked along the side facing the FPC reinforcing plate solid area according to the first pixel width corresponding to the outer contour pixel, and outer expansion pixels are marked along the side facing the FPC flexible substrate area according to the second pixel width corresponding to the outer contour pixel. All inner expansion pixels, outer expansion pixels and outer contour pixels corresponding to the outer contour pixels are merged to obtain the edge strip area continuously distributed along the edge of the FPC reinforcing plate.

[0033] In an embodiment of the present invention, generating an edge-fitting enhancement map includes: Based on the edge band area, the raised salience image is cropped to obtain the edge band salience image; Region cropping refers to retaining the pixel region corresponding to the edge band region in the warping saliency map according to the spatial distribution of the edge band region, and removing the pixel region outside the edge band region, so that the subsequent image processing range is limited to the area near the edge of the FPC reinforcement plate; the edge band saliency map is an image composed of the pixel grayscale values ​​located in the edge band region of the warping saliency map. The pixel grayscale values ​​in the edge band saliency map are used to represent the degree of brightness change of the area near the edge of the FPC reinforcement plate under dark field lighting conditions relative to bright field lighting conditions; Create a region labeling image with the same size as the raised saliency map. Mark all pixel positions in the edge band region as reserved region pixels, and mark all pixel positions outside the edge band region as non-reserved region pixels. Iterate through each pixel position in the raised saliency map. When the pixel position corresponding to the current pixel position in the region labeling image belongs to the reserved region pixel, write the gray value of the current pixel position in the raised saliency map to the corresponding pixel position in the output image. When the pixel position corresponding to the current pixel position in the region labeling image belongs to the non-reserved region pixel, write the gray value of the corresponding pixel position in the output image to zero gray value. By copying the gray values ​​of all pixel positions and clearing the gray values ​​of the non-reserved regions, the edge band saliency map is obtained.

[0034] A morphological opening operation is performed on the saliency map of the edge band to obtain the opening operation image; Morphological opening refers to the image processing procedure of shape shrinkage and shape restoration of bright areas in the edge band saliency map. Shape shrinkage is used to remove small isolated bright areas and narrowly connected bright areas, while shape restoration is used to maintain the overall distribution range of continuous areas. The opening image refers to the image obtained after the edge band saliency map has undergone morphological opening. The continuous bright areas in the opening image represent a large range of brightness distribution areas near the edge of the FPC reinforcement plate. Linear structural elements are constructed based on the extension direction of the edge band region along the edge of the FPC reinforcing plate. Each structural element pixel in the linear structural element is continuously arranged in the same direction. The linear structural elements are moved pixel by pixel in the edge band saliency map. At each moving position, all pixel grayscale values ​​within the coverage area of ​​the linear structural element are read. The minimum grayscale value within the coverage area is selected as the output grayscale value of the current pixel position after erosion processing. The eroded image is obtained by eroding all pixel positions. Then, the linear structural elements are moved pixel by pixel in the eroded image. At each moving position, all pixel grayscale values ​​within the coverage area of ​​the linear structural element are read. The maximum grayscale value within the coverage area is selected as the output grayscale value of the current pixel position after dilation processing. The opening operation image is obtained by dilation processing all pixel positions.

[0035] An edge enhancement map is generated based on the edge saliency map and the opening operation image.

[0036] Edge enhancement image refers to an image obtained by comparing the grayscale difference at the same pixel position between the edge saliency image and the opening operation image. The high grayscale area in the edge enhancement image is used to represent the local brightness change area near the edge of the FPC reinforcement plate due to local warping, local undulation, or edge flipping.

[0037] An enhanced image with the same size as the edge-fitting saliency map is created. Gray-scale traversal is performed on each pixel position in the edge-fitting saliency map and the corresponding pixel position in the opening operation image. The original gray-scale value of the current pixel position in the edge-fitting saliency map is read, and the background gray-scale value of the same pixel position in the opening operation image is read. The original gray-scale value is subtracted from the background gray-scale value to obtain the gray-scale difference value corresponding to the current pixel position. When the gray-scale difference value is less than zero, the gray-scale value of the current pixel position in the enhanced image is written as zero gray-scale value. When the gray-scale difference value is greater than or equal to zero, the gray-scale difference value is written to the current pixel position in the enhanced image. The edge-fitting enhanced image is obtained by calculating the gray-scale difference values ​​of all pixel positions.

[0038] It should be noted that under low-angle dark-field illumination, the local raised areas, local flipped areas, and edge undulations near the edge of the FPC reinforcement plate will generate strong scattered and reflected light, making the corresponding areas appear as locally high grayscale areas in the raised saliency image. However, the large-scale flat areas, continuous background areas, and slowly changing areas near the edge of the FPC reinforcement plate have spatial continuity. Therefore, when performing morphological opening operations on the edge saliency image, the erosion process will preferentially remove small-area locally bright areas and narrowly connected bright areas, and the dilation process will restore the overall grayscale distribution of the continuous background area. Thus, the opening operation image mainly retains the large-scale background brightness distribution near the edge of the FPC reinforcement plate. When the edge saliency image is subtracted from the opening operation image, the grayscale difference between continuous background areas tends to decrease, while the locally raised areas, locally flipped areas, and edge undulations corresponding to the locally high grayscale areas still maintain a large grayscale difference. Therefore, the resulting edge enhancement image can highlight the local brightness change areas near the edge of the FPC reinforcement plate caused by local height changes.

[0039] S3. Based on the outer contour line corresponding to the bright field image, establish a local coordinate system; perform binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generate the edge unfolded ridge line image according to the local coordinate system corresponding to the bright ridge centerline image. In an embodiment of the present invention, establishing a local coordinate system includes: Arc length sampling is performed on the outer contour line to obtain multiple outer contour sampling points; Arc length sampling refers to the position discretization processing along the extension direction of the outer contour line according to the continuous boundary path to obtain discrete boundary points that can characterize the spatial orientation of the outer contour line; outer contour sampling points refer to the discrete pixel positions of the boundary obtained after the outer contour line has undergone arc length sampling. The outer contour obtained by extracting the closed contour is represented as a sequence of boundary pixels arranged continuously according to the boundary direction. The starting boundary pixel in the boundary pixel sequence is taken as the arc length starting point, and the arc length value of the arc length starting point is recorded as zero. Each boundary pixel is processed sequentially along the arrangement direction of the boundary pixel sequence. The distance between the pixel coordinate difference between the current boundary pixel and the previous boundary pixel is calculated. This distance is accumulated to the arc length value corresponding to the previous boundary pixel to obtain the arc length value corresponding to the current boundary pixel. The boundary pixels with arc length values ​​are taken as arc length boundary pixels. Among the arc length boundary pixels, boundary pixels used to represent the geometric direction of the outer contour are selected in ascending order of arc length value to obtain multiple outer contour sampling points. The image coordinates and arc length value of each outer contour sampling point are retained.

[0040] Calculate the tangent direction of the outer contour sampling points; For any outer contour sampling point, determine the nearest forward boundary pixel whose arc length is less than that of the outer contour sampling point, and determine the nearest backward boundary pixel whose arc length is greater than that of the outer contour sampling point. Obtain the image coordinates of the forward and backward boundary pixels. Subtract the abscissa of the forward boundary pixel from the abscissa of the backward boundary pixel to obtain the lateral coordinate difference. Subtract the ordinate of the forward boundary pixel from the ordinate of the backward boundary pixel to obtain the lateral coordinate difference. Form a boundary extension vector from the lateral and lateral coordinate differences. Calculate the length of the boundary extension vector. Divide the lateral coordinate difference by the length of the boundary extension vector to obtain the lateral component of the tangent direction. Divide the lateral coordinate difference by the length of the boundary extension vector to obtain the lateral component of the tangent direction. Obtain the tangent direction of the outer contour sampling point.

[0041] Calculate the normal direction of the sampling points on the outer contour; The tangent direction refers to the direction in which the outer contour line extends along the boundary at a certain outer contour sampling point; the normal direction refers to the direction perpendicular to the tangent direction at the corresponding outer contour sampling point; the transverse direction refers to the direction in the local coordinate system that extends along the boundary of the outer contour line; the longitudinal direction refers to the direction in the local coordinate system that extends perpendicular to the boundary of the outer contour line. For any outer contour sampling point, obtain the lateral and longitudinal components of the tangent direction of the outer contour sampling point. Take the negative of the longitudinal component of the tangent direction as the lateral component of the candidate normal direction, and take the lateral component of the tangent direction as the longitudinal component of the candidate normal direction to obtain the candidate normal direction perpendicular to the tangent direction. Take the candidate normal direction and the opposite direction of the candidate normal direction as two normal directions to be judged. Take the adjacent pixel positions from the outer contour sampling point to both sides along the two normal directions to be judged. According to the gray distribution of the adjacent pixel positions on both sides in the bright field image and the positional relationship of the area enclosed by the outer contour line, determine the side facing the FPC reinforcing plate solid area and the side facing the FPC flexible substrate area. The side facing the FPC flexible substrate area is determined as the outer normal direction, and the side facing the FPC reinforcing plate solid area is determined as the inner normal direction.

[0042] A local coordinate system is established with each outer contour sampling point as the origin, the tangent direction of the outer contour sampling point as the horizontal direction, and the normal direction of the outer contour sampling point as the vertical direction.

[0043] The local coordinate system refers to a two-dimensional directional coordinate space established with the outer contour sampling points as the position reference and the corresponding tangent direction and normal direction. It is used to characterize the boundary extension relationship and normal offset relationship of the area near the edge of the FPC reinforcement plate.

[0044] For any outer contour sampling point, the image coordinates of the outer contour sampling point are taken as the origin of the local coordinate system, the tangent direction of the outer contour sampling point is taken as the positive horizontal direction of the local coordinate system, and the outer normal direction of the outer contour sampling point facing the FPC flexible substrate area is taken as the positive vertical direction of the local coordinate system. The image coordinates of any pixel to be located in the image plane are subtracted from the image coordinates of the outer contour sampling point to obtain the relative position vector. The projection value of the relative position vector in the tangent direction is taken as the horizontal coordinate of the pixel to be located in the local coordinate system, and the projection value of the relative position vector in the outer normal direction is taken as the vertical coordinate of the pixel to be located in the local coordinate system. Thus, a local coordinate system is obtained with the outer contour sampling point as the position reference and corresponding to the local direction of the outer contour line.

[0045] In an embodiment of the present invention, generating an edge-wrapped ridge map includes: Binarize the edge enhancement image to obtain an edge enhancement binary image; Binarization refers to the image processing process of converting the pixel grayscale state in the edge enhancement image into two types of pixel states: bright area pixels and background area pixels. The edge enhancement binary image is the image obtained after the edge enhancement image has been binarized, where bright area pixels correspond to local high grayscale areas in the edge enhancement image, and background area pixels correspond to non-local high grayscale areas in the edge enhancement image. The grayscale distribution of all pixels in the edge enhancement image is statistically analyzed. Based on the grayscale distribution, the pixels are divided into bright area pixels and background area pixels. An edge enhancement binary image with the same size as the edge enhancement image is generated. When any pixel position in the edge enhancement image is classified as a bright area pixel, the same pixel position in the edge enhancement binary image is written as a bright area marker. When any pixel position in the edge enhancement image is classified as a background area pixel, the same pixel position in the edge enhancement binary image is written as a background area marker. This results in an edge enhancement binary image used to represent the position of local high grayscale areas.

[0046] Connectivity regions were extracted from the edge-enhanced binary image to obtain multiple bright ridge connected regions. Connected component extraction refers to the process of grouping pixels that are adjacent in position and belong to the same bright region in the edge-attached enhanced binary image into the same pixel set; a bright ridge connected region refers to a continuous pixel set composed of adjacent bright region pixels, used to represent the bright ridge region continuously distributed near the edge of the FPC reinforcement plate in the edge-attached enhanced image. Bright region pixels in the edge-enhanced binary image are taken as pixels to be merged. Starting from any unmerged bright region pixel, check the positions of pixels that belong to bright region pixels among the surrounding pixels of the bright region pixel. Merge the adjacent bright region pixels into the same pixel set, and continue to check the bright region pixels among the surrounding pixels of the newly merged pixel until there are no more adjacent bright region pixels that can be merged into the pixel set. Record the pixel set as a bright ridge connected region. Repeat the same process for the unmerged bright region pixels in the edge-enhanced binary image to obtain multiple bright ridge connected regions.

[0047] The bright ridge connected regions are subjected to skeletonization processing to obtain the bright ridge centerline map; Skeletonization refers to the image processing process of gradually refining the bright ridge connected regions into a set of centerline pixels while maintaining the extension direction of the bright ridge connected regions; the bright ridge centerline map refers to the image obtained after skeletonization of the bright ridge connected regions, where the centerline pixels represent the middle extension position of the bright ridge connected regions. Each bright ridge connected region is treated as an independent processing object. The boundary pixels and internal pixels of the bright ridge connected region are determined. The boundary pixels of the bright ridge connected region are deleted in round by round. In each round of deletion, only the boundary pixels that will not break the bright ridge connected region are deleted, while the internal pixels that can represent the extension path of the bright ridge connected region are retained, until the bright ridge connected region is compressed to a single pixel width in the width direction or the boundary pixels can no longer be deleted, and the bright ridge centerline map composed of centerline pixels is obtained.

[0048] Determine the local coordinate system corresponding to the centerline diagram of the bright ridge; Obtain each centerline pixel in the bright ridge centerline map. Determine the outer contour sampling point closest to the centerline pixel among the outer contour sampling points. Use the local coordinate system corresponding to the outer contour sampling point as the local coordinate system corresponding to the centerline pixel. The local coordinate system takes the corresponding outer contour sampling point as the origin, the tangent direction of the corresponding outer contour sampling point as the horizontal direction, and the normal direction of the corresponding outer contour sampling point as the vertical direction.

[0049] Project each centerline pixel in the bright ridge centerline map onto the local coordinate system corresponding to the bright ridge centerline map to obtain the arc length coordinates and normal distance coordinates of the centerline pixel; The centerline pixel refers to the pixel position belonging to the centerline in the bright ridge centerline image; the arc length coordinate refers to the position coordinates of the centerline pixel after it is projected onto the local coordinate system corresponding to the outer contour line and measured along the extension direction of the outer contour line; the normal distance coordinate refers to the position coordinates of the centerline pixel after it is projected onto the local coordinate system corresponding to the outer contour line and measured along the normal direction of the outer contour line. Obtain the image coordinates of the centerline pixel in the bright ridge centerline image. Obtain the image coordinates, tangent direction, and normal direction of the corresponding outer contour sampling point. Subtract the image coordinates of the outer contour sampling point from the image coordinates of the centerline pixel to obtain the relative position vector. Add the projection result of the relative position vector in the tangent direction to the arc length value of the outer contour sampling point to obtain the arc length coordinates of the centerline pixel. Use the projection result of the relative position vector in the normal direction as the normal distance coordinates of the centerline pixel.

[0050] Based on the arc length coordinates and normal distance coordinates of the centerline pixel, generate an edge-fitting unfolded ridge map.

[0051] The edge-mounted unfolded ridge line diagram refers to an image generated with arc length coordinates as the horizontal position and normal distance coordinates as the vertical position, used to represent the unfolded distribution of the bright ridge center line relative to the outer contour line of the FPC reinforcement plate.

[0052] Create a blank image with arc length coordinates as the horizontal coordinates and normal distance coordinates as the vertical coordinates. Write each centerline pixel into the corresponding position in the blank image according to its arc length coordinates and normal distance coordinates. Set the pixel state of the written position as the bright ridge pixel state and the pixel state of the unwritten position as the background pixel state to obtain the edge-fitted unfolded ridge map.

[0053] S4. Obtain the ordered centerline point set based on the expanded connected region corresponding to the edge-attached ridge line diagram, and calculate the fitting deviation of the ordered centerline point set based on the fitted arc corresponding to the ordered centerline point set; sort the ordered centerline point set according to the fitting deviation to determine the candidate edge-attached short arc segments. In an embodiment of the present invention, calculating the fitting deviation of the ordered centerline point set includes: The connected pixel regions of the unfolded ridge map are extracted to obtain multiple unfolded connected regions. Connected pixel region extraction refers to the process of grouping bright ridge pixels with continuous horizontal coordinates and continuously changing vertical coordinates into the same pixel region in the edge-mounted ridge map, based on the spatial adjacency relationship between the positions of bright ridge pixels. The pixel region obtained by connected pixel region extraction is used to represent the continuous extension area of ​​the same bright ridge in the unfolded coordinates. The unfolded connected region refers to a pixel set composed of multiple spatially continuously distributed bright ridge pixels. The pixel positions in the unfolded connected region correspond to the spatial distribution range of the same local bright ridge structure near the edge of the FPC reinforcement plate in the unfolded coordinates. The bright ridge pixels in the edge-to-edge expanded ridge graph are taken as pixels to be processed. Starting from any unmerged bright ridge pixel, check whether there are other bright ridge pixels in the horizontal, vertical, and diagonal adjacent positions. The adjacent bright ridge pixels are grouped into the same pixel set, and the adjacent position check is continued for the newly grouped bright ridge pixels until there are no bright ridge pixels that can be further grouped into the pixel set. The pixel set is recorded as an expanded connected region. The same process is repeated for the bright ridge pixels in the edge-to-edge expanded ridge graph that have not yet been merged, resulting in multiple expanded connected regions.

[0054] The centerlines of the expanded connected regions are sorted to obtain an ordered set of centerline points. Centerline sorting refers to the process of arranging centerline pixels continuously according to the order of changes in the arc length coordinates of each centerline pixel in the expanded connected region; ordered centerline point set refers to the set of centerline pixels formed after centerline sorting. The centerline pixels in the ordered centerline point set are arranged continuously according to the extension direction of the bright ridge, which is used to represent the continuous extension trajectory of the bright ridge in the expanded coordinates. Treat each expanded connected region as an independent processing object, obtain the arc length coordinates and normal distance coordinates of all centerline pixels in the expanded connected region, determine the extension direction of the expanded connected region according to the increasing direction of the arc length coordinates, arrange all centerline pixels in ascending order of arc length coordinates, and arrange the corresponding centerline pixels in ascending order of normal distance coordinates when the arc length coordinates are the same, thus obtaining the ordered centerline point set corresponding to the expanded connected region.

[0055] By fitting a circular arc to an ordered set of centerline points, a fitted circular arc is obtained. Arc fitting refers to the process of calculating a circular arc trajectory that can represent the overall curvature of the centerline based on the arc length coordinates and normal distance coordinates of each centerline pixel in an ordered centerline point set; fitted arc refers to the circular arc trajectory obtained by arc fitting, and the fitted arc is used to represent the overall curvature state of the ordered centerline point set in the unfolded coordinates. Read all centerline pixels in the ordered centerline point set sequentially according to their arc length coordinates. Obtain the arc length coordinates and normal distance coordinates corresponding to each centerline pixel. Use the arc length coordinates as the horizontal coordinates in the two-dimensional plane and the normal distance coordinates as the vertical coordinates in the two-dimensional plane to establish a centerline point coordinate set. Select three non-collinear centerline pixels from the centerline point coordinate set. Calculate the center position and radius of the corresponding circle based on the positional relationship of the three centerline pixels. Use this circle as the current candidate circle. Calculate the total distance from all centerline pixels in the centerline point coordinate set to the current candidate circle. Repeatedly select three non-collinear centerline pixels at different positions and repeat the calculation of the center position, radius, and total distance. Determine the candidate circle with the smallest total distance as the fitted arc. Based on the corresponding positions of the starting and ending centerline pixels in the ordered centerline point set on the fitted arc, cut off the arc interval to obtain the fitted arc corresponding to the ordered centerline point set.

[0056] The fitting deviation of the ordered centerline point set is calculated based on the distance from the centerline pixel point in the ordered centerline point set to the fitted arc.

[0057] Fitting deviation refers to the difference in distance between each centerline pixel in the ordered centerline point set and the fitted arc. Fitting deviation is used to represent the degree of spatial fit between the ordered centerline point set and the fitted arc. The smaller the fitting deviation, the closer the ordered centerline point set is to a continuous arc distribution.

[0058] Obtain all centerline pixels in the ordered centerline point set. Obtain the arc length coordinates and normal distance coordinates corresponding to each centerline pixel. Use the arc length coordinates and normal distance coordinates of each centerline pixel as the planar coordinates of the centerline pixel in the edge-expanded ridge line map. Obtain the center coordinates, radius, start point, and end point of the fitted arc. For any centerline pixel, calculate the Euclidean distance between the centerline pixel and the center of the fitted arc. Take the absolute value of the difference between the Euclidean distance and the radius of the fitted arc to obtain the distance from the centerline pixel to the fitted arc. Repeat the above distance calculation for all centerline pixels in the ordered centerline point set to obtain the distance values ​​corresponding to all centerline pixels. Sum all the distance values ​​to obtain the total distance. Count the number of centerline pixels in the ordered centerline point set. Divide the total distance by the number of centerline pixels to obtain the fitting deviation of the ordered centerline point set.

[0059] It should be noted that the local upturned areas, local warped areas, and edge undulation areas formed near the edge of the FPC reinforcing plate will form a continuous curved spatial extension trajectory along the edge of the reinforcing plate when subjected to the pressure of the connector edge, the constraint of the reinforcing plate edge, and the bending deformation of the flexible substrate. This spatial extension trajectory is represented as a continuously changing curved bright ridge distribution along the arc length in the edge unfolding ridge line diagram. Since the arc is a geometric shape with a stable curvature distribution in the continuous curved trajectory, when approximating the ordered centerline point set by fitting the arc, if the centerline pixel distribution in the ordered centerline point set is consistent with the continuous curved trajectory formed by the local upturned areas, the distance change between each centerline pixel and the fitted arc is small. Conversely, if the centerline pixel distribution in the ordered centerline point set is affected by random reflections, isolated bright spots, or discontinuous edge disturbances, the distance change between each centerline pixel and the fitted arc increases. Therefore, the distance change result obtained by statistically calculating the distance between all centerline pixels and the fitted arc can characterize the degree of fit between the ordered centerline point set and the continuous arc trajectory, thus forming a fitting deviation used to represent the bending consistency.

[0060] In an embodiment of the present invention, determining candidate short arc segments for edge contact includes: Based on the fitting deviation, the ordered centerline point set is sorted to obtain the target centerline point set; Sorting refers to the process of arranging multiple ordered centerline point sets in order based on the magnitude of the fitting deviation corresponding to them; the target centerline point set refers to the ordered centerline point set with the smallest fitting deviation among the multiple ordered centerline point sets, and the centerline pixel distribution in the target centerline point set has a high degree of spatial fit with the fitted arc. Obtain the fitting deviation corresponding to each ordered centerline point set, establish a correspondence between each ordered centerline point set and its corresponding fitting deviation, arrange all ordered centerline point sets in ascending order of fitting deviation, determine the ordered centerline point set at the top of the list as the target centerline point set, and retain the expanded connected region identifier corresponding to the target centerline point set.

[0061] The expanded connected region corresponding to the target centerline point set is used as the candidate short arc segment for edge contact.

[0062] Candidate edge-attached short arc segments refer to the locally continuous curved regions formed by the expanded connected regions corresponding to the target centerline point set. Candidate edge-attached short arc segments are used to represent the locally curved bright ridge regions formed near the edge of the FPC reinforcement plate due to local upturning, local warping, or edge curling.

[0063] Based on the expanded connected region identifier corresponding to the target centerline point set, an expanded connected region consistent with the source of the target centerline point set is determined among multiple expanded connected regions. All bright ridge pixels in the expanded connected region and their corresponding arc length coordinates and normal distance coordinates are used as the data content of candidate edge-attached short arc segments to obtain candidate edge-attached short arc segments for subsequent refraction determination.

[0064] It should be noted that the bright ridge pixels in the expanded connected region corresponding to the target centerline point set originate from the same continuous bright ridge structure, and the target centerline point set corresponds to the smallest fitting deviation among all ordered centerline point sets, indicating that the distribution of centerline pixels in the target centerline point set has the highest spatial fit with the continuous arc trajectory. The bright ridge regions near the edge of the FPC reinforcement plate, formed by local flipping, local lifting, or edge rollback, usually exhibit a continuous short arc distribution along the edge direction. Therefore, the expanded connected region corresponding to the target centerline point set can more accurately represent the local continuous arc bright ridge region, thus determining the expanded connected region corresponding to the target centerline point set as the candidate edge-fitting short arc segment.

[0065] S5. Based on the normal distance coordinate range of the candidate edge-attaching short arc segment, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; and obtain the detection result based on the normal offset direction.

[0066] In embodiments of the present invention, obtaining the detection results includes: The range of normal distance coordinates for the candidate short arc segments is determined based on their pixel coordinates in the edge unfolded ridge map. Obtain the pixel coordinates of all bright ridge pixels in the candidate edge-attached short arc segment in the edge-attached unfolded ridge line map. Take the vertical coordinate of each bright ridge pixel as the normal distance coordinate of the corresponding bright ridge pixel. Compare the normal distance coordinates of all bright ridge pixels to determine the minimum and maximum normal distance coordinates among all normal distance coordinates. Take the continuous coordinate range between the minimum and maximum normal distance coordinates as the normal distance coordinate range of the candidate edge-attached short arc segment.

[0067] Based on the range of normal distance coordinates, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; Pixel coordinates refer to the image position of the bright ridge pixels in the candidate edge-attached short arc segment in the edge-attached unfolded ridge line diagram; normal distance coordinate range refers to the range of values ​​formed by the normal distance coordinates corresponding to all bright ridge pixels in the candidate edge-attached short arc segment, used to represent the longitudinal distribution position of the candidate edge-attached short arc segment relative to the outer contour line; normal offset direction refers to the direction in which the candidate edge-attached short arc segment deviates from the outer contour line along the normal direction towards the solid area of ​​the FPC reinforcing plate or towards the flexible FPC substrate area. Using the unfolded position of the outer contour line in the edge-attached ridge line diagram as the zero position of the normal distance coordinate, each normal distance coordinate in the normal distance coordinate range is compared with the zero position. When the normal distance coordinate range is located on the side of the zero position facing the FPC flexible substrate area, the normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line is determined as the outer offset direction. When the normal distance coordinate range is located on the side of the zero position facing the FPC reinforcing plate solid area, the normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line is determined as the inner offset direction.

[0068] Based on the normal offset direction, the candidate edge-fitting short arc segments are judged by reverse bending to obtain the detection result; the detection result is either that there is an edge reverse bending or that there is no edge reverse bending.

[0069] The reverse fold judgment refers to the process of determining whether the candidate edge short arc segment corresponds to the reverse fold and warping state at the edge of the reinforcing plate based on the normal offset direction of the candidate edge short arc segment; the detection result refers to the final judgment result of the edge state of the reinforcing plate in the FPC assembly; the edge reverse fold and warping refers to the local outward warping area located near the edge of the FPC reinforcing plate and continuously distributed along the boundary direction. The area corresponding to the edge reverse fold and warping has height changes and boundary bending changes relative to the surrounding flat area. In terms of spatial morphology, the edge reverse fold and warping is a local arc-shaped warping structure extending along the edge direction of the reinforcing plate, and there is a continuous normal offset relationship between the area corresponding to the edge reverse fold and warping and the outer contour line.

[0070] The normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line is compared with the outer normal direction of the outer contour line of the FPC reinforcing plate. When the normal offset direction of the candidate edge-attached short arc segment is the outer offset direction towards the FPC flexible substrate area, it is determined that the local arc-shaped bright ridge area corresponding to the candidate edge-attached short arc segment is located outside the outer contour line of the FPC reinforcing plate and forms an outward flip state. The detection result is that there is an edge folding and curling. When the normal offset direction of the candidate edge-attached short arc segment is the inner offset direction towards the solid area of ​​the FPC reinforcing plate, it is determined that the local arc-shaped bright ridge area corresponding to the candidate edge-attached short arc segment does not form an outward flip state. The detection result is that there is no edge folding and curling.

[0071] It should be noted that the local upturned areas, local warped areas, and edge rollback areas formed near the edge of the FPC reinforcing plate will cause the corresponding bright ridge area to spatially shift along the outer normal direction of the outer contour line. The normal distance coordinate in the edge unfolding ridge line diagram is used to characterize the spatial positional relationship of the bright ridge area relative to the outer contour line. When the normal offset direction corresponding to the candidate edge short arc segment is towards the outer direction of the FPC flexible substrate area, it indicates that the corresponding bright ridge area has crossed the outer contour line of the FPC reinforcing plate and formed an arc-shaped upturned state. When the normal offset direction corresponding to the candidate edge short arc segment is towards the inner direction of the FPC reinforcing plate solid area, it indicates that the corresponding bright ridge area is still located inside the outer contour line of the FPC reinforcing plate and has not formed an outward flipped state. Therefore, the edge upturned state and non-upturned state can be distinguished by the normal offset direction of the candidate edge short arc segment relative to the outer contour line, thereby obtaining the detection result corresponding to the edge folding and warping.

[0072] It should be noted that the candidate edge-attachment short arc segment can only characterize a bright ridge area with localized continuous curvature near the edge of the FPC reinforcing plate. In actual assembly, burrs, adhesive residue, indentations, random reflections, and edge shadows may also form localized arc-shaped bright ridge areas. The presence of the candidate edge-attachment short arc segment alone cannot directly determine whether an edge flaring state caused by a directional error has occurred. The localized bright ridge area corresponding to the edge folding and warping will form a clear outward normal offset relationship relative to the outer contour line. By performing a folding and folding judgment on the candidate edge-attachment short arc segment, it is possible to further confirm whether the candidate edge-attachment short arc segment corresponds to the localized arc-shaped warping area formed by the edge folding of the FPC reinforcing plate. This distinguishes the edge folding and warping caused by a real directional error from the localized bright ridge area caused by ordinary reflections, adhesive marks, or indentations, and outputs a clear detection result.

[0073] This solution demonstrates the automatic recognition process of the spatial state of target objects by visually acquiring, analyzing, and judging the image information corresponding to FPC assemblies. It acquires image data of the edge area of ​​the FPC reinforcement plate through bright field illumination and low-angle dark field illumination, realizing the collection of visual information of the edge area. Through edge extraction, bright ridge enhancement, skeletonization processing, and local coordinate unfolding, it realizes the visual structural analysis of the local morphology of the FPC reinforcement plate edge. Through arc fitting, fitting deviation analysis, and normal offset direction determination, it realizes the visual understanding and state recognition of the spatial distribution state of the local bright ridge area of ​​the edge.

[0074] One embodiment of the present invention also provides a real-time inspection system for production line assembly using computer vision; in this embodiment, the functions of each module / unit are as follows: The salient generation module generates salient images of FPC assemblies; The edge-attaching enhancement module obtains the outer contour line based on the bright-field image of the FPC assembly, and expands the outer contour line on both sides to obtain the edge-attaching strip area; based on the edge-attaching strip saliency map corresponding to the edge-attaching strip area, it generates an edge-attaching enhancement map; The ridge unfolding module establishes a local coordinate system based on the outer contour line corresponding to the bright field image; it performs binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generates the edge unfolded ridge image according to the local coordinate system corresponding to the bright ridge centerline image. The arc segment filtering module obtains an ordered set of centerline points based on the expanded connected regions corresponding to the edge-attached ridge map, and calculates the fitting deviation of the ordered centerline point set based on the fitted arcs corresponding to the ordered centerline point set; according to the fitting deviation, the ordered centerline point set is sorted to determine candidate edge-attached short arc segments. The reverse folding determination module determines the normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line based on the normal distance coordinate range of the candidate edge-attached short arc segment; and obtains the detection result based on the normal offset direction.

[0075] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A real-time inspection method for production line assembly using computer vision, characterized in that, The steps include: S1, generating a significant image of the warping of the FPC assembly; S2. Obtain the outer contour line based on the bright field image of the FPC assembly, and expand the outer contour line on both sides to obtain the edge banding area; generate the edge enhancement map according to the edge banding saliency map corresponding to the edge banding area. S3. Based on the outer contour line corresponding to the bright field image, establish a local coordinate system; perform binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generate the edge unfolded ridge line image according to the local coordinate system corresponding to the bright ridge centerline image. S4. Obtain the ordered centerline point set based on the expanded connected region corresponding to the edge-attached ridge line diagram, and calculate the fitting deviation of the ordered centerline point set based on the fitted arc corresponding to the ordered centerline point set; sort the ordered centerline point set according to the fitting deviation to determine the candidate edge-attached short arc segments. S5. Based on the normal distance coordinate range of the candidate edge-attaching short arc segment, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; and obtain the detection result based on the normal offset direction.

2. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, Generate a saliency map of the raised area, including: Bright-field images were obtained by performing bright-field illumination acquisition on the FPC assembly. Low-angle dark-field illumination was used to acquire dark-field images of the FPC assembly. A salience map of warping is generated based on the bright field image and the dark field image.

3. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, The edge-attaching area is obtained, including: Edge extraction is performed on the bright field image to obtain the outer contour edge image; The outer contour line is obtained by extracting the closed contour from the outer contour edge image; The width of the first pixel is determined based on the normal direction of the outer contour line; The width of the second pixel is determined based on the normal direction of the outer contour line; Based on the first pixel width and the second pixel width, the outer contour line is extended on both sides to obtain the edge band area.

4. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, Generate an edge-fitting enhancement map, including: Based on the edge band area, the raised salience image is cropped to obtain the edge band salience image; A morphological opening operation is performed on the saliency map of the edge band to obtain the opening operation image; An edge enhancement map is generated based on the edge saliency map and the opening operation image.

5. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, Establishing a local coordinate system includes: Arc length sampling is performed on the outer contour line to obtain multiple outer contour sampling points; Calculate the tangent direction of the outer contour sampling points; Calculate the normal direction of the sampling points on the outer contour; A local coordinate system is established with each outer contour sampling point as the origin, the tangent direction of the outer contour sampling point as the horizontal direction, and the normal direction of the outer contour sampling point as the vertical direction.

6. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, Generate an edge-wrapped ridge map, including: Binarize the edge enhancement image to obtain an edge enhancement binary image; Connectivity regions were extracted from the edge-enhanced binary image to obtain multiple bright ridge connected regions. The bright ridge connected regions are subjected to skeletonization processing to obtain the bright ridge centerline map; Determine the local coordinate system corresponding to the centerline diagram of the bright ridge; Project each centerline pixel in the bright ridge centerline map onto the local coordinate system corresponding to the bright ridge centerline map to obtain the arc length coordinates and normal distance coordinates of the centerline pixel; Based on the arc length coordinates and normal distance coordinates of the centerline pixel, generate an edge-fitting unfolded ridge map.

7. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, Calculate the fitting bias of the ordered centerline point set, including: The connected pixel regions of the unfolded ridge map are extracted to obtain multiple unfolded connected regions. The centerlines of the expanded connected regions are sorted to obtain an ordered set of centerline points. By fitting a circular arc to an ordered set of centerline points, a fitted circular arc is obtained. The fitting deviation of the ordered centerline point set is calculated based on the distance from the centerline pixel point in the ordered centerline point set to the fitted arc.

8. The real-time inspection method for production line assembly using computer vision according to claim 7, characterized in that, Determine candidate short arc segments for edge contact, including: Based on the fitting deviation, the ordered centerline point set is sorted to obtain the target centerline point set; The expanded connected region corresponding to the target centerline point set is used as the candidate short arc segment for edge contact.

9. The real-time inspection method for production line assembly using computer vision according to claim 1, characterized in that, The test results include: The range of normal distance coordinates for the candidate short arc segments is determined based on their pixel coordinates in the edge unfolded ridge map. Based on the range of normal distance coordinates, determine the normal offset direction of the candidate edge-attaching short arc segment relative to the outer contour line; Based on the normal offset direction, the candidate edge-fitting short arc segments are judged by reverse bending to obtain the detection result; the detection result is either that there is an edge reverse bending or that there is no edge reverse bending.

10. A system applied to the real-time inspection method for production line assembly using computer vision as described in any one of claims 1-9, characterized in that, The system includes: The salient generation module generates salient images of FPC assemblies; The edge-attaching enhancement module obtains the outer contour line based on the bright-field image of the FPC assembly, and expands the outer contour line on both sides to obtain the edge-attaching strip area; based on the edge-attaching strip saliency map corresponding to the edge-attaching strip area, it generates an edge-attaching enhancement map; The ridge unfolding module establishes a local coordinate system based on the outer contour line corresponding to the bright field image; it performs binarization and skeletonization processing on the edge enhancement image to obtain the bright ridge centerline image, and generates the edge unfolded ridge image according to the local coordinate system corresponding to the bright ridge centerline image. The arc segment filtering module obtains an ordered set of centerline points based on the expanded connected regions corresponding to the edge-attached ridge map, and calculates the fitting deviation of the ordered centerline point set based on the fitted arcs corresponding to the ordered centerline point set; according to the fitting deviation, the ordered centerline point set is sorted to determine candidate edge-attached short arc segments. The reverse folding determination module determines the normal offset direction of the candidate edge-attached short arc segment relative to the outer contour line based on the normal distance coordinate range of the candidate edge-attached short arc segment; and obtains the detection result based on the normal offset direction.

Images