Automatic flexible visual identification method and system for large-format glass profile

Abstract

The application discloses a large-format glass contour automatic flexible visual identification method and system, which comprises the following steps: collecting a contour image of large-format glass to be detected; regarding a local glass area to be detected, a local contour area to be detected, and other areas except a supporting platform area as a background area, and removing the background area of the image; extracting a rough contour area of the glass to be detected in the image; extracting a sub-pixel contour of the glass to be detected in the image, and extracting shape information of the rough contour area by using a skeletonization algorithm; extracting an outer contour area of the large-format glass contour; and extracting a sub-pixel precision edge of the outer contour area of the large-format glass by using a canny filter, so as to serve as an identification result of the local contour of the large-format glass. The large-format glass contour local image is collected by using a plane array camera, the sub-pixel contour of the glass to be detected in the image can be extracted, the method has good flexibility to contour position change, the detection efficiency can be improved, and the hardware cost is reduced.

Description

Technical Field

[0001] This invention belongs to the field of automatic glass contour detection technology, and relates to a method and system for automatic flexible visual recognition of large-format glass contours. Background Technology

[0002] With advancements in glass manufacturing processes and performance, large-format glass is widely used in high-end emerging fields such as automobiles, aerospace, large displays, large optical systems, and photovoltaic solar substrates. Large-format glass has a large size and irregular contours, and the dimensional and positional errors in contour processing significantly impact the assembly accuracy and performance of subsequent products. Currently, large-format glass inspection relies on manual labor. Workers use customized inspection tools to perform non-quantitative sampling and comparison of local contours. On the one hand, manual inspection is susceptible to subjective factors, resulting in unstable inspection quality; improper operation can lead to worker injuries or damage to the inspected contours. On the other hand, the significant differences in contour shapes between different batches of glass easily lead to wasted inspection tools and low inspection efficiency. Therefore, automated and quantitative inspection of large-format glass is crucial for improving inspection efficiency, reducing production costs, and ensuring product quality.

[0003] A search revealed no publicly available methods or technologies for visual recognition and detection of the contours of large-format glass. Regarding glass visual inspection, announcement CN112629441A discloses an imaging method for curved glass surface contours based on line structured light, but its design is primarily for acquiring the three-dimensional contours of curved glass surfaces and cannot be used for imaging and detecting the contours of the glass's perimeter. Announcement CN109886960A discloses a machine vision-based method for detecting glass edge defects, but this method requires pre-acquiring the coordinate points of the corresponding image of the glass and does not explain the process of acquiring these coordinate points, thus failing to achieve automatic acquisition of the glass contour. Announcement CN111369593A discloses a glass coating method, apparatus, electronic device, and storage medium; however, this method does not disclose the method for acquiring the glass edge contour point cloud and cannot be used for glass contour recognition. Regarding other objects' wheels... Regarding the visual inspection of contours, announcements CN109579735A and CN109579734A disclose a device and apparatus for detecting, identifying, classifying, and classifying the taper, slope, and gradient of the outer circular profile of piston ring contours with small radial diameters. However, this device cannot acquire large-format glass contours, and the method steps cannot identify the contours around large-diameter glass. Announcement CN108876773A discloses an online detection method for the aperture of LED glass lamp cups. This method acquires a complete image of the object being inspected through a single frame image, but it cannot acquire complete large-format glass contours. Announcement CN110607405A discloses a device and method for acquiring, processing, and guiding the cutting of leather inner contour images. However, the imaging device and image processing method of this system cannot be used for imaging and detecting glass contours. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic flexible visual recognition method and system for large-format glass contours. The method uses an area scan camera to acquire local images of large-format glass contours and can extract the sub-pixel contours of the glass to be detected in the images. This method has good flexibility for contour position changes, which can improve detection efficiency and reduce hardware costs.

[0005] The technical solution to achieve the purpose of this invention is as follows:

[0006] A method for automatic flexible visual recognition of large-format glass contours includes the following steps:

[0007] S01: Acquire the outline image of the large-format glass to be inspected;

[0008] S02: Take the areas other than the local glass region to be detected, the local contour region of the glass to be detected, and the support platform region in the image as background regions, and remove the background regions of the image.

[0009] S03: Extract the coarse outline region of the glass to be detected in the image;

[0010] S04: Extract the sub-pixel contour of the glass to be detected in the image, specifically including the following steps:

[0011] S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region;

[0012] S42: Extract the outer contour area of ​​the large-format glass outline;

[0013] S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

[0014] In a preferred embodiment, the method for removing the image background region in step S02 includes:

[0015] S21: Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T are extracted from the image. T is the threshold. The gray value of this region is set to 1, and the gray values ​​of other regions are set to 0. The binarized image is then denoted as G2.

[0016] S22: Closing operation: A morphological closing operation is applied to the region with a gray value of 1 in the binary image G2. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the glass contour region that was removed due to the above threshold segmentation and set the gray value of the region to 1. The binarized image at this time is denoted as G3.

[0017] S23: Hole filling: Morphological filling is used to extract the internal regions in the glass region and support platform region of the acquired image G1 that have been removed by threshold segmentation, and the gray value of the region is set to 1. The binarized image is then denoted as G4.

[0018] S24: Image multiplication operation: Perform a cross product calculation on the binary image G4 obtained in the previous step and the original image G1 to extract the glass region, glass contour region and support platform region in the original image. The grayscale image at this time is recorded as G5.

[0019] In a preferred embodiment, step S03, extracting the coarse outline region of the glass to be detected in the image, includes:

[0020] S31: Threshold segmentation: The image extracted in the previous step is binarized using a fixed threshold method. Regions with gray values ​​less than T1 are extracted and their gray values ​​are set to 1. Other regions are set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6.

[0021] S32: 8-Neighborhood Connection: Perform neighborhood connection on the large-format glass local contour area obtained in the previous step, and set the gray value of the area to 1. The image is then recorded as G7.

[0022] S33: Feature Selection and Contour Extraction: Using area and density as features of the region of interest, extract the large-format glass contour region in the binary image G7 from the previous step that satisfies the area and density features. At the same time, remove noise information in the glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass contour, denoted as G8.

[0023] In a preferred embodiment, steps S41 and S42 further include:

[0024] S411: Image Complement: Using the skeletalized image, the background removal area is segmented into two parts: the glass region and the support platform region. Gray-scale features are then used to extract the support platform region, denoted as G. 10 ;

[0025] S412: Coarse Outline Region Dilation: Performs a morphological dilation operation on the coarse outline region of a large-format glass panel, denoted as G. 11 ;

[0026] S413: Image intersection: Calculate the dilated region G of the coarse outline from the previous step. 11 With support platform area G 10 The intersection of these points is used to extract the outer contour of the large-format glass outline, denoted as G. 12 ;

[0027] S414: Image multiplication operation: Multiply the binary image G obtained in the previous step.12 Perform a cross product calculation with the original image G1 to extract the outer contour region of the glass contour in the original image.

[0028] This invention also discloses an automatic flexible visual recognition system for large-format glass contours, comprising:

[0029] The contour image acquisition module acquires contour images of the large-format glass to be inspected;

[0030] The background removal module takes the area outside the local glass region to be detected, the local outline region of the glass to be detected, and the support platform region in the image as the background region and removes the background region of the image.

[0031] The coarse contour extraction module extracts the coarse contour region of the glass to be detected in the image;

[0032] The subpixel contour extraction module extracts the subpixel contour of the glass to be detected in the image, specifically including the following steps:

[0033] S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region;

[0034] S42: Extract the outer contour area of ​​the large-format glass outline;

[0035] S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

[0036] In a preferred embodiment, the method for removing the background region of an image in the background removal module includes:

[0037] S21: Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T are extracted from the image. T is the threshold. The gray value of this region is set to 1, and the gray values ​​of other regions are set to 0. The binarized image is then denoted as G2.

[0038] S22: Closing operation: A morphological closing operation is applied to the region with a gray value of 1 in the binary image G2. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the glass contour region that was removed due to the above threshold segmentation and set the gray value of the region to 1. The binarized image at this time is denoted as G3.

[0039] S23: Hole filling: Morphological filling is used to extract the internal regions in the glass region and support platform region of the acquired image G1 that have been removed by threshold segmentation, and the gray value of the region is set to 1. The binarized image is then denoted as G4.

[0040] S24: Image multiplication operation: Perform a cross product calculation on the binary image G4 obtained in the previous step and the original image G1 to extract the glass region, glass contour region and support platform region in the original image. The grayscale image at this time is recorded as G5.

[0041] In a preferred embodiment, the coarse contour extraction module extracts the coarse contour region of the glass to be detected in the image, including:

[0042] S31: Threshold segmentation: The image extracted in the previous step is binarized using a fixed threshold method. Regions with gray values ​​less than T1 are extracted and their gray values ​​are set to 1. Other regions are set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6.

[0043] S32: 8-Neighborhood Connection: Perform neighborhood connection on the large-format glass local contour area obtained in the previous step, and set the gray value of the area to 1. The image is then recorded as G7.

[0044] S33: Feature Selection and Contour Extraction: Using area and density as features of the region of interest, extract the large-format glass contour region in the binary image G7 from the previous step that satisfies the area and density features. At the same time, remove noise information in the glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass contour, denoted as G8.

[0045] In a preferred embodiment, the sub-pixel contour extraction module further includes the following step between S41 and S42:

[0046] S411: Image Complement: Using the skeletalized image, the background removal area is segmented into two parts: the glass region and the support platform region. Gray-scale features are then used to extract the support platform region, denoted as G. 10 ;

[0047] S412: Coarse Outline Region Dilation: Performs a morphological dilation operation on the coarse outline region of a large-format glass panel, denoted as G. 11 ;

[0048] S413: Image intersection: Calculate the dilated region G of the coarse outline from the previous step. 11 With support platform area G 10 The intersection of these points is used to extract the outer contour of the large-format glass outline, denoted as G. 12 ;

[0049] S414: Image multiplication operation: Multiply the binary image G obtained in the previous step. 12 Perform a cross product calculation with the original image G1 to extract the outer contour region of the glass contour in the original image.

[0050] The present invention also discloses a computer storage medium storing a computer program, which, when executed, implements the above-described automatic flexible visual recognition method for large-format glass contours.

[0051] This invention also discloses a large-format glass contour automatic flexible visual recognition device, including a transparent support platform, a camera, a lens, a camera moving platform, a light source, a light source moving platform, and an industrial control computer;

[0052] The transparent support platform is a horizontal support platform made of transparent material, used to horizontally place the large-format glass to be inspected, and the transparent support platform is larger than the area of ​​the large-format glass to be inspected;

[0053] The camera is an area scan camera, fixed on the camera moving platform. The camera axis is perpendicular to the transparent support platform. The outline of the large-format glass to be detected is always within the camera's field of view. It is used to acquire local images of the outline of the large-format glass to be detected and transmit them to the industrial control computer through the camera interface.

[0054] The lens is a telecentric lens, and the lens is fixedly connected to the camera. The lowest mechanical surface of the lens is parallel to the plane of the glass to be tested.

[0055] The camera moving platform is driven by a servo motor, which controls the camera to move to each detection position in the XY plane according to a set path. At the same time, it ensures that the image captured at each shooting point contains the outline of the glass to be detected, and that the glass outline images obtained at adjacent detection points have overlapping areas.

[0056] The light source is a parallel backlight source located below the transparent support platform. The center of the parallel light source is concentric with the center of the field of view of the area array camera. The backlighting method is used to illuminate the outline of the glass to be inspected, so that the outline area of ​​the large-format glass to be inspected forms a dark area, and the other areas of the large-format glass to be inspected form a bright area.

[0057] The light source moving platform is driven by a servo motor, which controls the light source to move in real time in the XY plane to follow the camera, ensuring that the relative positions of the camera and the light source remain unchanged.

[0058] The industrial control computer is the main processor and is connected to the camera and servo motor. The industrial control computer is connected to the camera interface to acquire digital images of the large-format glass contour to be detected in real time, and performs image processing on the acquired images. The above-mentioned automatic flexible visual recognition method for large-format glass contour is used to realize contour recognition and measurement. The industrial control computer is connected to the servo motor so that the position, orientation and status of the camera moving platform and the light source moving platform can be moved according to requirements.

[0059] Compared with the prior art, the significant advantages of this invention are:

[0060] (1) The present invention uses machine vision technology to realize image acquisition and contour recognition of large-format glass contours, which overcomes the drawbacks of existing manual inspection methods and effectively improves inspection efficiency and automation.

[0061] (2) The present invention uses an area array camera to acquire local images of large-format glass contours, providing a new approach to image acquisition required for large-format glass contour detection. Compared with image acquisition methods such as line scanning, it can improve detection efficiency and reduce hardware costs.

[0062] (3) This invention proposes an automatic identification method and steps for subpixel contours of large-format glass. This method can automatically and accurately identify the local contours of large-format glass in an image. The actual contour identification time for a single image is less than 300ms. This method has good flexibility for contour position changes, laying the foundation for subsequent high-precision image stitching. Attached Figure Description

[0063] Figure 1 A flowchart of a preferred embodiment of an automatic flexible visual recognition method for large-format glass contours;

[0064] Figure 2 A schematic diagram of a large-format glass contour automatic flexible vision recognition system according to a preferred embodiment;

[0065] Figure 3 This is a schematic diagram of a preferred embodiment;

[0066] Figure 4 Electrical connection diagram for a preferred embodiment;

[0067] Figures 5a-5b A large-format glass partial contour imaging effect diagram obtained by the imaging scheme of a preferred embodiment;

[0068] Figure 6 This is a flowchart illustrating the image processing of a preferred embodiment.

[0069] Figure 7 This is a diagram illustrating the effect of large-format glass outline recognition in a preferred embodiment.

[0070] In the diagram: 1. Transparent support platform; 2. Large-format glass to be inspected; 3. Camera; 4. Lens; 5. Camera moving platform; 6. Light source; 7. Light source moving platform; 8. Industrial computer; 9. Glass area to be inspected; 10. Glass outline to be inspected; 11. Transparent support platform area; 12. Background area. Detailed Implementation

[0071] The principle of this invention is as follows: an optical imaging scheme with an area array camera for automatic detection is used to acquire a large-format local image of the glass outline, which can extract the sub-pixel outline of the glass to be detected in the image. This method has good flexibility for changes in the outline position.

[0072] Example 1:

[0073] like Figure 1 As shown, an automatic flexible visual recognition method for large-format glass contours includes the following steps:

[0074] S01: Acquire the outline image of the large-format glass to be inspected;

[0075] S02: Take the areas other than the local glass region to be detected, the local contour region of the glass to be detected, and the support platform region in the image as background regions, and remove the background regions of the image.

[0076] S03: Extract the coarse outline region of the glass to be detected in the image;

[0077] S04: Extract the sub-pixel contour of the glass to be detected in the image, specifically including the following steps:

[0078] S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region;

[0079] S42: Extract the outer contour area of ​​the large-format glass outline;

[0080] S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

[0081] In a preferred embodiment, the method for removing the image background region in step S02 includes:

[0082] S21: Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T are extracted from the image. T is the threshold. The gray value of this region is set to 1, and the gray values ​​of other regions are set to 0. The binarized image is then denoted as G2.

[0083] S22: Closing operation: A morphological closing operation is applied to the region with a gray value of 1 in the binary image G2. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the glass contour region that was removed due to the above threshold segmentation and set the gray value of the region to 1. The binarized image at this time is denoted as G3.

[0084] S23: Hole filling: Morphological filling is used to extract the internal regions in the glass region and support platform region of the acquired image G1 that have been removed by threshold segmentation, and the gray value of the region is set to 1. The binarized image is then denoted as G4.

[0085] S24: Image multiplication operation: Perform a cross product calculation on the binary image G4 obtained in the previous step and the original image G1 to extract the glass region, glass contour region and support platform region in the original image. The grayscale image at this time is recorded as G5.

[0086] In a preferred embodiment, step S03 involves extracting the coarse outline region of the glass to be detected in the image, including:

[0087] S31: Threshold segmentation: The image extracted in the previous step is binarized using a fixed threshold method. Regions with gray values ​​less than T1 are extracted and their gray values ​​are set to 1. Other regions are set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6.

[0088] S32: 8-Neighborhood Connection: Perform neighborhood connection on the large-format glass local contour area obtained in the previous step, and set the gray value of the area to 1. The image is then recorded as G7.

[0089] S33: Feature Selection and Contour Extraction: Using area and density as features of the region of interest, extract the large-format glass contour region in the binary image G7 from the previous step that satisfies the area and density features. At the same time, remove noise information in the glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass contour, denoted as G8.

[0090] In a preferred embodiment, steps S41 and S42 further include:

[0091] S411: Image Complement: Using the skeletalized image, the background removal area is segmented into two parts: the glass region and the support platform region. Gray-scale features are then used to extract the support platform region, denoted as G. 10 ;

[0092] S412: Coarse Outline Region Dilation: Performs a morphological dilation operation on the coarse outline region of a large-format glass panel, denoted as G. 11 ;

[0093] S413: Image intersection: Calculate the dilated region G of the coarse outline from the previous step. 11 With support platform area G 10 The intersection of these points is used to extract the outer contour of the large-format glass outline, denoted as G. 12 ;

[0094] S414: Image multiplication operation: Multiply the binary image G obtained in the previous step. 12 Perform a cross product calculation with the original image G1 to extract the outer contour region of the glass contour in the original image.

[0095] In another embodiment, such as Figure 2 As shown, a large-format glass contour automatic flexible visual recognition system includes:

[0096] The contour image acquisition module acquires contour images of the large-format glass to be inspected;

[0097] The background removal module takes the area outside the local glass region to be detected, the local outline region of the glass to be detected, and the support platform region in the image as the background region and removes the background region of the image.

[0098] The coarse contour extraction module extracts the coarse contour region of the glass to be detected in the image;

[0099] The subpixel contour extraction module extracts the subpixel contour of the glass to be detected in the image, specifically including the following steps:

[0100] S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region;

[0101] S42: Extract the outer contour area of ​​the large-format glass outline;

[0102] S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

[0103] In another embodiment, a computer storage medium stores a computer program that, when executed, implements the above-described automatic flexible visual recognition method for large-format glass contours.

[0104] In another embodiment, a large-format glass contour automatic flexible visual recognition device includes a transparent support platform, a camera, a lens, a camera moving platform, a light source, a light source moving platform, and an industrial control computer; the industrial control computer integrates the aforementioned large-format glass contour automatic flexible visual recognition system, which includes a background removal algorithm (threshold segmentation, closing operation, hole filling, image multiplication operation), a coarse contour extraction algorithm (threshold segmentation, 8-neighborhood connection, feature selection and contour extraction), and a fine contour extraction algorithm (skeletonization, image complement, coarse contour region dilation, image intersection, area filtering, image multiplication operation, subpixel contour extraction).

[0105] Specifically, the transparent support platform is a horizontal support platform made of transparent material, used to horizontally place the large-format glass to be inspected, and the transparent support platform is larger than the area of ​​the large-format glass to be inspected;

[0106] The camera is an area scan camera, fixed on the camera moving platform. The camera axis is perpendicular to the transparent support platform. The outline of the large-format glass to be detected is always within the camera's field of view. It is used to acquire local images of the outline of the large-format glass to be detected and transmit them to the industrial control computer through the camera interface.

[0107] The lens is a telecentric lens, and the lens is fixedly connected to the camera. The lowest mechanical surface of the lens is parallel to the plane of the glass to be tested.

[0108] The camera moving platform is driven by a servo motor, which controls the camera to move to each detection position in the XY plane according to a set path. At the same time, it ensures that the image captured at each shooting point contains the outline of the glass to be detected, and that the glass outline images obtained at adjacent detection points have overlapping areas.

[0109] The light source is a parallel backlight source located below the transparent support platform. The center of the parallel light source is concentric with the center of the field of view of the area array camera. The backlighting method is used to illuminate the outline of the glass to be inspected, so that the outline area of ​​the large-format glass to be inspected forms a dark area, and the other areas of the large-format glass to be inspected form a bright area.

[0110] The light source moving platform is driven by a servo motor, which controls the light source to move in real time in the XY plane to follow the camera, ensuring that the relative positions of the camera and the light source remain unchanged.

[0111] The industrial control computer is the main processor and is connected to the camera and servo motor. The industrial control computer is connected to the camera interface to acquire digital images of the large-format glass contour to be detected in real time, and performs image processing on the acquired images. The above-mentioned automatic flexible visual recognition method for large-format glass contour is used to realize contour recognition and measurement. The industrial control computer is connected to the servo motor so that the position, orientation and status of the camera moving platform and the light source moving platform can be moved according to requirements.

[0112] Specifically, such as Figure 3 , 4 As shown, the workflow of the large-format glass contour automatic flexible visual recognition device is described below using a preferred embodiment as an example:

[0113] (1) Place the large-format glass 2 to be tested horizontally above the transparent support platform 1. The area of ​​the large-format glass 2 to be tested is smaller than the area of ​​the transparent support platform 1.

[0114] (2) Power on, the servo motor drives the camera moving platform 5 and the light source moving platform 7 back to the zero position. The axes of the camera 3, lens 4 and light source 6 are all perpendicular to the transparent support platform 1. The center of the light source 6 is concentric with the center of the field of view of the camera 3. The local outline of the large-format glass 2 to be inspected is within the field of view of the camera 3.

[0115] (3) Start the camera 3 and the light source 6, drive the servo motor to move the camera moving platform and the light source moving platform along the outer contour of the large-format glass 2 in the XY plane, and the camera moving platform 5 and the light source moving platform 7 move to the first detection position according to the set path, and then acquire the first local image of the large-format glass 2 contour in real time.

[0116] (4) Next, drive the camera moving platform 5 and the light source moving platform 7 to move to the second detection position in the XY plane, and acquire a local image of the second large-format glass 2 contour. It is necessary to ensure that the glass contour images acquired at adjacent detection point positions have overlapping areas.

[0117] (5) Repeat step (4) above, drive the camera moving platform 5 and the light source moving platform 7 to each position to be detected in sequence, and acquire local contour images of the large-format glass 2 to be detected until the acquired local images contain all the contours of the entire large-format glass 2.

[0118] (6) The industrial computer 8 acquires and processes the real-time images collected by the current imaging scheme, such as... Figures 5a-5b As shown.

[0119] (7) Figure 6 As shown, through image analysis, the regions other than the local glass region 9 to be detected, the local contour region 10 of the glass to be detected, and the transparent support platform region 11 in the image are designated as background regions 12. Background regions 12 are then removed from the image, specifically:

[0120] (71) Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T (i.e., glass region and transparent support platform region) are extracted from the image. The gray value of this region is set to 1, and the gray value of other regions is set to 0. This is used to remove irrelevant background information such as pads in the image. The binarized image is then recorded as G2.

[0121] (72) Closing operation: Morphological closing operation is applied to the region with a gray value of 1 in the binary image G2 above. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the glass contour region that was removed due to the threshold segmentation above, and the gray value of the region is set to 1. The binarized image at this time is called G3.

[0122] (73) Hole filling: Morphological filling is used to extract the internal regions in the glass region and transparent support platform region of the acquired image G1 that have been removed by threshold segmentation due to noise, dust, etc., and the gray value of the region is set to 1. The binarized image is then recorded as G4.

[0123] (74) Image multiplication operation: perform cross multiplication calculation on the binary image G4 obtained in the previous step and the original image G1, extract the glass region 9, glass contour region 10 and transparent support platform region 11 in the original image, and remove other background regions 12. At this time, the grayscale image is recorded as G5.

[0124] (8) Analyze the image after background removal and further extract the coarse outline of the glass to be detected in the image, specifically:

[0125] (81) Threshold segmentation: The image G5 extracted in the previous step is binarized using a fixed threshold method. The region with a gray value less than T1 (i.e., the local contour region of large-format glass) is extracted. The gray value of this region is set to 1, and the gray value of other regions is set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6.

[0126] (82)8-Neighborhood connection: perform neighborhood connection on the large-format glass local contour area obtained in the previous step to extract the contour break area caused by noise, dust, dirt, etc., and set the gray value of the area to 1. The image is then recorded as G7.

[0127] (83) Feature selection and contour extraction: Area and density are used as features of region of interest. Large-format glass contour regions that satisfy the area and density features in the binary image G7 of the previous step are extracted. At the same time, noise information in the glass region and the transparent support platform region is deleted to achieve the preliminary extraction of the position and region information of the large-format glass contour, which is denoted as G8.

[0128] (9) Analyze the coarse contour region in the above large-format glass local image, and further extract the sub-pixel contour of the glass to be detected in the image, specifically:

[0129] (91) Skeletonization: The skeletonization algorithm is used to extract the shape information of the coarse contour region G8 in the previous step, so as to achieve the initial acquisition of the edge position of the large-format contour. The image after skeletonization is denoted as G9.

[0130] (92) Image complement: Using the skeletonized region G9 from the previous step, the background region image G5 is segmented into two parts: the glass region and the transparent support platform region. Then, using grayscale features, the transparent support platform region in the image is extracted and denoted as G. 10 ;

[0131] (93) Expansion of the coarse outline region: A morphological expansion operation is performed on the coarse outline region G8 of the large-format glass outline, denoted as G. 11 ;

[0132] (94) Take the intersection of the images and calculate the dilated region G of the coarse contour from the previous step. 11 With transparent support platform area G 10 The intersection of these two points is used to extract the outer contour of large-format glass outlines to meet practical industrial inspection needs, denoted as G. 12 ;

[0133] (95) Image multiplication operation: multiply the binary image G obtained in the previous step. 12 Perform a cross product with the original image G1 to extract the outer contour region of the glass outline in the original image. The resulting grayscale image is denoted as G. 13 ;

[0134] (96) Subpixel contour extraction: The Canny filter is used to extract the subpixel precision edges of the outer contour region of the large-format glass, which is used as the recognition result of the local contour of the large-format glass, denoted as G. 14 ,like Figure 7 As shown.

[0135] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for automatic flexible visual recognition of large-format glass contours, characterized in that, Includes the following steps: S01: Acquire the outline image of the large-format glass to be inspected; S02: Take the areas other than the local area of ​​the glass to be detected, the local contour area of ​​the glass to be detected, and the support platform area in the image as background areas, and remove the background areas of the image. S03: Extract the coarse outline region of the glass to be detected in the image: Using area and density as features of the region of interest, extract the local outline region of the large-format glass that satisfies the area and density features, and at the same time delete noise information in the local glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass outline. S04: Extract the sub-pixel contour of the glass to be detected in the image, specifically including the following steps: S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region; S42: Extract the outer contour area of ​​the large-format glass outline; S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

2. The automatic flexible visual recognition method for large-format glass contours according to claim 1, characterized in that, The method for removing the image background region in step S02 includes: S21: Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T are extracted from the image. T is the threshold. The gray value of this region is set to 1, and the gray values ​​of other regions are set to 0. The binarized image is then denoted as G2. S22: Closing operation: A morphological closing operation is applied to the region with a gray value of 1 in the binary image G2. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the local glass contour region that was removed due to the above threshold segmentation, and the gray value of the region is set to 1. The binarized image at this time is denoted as G3. S23: Hole filling: Morphological filling is used to extract the internal regions that have been removed by threshold segmentation in the glass local area and support platform area of ​​the acquired image G1, and the gray value of the region is set to 1. The binarized image is then denoted as G4. S24: Image multiplication operation: Perform a cross product calculation on the binary image G4 obtained in the previous step and the original image G1 to extract the local glass region, local glass contour region and support platform region in the original image. The grayscale image at this time is recorded as G5.

3. The automatic flexible visual recognition method for large-format glass contours according to claim 1, characterized in that, Step S03 involves extracting the coarse outline region of the glass to be detected in the image, including: S31: Threshold segmentation: The image extracted in the previous step is binarized using a fixed threshold method. Regions with gray values ​​less than T1 are extracted and their gray values ​​are set to 1. Other regions are set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6. S32: 8-Neighborhood Connection: Perform neighborhood connection on the large-format glass local contour area obtained in the previous step, and set the gray value of the area to 1. The image is then recorded as G7. S33: Feature Selection and Contour Extraction: Using area and density as features of the region of interest, extract the large-format glass contour region that satisfies the area and density features in the binary image G7 from the previous step. At the same time, remove noise information in the local glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass contour, denoted as G8.

4. The automatic flexible visual recognition method for large-format glass contours according to claim 1, characterized in that, The steps S41 and S42 also include: S411: Image Complement: Using the skeletalized image, the background-removed region is segmented into two parts: the glass local region and the support platform region. Gray-scale features are then used to extract the support platform region, denoted as G. 10 ; S412: Coarse Outline Region Dilation: Performs a morphological dilation operation on the coarse outline region of a large-format glass panel, denoted as G. 11 ; S413: Image intersection: Calculate the dilated region G of the coarse outline from the previous step. 11 With support platform area G 10 The intersection of these points is used to extract the outer contour of the large-format glass outline, denoted as G. 12 ; S414: Image multiplication operation: Multiply the binary image G obtained in the previous step. 12 Perform a cross product calculation with the original image G1 to extract the outer contour region of the glass contour in the original image.

5. A large-format glass contour automatic flexible visual recognition system, characterized in that, include: The contour image acquisition module acquires contour images of the large-format glass to be inspected; The background removal module takes the local area of ​​the glass to be detected, the local outline area of ​​the glass to be detected, and other areas other than the support platform area in the image as the background area and removes the background area of ​​the image. The coarse contour extraction module extracts the coarse contour region of the glass to be detected in the image: using area and density as region of interest features, it extracts the large-format glass contour region that satisfies the area and density features, while deleting noise information in local glass regions and support platform regions, thus achieving the preliminary extraction of the position and region information of the large-format glass contour. The subpixel contour extraction module extracts the subpixel contour of the glass to be detected in the image, specifically including the following processing: S41: Use a skeletonization algorithm to extract the shape information of the coarse outline region; S42: Extract the outer contour area of ​​the large-format glass outline; S43: The Canny filter is used to extract the sub-pixel precision edges of the outer contour region of the large-format glass as the recognition result of the local contour of the large-format glass.

6. The large-format glass contour automatic flexible visual recognition system according to claim 5, characterized in that, The method for removing the background region of an image in the background removal module includes: S21: Threshold segmentation: The original image G1 is binarized using a fixed threshold segmentation method. Regions with gray values ​​greater than T are extracted from the image. T is the threshold. The gray value of this region is set to 1, and the gray values ​​of other regions are set to 0. The binarized image is then denoted as G2. S22: Closing operation: A morphological closing operation is applied to the region with a gray value of 1 in the binary image G2. The width of the structuring element of the closing operation is not less than the width of the glass contour to be detected. It is used to extract the local glass contour region that was removed due to the above threshold segmentation, and the gray value of the region is set to 1. The binarized image at this time is denoted as G3. S23: Hole filling: Morphological filling is used to extract the internal regions that have been removed by threshold segmentation in the glass local area and support platform area of ​​the acquired image G1, and the gray value of the region is set to 1. The binarized image is then denoted as G4. S24: Image multiplication operation: Perform a cross product calculation on the binary image G4 obtained in the previous step and the original image G1 to extract the local glass region, local glass contour region and support platform region in the original image. The grayscale image at this time is recorded as G5.

7. The large-format glass contour automatic flexible visual recognition system according to claim 5, characterized in that, The coarse contour extraction module extracts the coarse contour region of the glass to be detected in the image, including: S31: Threshold segmentation: The image extracted in the previous step is binarized using a fixed threshold method. Regions with gray values ​​less than T1 are extracted and their gray values ​​are set to 1. Other regions are set to 0. This is used to extract the local contour region of large-format glass in the image, denoted as G6. S32: 8-Neighborhood Connection: Perform neighborhood connection on the large-format glass local contour area obtained in the previous step, and set the gray value of the area to 1. The image is then recorded as G7. S33: Feature Selection and Contour Extraction: Using area and density as features of the region of interest, extract the large-format glass contour region that satisfies the area and density features in the binary image G7 from the previous step. At the same time, remove noise information in the local glass region and the support platform region to achieve the preliminary extraction of the position and region information of the large-format glass contour, denoted as G8.

8. The large-format glass contour automatic flexible visual recognition system according to claim 5, characterized in that, The sub-pixel contour extraction module further includes processing between S41 and S42 as follows: S411: Image Complement: Using the skeletalized image, the background-removed region is segmented into two parts: the glass local region and the support platform region. Gray-scale features are then used to extract the support platform region, denoted as G. 10 ; S412: Coarse Outline Region Dilation: Performs a morphological dilation operation on the coarse outline region of a large-format glass panel, denoted as G. 11 ; S413: Image intersection: Calculate the dilated region G of the coarse outline from the previous step. 11 With support platform area G 10 The intersection of these points is used to extract the outer contour of the large-format glass outline, denoted as G. 12 ; S414: Image multiplication operation: Multiply the binary image G obtained in the previous step. 12 Perform a cross product calculation with the original image G1 to extract the outer contour region of the glass contour in the original image.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer executes the computer program, it implements the automatic flexible visual recognition method for large-format glass contours as described in any one of claims 1-4.

10. A large-format glass contour automatic flexible visual recognition device, characterized in that, Includes a transparent support platform, camera, lens, camera moving platform, light source, light source moving platform, and industrial control computer; The transparent support platform is a horizontal support platform made of transparent material, used to horizontally place the large-format glass to be inspected, and the transparent support platform is larger than the area of ​​the large-format glass to be inspected; The camera is an area scan camera, fixed on the camera moving platform. The camera axis is perpendicular to the transparent support platform. The outline of the large-format glass to be detected is always within the camera's field of view. It is used to acquire local images of the outline of the large-format glass to be detected and transmit them to the industrial control computer through the camera interface. The lens is a telecentric lens, and the lens is fixedly connected to the camera. The lowest mechanical surface of the lens is parallel to the plane of the glass to be tested. The camera moving platform is driven by a servo motor, which controls the camera to move to each detection position in the XY plane according to a set path. At the same time, it ensures that the image captured at each shooting point contains the outline of the glass to be detected, and that the glass outline images obtained at adjacent detection points have overlapping areas. The light source is a parallel backlight source located below the transparent support platform. The center of the parallel light source is concentric with the center of the field of view of the area array camera. The backlighting method is used to illuminate the outline of the glass to be inspected, so that the outline area of ​​the large-format glass to be inspected forms a dark area, and the other areas of the large-format glass to be inspected form a bright area. The light source moving platform is driven by a servo motor, which controls the light source to move in real time in the XY plane to follow the camera, ensuring that the relative positions of the camera and the light source remain unchanged. The industrial control computer is the main processor and is connected to the camera and servo motor. The industrial control computer is connected to the camera interface and acquires digital images of the large-format glass contour to be detected in real time at the detection position. The acquired images are processed and the contour recognition and measurement are realized by the automatic flexible visual recognition method of large-format glass contour as described in any one of claims 1-4. The industrial control computer is connected to the servo motor so that the positions of the camera moving platform and the light source moving platform can be moved as required.

Images