Blister product defect intelligent detection method and system based on industrial vision

Abstract

The application relates to the technical field of image data processing, in particular to a blister product defect intelligent detection method and system based on industrial vision. The method comprises the following steps: acquiring product images of to-be-detected blister products and calibration plate images of calibration plates respectively photographed on multiple detection stations; determining a relative offset amount of a finished product image relative to a finished calibration plate image based on the positions of physical marking points in the product images and the calibration plate images photographed on the multiple detection stations; evaluating a perspective distortion degree of the finished product image based on the morphological feature difference between a blister product region in the finished product image and a calibration plate region in the finished calibration plate image; and performing perspective transformation calibration on the finished product image based on the relative offset amount and the perspective distortion degree, wherein the calibrated finished product image is used for intelligent defect detection. The method can eliminate the influence of distortion on defect detection and improve the accuracy of defect detection.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, specifically to an intelligent detection method and system for defects in thermoformed products based on industrial vision. Background Technology

[0002] In vacuum thermoforming, plastic sheets are heated and softened before being vacuum-formed onto a mold surface to create various thermoformed products. Due to the high degree of uncontrollability in temperature changes during the heating, softening, and cooling processes, the resulting thermoformed products often have defects such as notches and scratches. Therefore, industrial vision systems are necessary for defect detection in the finished products.

[0003] Currently, in actual inspection processes, industrial cameras are typically installed above the conveyor belt to photograph moving thermoformed products, and then the photographed product images are used for defect detection.

[0004] However, due to factors such as lens installation position, product placement angle, and objective displacement during conveyor belt operation, it is often difficult to maintain a strict perpendicularity between the camera's optical axis and the surface of the thermoformed product, resulting in perspective distortion in the captured images. Perspective distortion causes the thermoformed product area in the image to appear larger when closer and smaller when farther away, making defects on the product surface appear as partial obscuration or distortion in the image. This can lead to misjudgments or missed detections during defect inspection, affecting the final product quality assessment. Summary of the Invention

[0005] To address the technical problem of perspective distortion in captured images leading to misjudgments or missed detections during defect detection, this application aims to provide an intelligent defect detection method and system for thermoformed products based on industrial vision. The specific technical solution adopted is as follows: This application provides an intelligent defect detection method for thermoformed products based on industrial vision, comprising: acquiring product images of the thermoformed product to be inspected and calibration plate images of a calibration plate, respectively captured at multiple inspection stations, wherein both the thermoformed product to be inspected and the calibration plate contain preset physical marker points, and the multiple inspection stations include a finished product inspection station; determining the relative offset of the finished product image relative to the finished product calibration plate image based on the positions of the physical marker points in the product images and calibration plate images captured at the multiple inspection stations, wherein the finished product image is the product image captured at the finished product inspection station, and the finished product calibration plate image is the calibration plate image captured at the finished product inspection station; evaluating the degree of perspective distortion of the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, wherein the degree of perspective distortion is used to characterize the degree of morphological distortion of the thermoformed product area; and performing perspective transformation calibration on the finished product image based on the relative offset and the degree of perspective distortion, wherein the calibrated finished product image is used for intelligent defect detection.

[0006] Optionally, determining the relative offset of the finished product image relative to the finished product calibration plate image based on the positions of physical marker points in the product images and calibration plate images captured at multiple inspection stations includes: determining the product offset vector of the product image at each other inspection station relative to the finished product image based on the positions of physical marker points in the product images captured at multiple inspection stations, where other inspection stations are those other than the finished product inspection station; determining the calibration plate offset vector of the calibration plate image at each other inspection station relative to the finished product calibration plate image based on the positions of physical marker points in the calibration plate images captured at multiple inspection stations; determining the offset distance of the finished product marker points based on the positional differences between the physical marker points in the finished product image and the physical marker points in the finished product calibration plate image; determining the relative offset degree based on the difference between the product offset vector and the calibration plate offset vector corresponding to each other inspection station, where the relative offset degree characterizes the degree of difference in the offset patterns between the thermoformed product to be inspected and the calibration plate among multiple inspection stations; and determining the relative offset amount by weighting and correcting the offset distance of the finished product marker points based on the relative offset degree.

[0007] Optionally, the determination of the relative offset degree based on the difference between the product offset vector and the calibration plate offset vector corresponding to each other inspection station includes: determining the modulus of the difference between the product offset vector and the calibration plate offset vector corresponding to each other inspection station; taking the arithmetic mean of the modulus corresponding to multiple other inspection stations to obtain the mean modulus; and normalizing the mean modulus to obtain the relative offset degree.

[0008] Optionally, the above-mentioned assessment of the perspective distortion degree of the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image includes: determining the displacement vector between the centroid of the calibration plate area in the finished product calibration plate image and the centroid of the thermoformed product area in the finished product image, and determining the direction of the displacement vector as the relative displacement direction; constructing multiple detection lines perpendicular to the relative displacement direction; determining the length of a first segment intercepted by each detection line in the thermoformed product area and the length of a second segment intercepted in the calibration plate area; determining a first difference between the lengths of the first segments corresponding to every two adjacent detection lines and a second difference between the lengths of the second segments corresponding to them; and assessing the perspective distortion degree based on the difference between the first difference and the second difference corresponding to every two adjacent detection lines.

[0009] Optionally, the above-mentioned assessment of the degree of perspective distortion based on the difference between the first difference and the second difference corresponding to each two adjacent detection lines includes: determining the degree of difference in width change between the vacuum-formed product area and the calibration plate area based on the absolute value of the difference between the first difference and the second difference corresponding to each two adjacent detection lines; and determining the degree of perspective distortion as the product of the degree of difference in width change and the degree of relative offset.

[0010] Optionally, the above-mentioned perspective transformation calibration of the finished product image based on the relative offset and the degree of perspective distortion includes: performing edge detection on each product image to identify the defective region in each product image; when there is at least one common defective region in multiple product images, determining the degree of compensation optimization based on the degree of perspective distortion and the feature parameters of the common defective region in each product image, wherein the degree of compensation optimization is used to characterize the weight of correcting the relative offset, and the feature parameters include the centroid position of the common defective region and the area ratio of the common defective region to the area of ​​the thermoformed product; performing a weighted correction on the relative offset based on the degree of compensation optimization to obtain the final offset; and performing perspective transformation calibration on the finished product image based on the final offset.

[0011] Optionally, the above-mentioned determination of the compensation optimization degree based on the degree of perspective distortion and the feature parameters of the common defect area in each product image includes: determining the offset vector of each common defect area relative to the physical marker point in each product image based on the centroid position of each common defect area; determining the similarity parameter of each common defect area among multiple product images based on the offset vector of each common defect area relative to the physical marker point in each product image, the similarity parameter being used to characterize the consistency of the positional offset of the same common defect area in different product images; and determining the compensation optimization degree based on the ratio of the area ratio to the similarity parameter and the degree of perspective distortion.

[0012] Optionally, the above-mentioned perspective transformation calibration of the finished product image based on the final offset includes: determining the amplitude parameter of the perspective transformation based on the final offset; determining the offset trend direction based on the relative position of the physical marker points in the finished product image and the physical marker points in the finished product calibration plate image, the offset trend direction being used to characterize the offset direction of the thermoformed product to be tested relative to the calibration plate; constructing a perspective transformation matrix based on the amplitude parameter and the offset trend direction, the perspective transformation matrix being used to characterize the coordinate mapping relationship from the current distortion state to the standard distortion-free state; and performing back projection resampling on the finished product image based on the perspective transformation matrix to obtain the calibrated finished product image.

[0013] Optionally, before assessing the degree of perspective distortion of the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished calibration plate image, the method further includes: extracting the thermoformed product area from the finished product image and extracting the calibration plate area from the finished calibration plate image using a threshold segmentation algorithm.

[0014] This application also provides an intelligent defect detection system for thermoformed products based on industrial vision, including an image acquisition module, an image analysis module, and a calibration module: the image acquisition module is used to acquire product images of the thermoformed product to be inspected and calibration plate images of the calibration plate, which are respectively captured at multiple inspection stations. Both the thermoformed product to be inspected and the calibration plate contain preset physical marker points. The multiple inspection stations include a finished product inspection station; the image analysis module is used to determine the relative position of the finished product image to the finished product calibration plate image based on the positions of the physical marker points in the product images and calibration plate images captured at multiple inspection stations. The offset is defined as follows: the finished product image is a product image taken at the finished product inspection station, and the finished product calibration plate image is a calibration plate image taken at the finished product inspection station; the image analysis module is also used to evaluate the degree of perspective distortion of the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, and the degree of perspective distortion is used to characterize the degree of morphological distortion of the thermoformed product area; the calibration module is used to perform perspective transformation calibration on the finished product image based on the relative offset and the degree of perspective distortion, and the calibrated finished product image is used for intelligent defect detection.

[0015] This application has the following beneficial effects: By acquiring images from multiple inspection stations to establish a spatial reference, and combining the offset relationship and morphological differences between the thermoformed product under inspection and the calibration plate, the degree and direction of perspective distortion are assessed, and perspective transformation calibration is performed accordingly. Through data fusion from multiple inspection stations, systematic offset and distortion offset can be distinguished, improving the accuracy of distortion detection. Combined with the morphological characteristics of the thermoformed product under inspection, the applicability to thermoformed products of different shapes is improved. Finally, perspective transformation calibration eliminates the impact of distortion on defect detection, thereby improving the accuracy of defect detection when performing subsequent defect detection based on the calibrated image, and ultimately enhancing the production quality of thermoformed products based on industrial vision. Attached Figure Description

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating an intelligent defect detection method for thermoformed products based on industrial vision, provided as an embodiment of this application; Figure 2 This is a structural diagram of an intelligent defect detection system for thermoformed products based on industrial vision, provided as an embodiment of this application. Detailed Implementation

[0018] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an intelligent defect detection method and system for thermoforming products based on industrial vision proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0020] The following description, in conjunction with the accompanying drawings, details the specific scheme of the intelligent detection method and system for defects in thermoformed products based on industrial vision provided in this application.

[0021] Please see Figure 1 The diagram illustrates a flowchart of an intelligent defect detection method for thermoformed products based on industrial vision, provided in one embodiment of this application.

[0022] like Figure 1 As shown, the intelligent defect detection method for thermoformed products based on industrial vision includes S101-S104.

[0023] S101. Acquire product images of the thermoformed product to be tested and calibration plate images of the calibration plate, taken at multiple testing stations respectively.

[0024] The blister product to be tested and the calibration plate both contain preset physical marking points, and the multiple testing stations include the finished product testing station.

[0025] In this embodiment of the application, during the production process of the thermoformed product to be tested, the thermoformed product to be tested is placed on the feed roll and passes through multiple testing stations in sequence via the conveyor belt of the production line.

[0026] Optionally, two cameras can be set on both sides of the feed roll as feed inspection stations, and a single area scan camera can be installed at the finished product inspection station as the finished product inspection station.

[0027] It should be understood that the physical markers contained in the thermoformed product to be inspected are physical labels (such as QR code labels or markers of a specific shape) that are pre-set at specific locations on the feed roll to track the position of the thermoformed product to be inspected during the conveying process.

[0028] It is understandable that a calibration plate (such as a checkerboard calibration plate or a planar calibration plate with specific marking points) is a standard template with known shape and size and no distortion. Its surface is provided with preset physical marking points (such as corner points or specific marks) to characterize the imaging features under standard distortion-free conditions.

[0029] It should be noted that the number of preset physical markers in the product to be tested and the calibration board is at least one. This embodiment of the application uses one preset physical marker as an example. In practical applications, multiple physical markers can be set to improve the robustness and reliability of the system.

[0030] In one alternative implementation, after acquiring product images of the blister packaging product to be inspected and calibration plate images of the calibration plate taken at multiple inspection stations, each image can be filtered and denoised (e.g., median filtering to remove salt-and-pepper noise, high-pass filtering to sharpen edges) to enhance image quality.

[0031] Optionally, the thermoformed product area can be extracted from the finished product image and the calibration plate area can be extracted from the finished calibration plate image using a threshold segmentation algorithm.

[0032] Among them, the finished product image is a product image taken at the finished product inspection station, and the finished product calibration plate image is a calibration plate image taken at the finished product inspection station.

[0033] Specifically, threshold segmentation algorithms (such as Otsu adaptive threshold segmentation or grayscale histogram-based dual threshold segmentation) can be used to binarize the finished product image and the finished calibration plate image to separate the foreground and background. Through connected component analysis or contour detection, the blister packaging area can be extracted from the finished product image, and the calibration plate area can be extracted from the finished calibration plate image.

[0034] S102. Based on the product images captured at multiple testing stations and the positions of physical marker points in the calibration board images, determine the relative offset of the finished product image relative to the finished product calibration board image.

[0035] It should be understood that perspective distortion causes the position of the thermoformed product under inspection in the image to shift relative to the calibration plate. This shift can be quantified by comparing the positional differences of the physical marker points in different images. Specifically, by analyzing the offset relationship of the physical marker points between the product images at other inspection stations and the finished product inspection station, as well as the offset relationship between the physical marker points between the calibration plate images at other inspection stations and the finished product inspection station, it is possible to distinguish between systematic shifts caused by camera installation position and abnormal shifts caused by perspective distortion, thereby determining the actual offset of the thermoformed product relative to the calibration plate in the finished product inspection station image.

[0036] Other testing stations refer to the testing stations other than the finished product testing station among these multiple testing stations.

[0037] In one implementation of this application, the product offset vector of each other inspection station relative to the finished product image can be determined based on the position of physical marker points in product images captured at multiple inspection stations; the calibration board offset vector of each other inspection station relative to the finished calibration board image can be determined based on the position of physical marker points in calibration board images captured at multiple inspection stations; the finished product marker point offset distance can be determined based on the positional difference between the physical marker points in the finished product image and the physical marker points in the finished calibration board image; the relative offset degree can be determined based on the difference between the product offset vector and the calibration board offset vector corresponding to each other inspection station; and the relative offset amount can be determined by weighting and correcting the finished product marker point offset distance based on the relative offset degree.

[0038] The relative offset degree is used to characterize the difference in offset patterns between the thermoformed product under test and the calibration plate across multiple testing stations.

[0039] Optionally, a preset camera calibration algorithm (such as a camera calibration method based on a checkerboard or dot array) can be used to determine the intrinsic parameter matrix (including focal length and principal point coordinates) and camera coordinate system (such as a pixel coordinate system with the top left corner of the image as the origin) of the camera at each inspection station. The camera coordinate system of the finished product inspection station is then determined as the reference coordinate system. The extrinsic parameter matrix (including rotation matrix and translation vector) between each camera coordinate system and the reference coordinate system is also determined. For the pixel coordinates of the physical marker points identified in the product images of each other inspection station, the coordinates in the reference coordinate system are calculated based on the intrinsic and extrinsic parameters of the camera at that other inspection station through inverse camera projection operation or homography transformation. Then, vector operations are performed in the same coordinate system as the physical marker points corresponding to the finished product inspection station. The vector pointing from the position coordinates of the physical marker points in the product images of each other inspection station to the position coordinates of the physical marker points in the finished product image is denoted as the product offset vector.

[0040] Alternatively, the image processing algorithm can be a region of interest (ROI) tool.

[0041] Similarly, identify the position coordinates of the physical marker points in the calibration plate image of each testing station, and denote the vector pointing from the position coordinates of the physical marker points in the calibration plate image of each other testing station to the position coordinates of the physical marker points in the finished product calibration plate image as the calibration plate offset vector.

[0042] Additionally, the Euclidean distance between the physical marker points in the finished product image and the physical marker points in the finished product calibration plate image is denoted as the finished product marker point offset distance.

[0043] Understandably, the offset distance of the finished product mark point represents the spatial offset between the thermoformed product area and the calibration plate area at the finished product inspection station.

[0044] It should be understood that the calibration plate offset vector represents the offset relationship between the images taken by the standard calibration plate at different inspection stations and the images taken at the finished product inspection station, serving as the standard offset reference under distortion-free conditions.

[0045] In one optional implementation, the method for determining the relative offset degree based on the difference between the product offset vector and the calibration plate offset vector corresponding to each other inspection station is as follows: determine the absolute value of the difference in modulus length between the product offset vector and the calibration plate offset vector corresponding to each other inspection station; take the arithmetic mean of the absolute values ​​of the difference in modulus length corresponding to multiple other inspection stations to obtain the modulus mean; and normalize the modulus mean to obtain the relative offset degree.

[0046] Optionally, the relative offset satisfies the following formula:

[0047] in, Indicates the degree of relative offset. Indicates the first The position coordinates of physical marker points in the product images of other inspection stations. This indicates the position coordinates of the physical marker points in the product image at the finished product inspection station. Indicates the first The product offset vector corresponding to each of the other inspection stations. Indicates the first The coordinates of the physical marker points in the calibration plate image of another testing station. This indicates the position coordinates of the physical marker points in the calibration plate image of the finished product inspection station. Indicates the first The calibration plate offset vector corresponding to each of the other testing stations. Represents the magnitude of a vector (Euclidean distance). This indicates the number of other testing stations. This represents the normalization function, used to map the calculation results to the interval [0, 1].

[0048] It should be noted that the embodiments of this application limit multiple testing stations; therefore, the number of other testing stations should be greater than or equal to 1.

[0049] Optionally, the normalization function can be obtained by using the maximum and minimum value normalization method, with its maximum and minimum values ​​calibrated from boundary sample data during the system's historical normal operation.

[0050] It should be understood that the higher the relative offset, the greater the difference between the offset pattern of the thermoformed product under test and the standard offset pattern of the calibration plate between multiple testing stations. In other words, the higher the possibility of abnormal offset in the thermoformed product under test. This abnormal offset (i.e., the offset distance of the finished product mark point) is often caused by perspective distortion. The offset distance of the finished product mark point is more often caused by perspective distortion, and the relative offset should be larger. The lower the relative offset, the more likely this abnormal offset is caused by the camera installation position (or systematic installation error), and the smaller the relative offset should be.

[0051] The method described above for determining relative offset, by introducing comparative analysis of offset vectors from multiple inspection stations, calculates the degree of relative offset, which can distinguish between systematic offset caused by camera installation position and abnormal offset caused by perspective distortion. Then, by weighted correction of the offset distance of the marker points on the finished product inspection station, a more accurate relative offset caused by perspective distortion can be obtained.

[0052] S103. Based on the morphological differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, assess the degree of perspective distortion in the finished product image.

[0053] The degree of perspective distortion is used to characterize the degree of shape distortion of the area of ​​the thermoformed product. The higher the degree of perspective distortion, the greater the influence of perspective distortion on the area of ​​the thermoformed product and the more severe the local deformation.

[0054] It should be understood that perspective distortion causes localized deformation in the vacuum-formed product area, manifesting as inconsistent width changes in different local areas relative to the calibration plate area (i.e., near areas appear larger than distant areas). By comparing the differences in local width changes between the vacuum-formed product area and the calibration plate area in the same spatial direction, the severity of perspective distortion can be assessed.

[0055] In one implementation of this application, a displacement vector can be determined between the centroid of the calibration plate region in the finished product calibration plate image and the centroid of the thermoformed product region in the finished product image, and the direction of this displacement vector can be determined as the relative displacement direction; multiple detection lines perpendicular to the relative displacement direction can be constructed; the length of a first line segment intercepted by each detection line in the thermoformed product region and the length of a second line segment intercepted in the calibration plate region can be determined; a first difference between the lengths of the first line segments corresponding to each two adjacent detection lines and a second difference between the lengths of the second line segments corresponding to each two adjacent detection lines can be determined; and the degree of perspective distortion can be evaluated based on the difference between the first difference and the second difference corresponding to each two adjacent detection lines.

[0056] It should be understood that the centroid refers to the geometric center of the region, that is, the position point corresponding to the arithmetic mean of the coordinates of all pixels in the region. The centroid of the thermoforming product region is the geometric center of the thermoforming product region, and the centroid of the calibration plate region is the geometric center of the calibration plate region.

[0057] Optionally, the centroid coordinates of the thermoformed product area and the calibration plate area can be calculated separately using image processing algorithms. A displacement vector is constructed with the centroid of the calibration plate area as the starting point and the centroid of the thermoformed product area as the ending point. The direction of this displacement vector is the relative displacement direction, which represents the overall offset direction of the thermoformed product area relative to the calibration plate area.

[0058] It should be understood that multiple detection lines pass through both the thermoformed product area and the calibration plate area simultaneously along a direction perpendicular to the centroid line, and perform local morphological sampling detection lines at equal intervals in both areas.

[0059] Optionally, the number of detection lines can be determined based on the image resolution and computational accuracy requirements; for example, an empirical value of 50 lines can be used.

[0060] Optionally, for each detection line, record the two intersection points with the boundary of the thermoformed product area, and use the distance between the two intersection points as the first line segment length; similarly, record the two intersection points with the boundary of the calibration plate area for each detection line, and calculate the distance between the two intersection points as the second line segment length.

[0061] It should be understood that the lengths of the first and second segments corresponding to a detection line reflect the local widths of the thermoformed product area and the calibration plate area at that detection line, respectively.

[0062] Optionally, the detection lines are arranged in order from left to right (or from top to bottom). For two adjacent detection lines, the difference between the length of the first segment corresponding to the latter detection line and the length of the first segment corresponding to the former detection line is calculated and recorded as the first difference; similarly, the difference between the lengths of the second segment is calculated and recorded as the second difference. By traversing all adjacent detection lines, the first difference sequence and the second difference sequence are obtained.

[0063] It should be understood that the first and second differences reflect the rate of change of local width between the thermoformed product area and the calibration plate area at adjacent sampling locations.

[0064] Understandably, perspective distortion causes near-to-far areas of the vacuum-formed product to appear larger than distant areas, resulting in a different rate of width change compared to the standard calibration plate area. By comparing the first and second differences, the degree of this local shape distortion can be quantified.

[0065] In one alternative implementation, the degree of width variation between the thermoformed product area and the calibration plate area can be determined based on the absolute value of the difference between the first difference and the second difference corresponding to each two adjacent detection lines; the product of the degree of width variation and the degree of relative offset is determined as the degree of perspective distortion.

[0066] It should be understood that the larger the absolute value of the difference, the greater the deviation of the width change of the thermoformed product area at that local position relative to the calibration plate area, that is, the more obvious the near-large and far-small distortion feature exists.

[0067] Optionally, the degree of perspective distortion satisfies the following formula:

[0068] in, Indicates the degree of perspective distortion. This represents the total number of lines detected. This indicates the number of adjacent detection lines. Indicates the first The length of the first line segment corresponding to the detected straight line is related to the length of the second line segment. The first difference between the lengths of the first line segments corresponding to the detected straight lines. Indicates the first The length of the second line segment corresponding to the detected straight line is related to the length of the first line segment. The second difference between the lengths of the second line segments corresponding to the detected straight lines. This represents the absolute value operation. This indicates the operation of finding the maximum value. This represents a smoothing constant (used to avoid a denominator of zero; for example, it takes a value of 0.1 pixels). Indicates the degree of relative offset.

[0069] Based on this formula, it should be understood that the first and second differences can be positive or negative. A positive value indicates that the width of the region along the detection line is increasing, while a negative value indicates that the width of the region is decreasing. The difference is determined by taking the absolute value of the difference between the first and second differences. To eliminate directional influences, only the degree of difference in the width change trends of the two regions is retained.

[0070] In this formula, This indicates the degree of difference in width variation, which characterizes the cumulative difference in the rate of change of the local width of the thermoformed product area relative to the calibration plate area at all sampling locations. The denominator... Used for molecular normalization to ensure that changes in line segment lengths of different orders of magnitude do not affect the relative comparison of differences.

[0071] It is understandable that when there is perspective distortion in the area of ​​the thermoformed product, its local width change rate will systematically deviate from the calibration plate area, resulting in an increase in the absolute value of the difference, which in turn increases the degree of perspective distortion. By multiplying by the relative offset degree, the distortion assessment deviation caused by the difference in the overall offset mode can be further corrected.

[0072] The aforementioned method for determining the degree of perspective distortion involves constructing a detection line perpendicular to the displacement direction to sample the local morphology of the vacuum-formed product area and the calibration plate area. Then, it compares the differences in line segment length changes at adjacent sampling locations to accurately detect the perspective distortion characteristic of near objects appearing larger than distant ones. This method, which assesses distortion by analyzing the rate of change of local geometric shapes, has good universality. Furthermore, combining this method with a weighted average based on the relative offset improves the accuracy and robustness of distortion assessment, providing a reliable quantitative indicator of distortion degree for subsequent defect compensation.

[0073] S104. Based on the relative offset and the degree of perspective distortion, perform perspective transformation calibration on the finished product image.

[0074] It should be understood that the calibrated finished product images are used for intelligent defect detection.

[0075] In one implementation of this application, edge detection can be performed on each product image to identify defective areas in each product image; if at least one common defective area exists in multiple product images, the degree of compensation optimization is determined based on the degree of perspective distortion and the feature parameters of the common defective area in each product image; the relative offset is weighted and corrected based on the degree of compensation optimization to obtain the final offset; and the finished product image is calibrated by perspective transformation based on the final offset.

[0076] The degree of compensation optimization is used to characterize the weight of correcting the relative offset. The characteristic parameters include the centroid position of the common defect area and the area ratio of the area to the thermoformed product.

[0077] It should be understood that a defect area is the imaged area in an image of a geometric defect or surface damage present on the surface of the thermoformed product to be inspected. A common defect area refers to the same physical defect area that can be identified in product images at multiple inspection stations (such as the feeding station and the finished product inspection station).

[0078] Optionally, edge detection algorithms (such as Canny operator, Sobel operator, etc.) can be used to process the product images at each inspection station to identify closed edge curves in the thermoforming product area. The area formed by these closed edge curves is the defect area (such as gaps, scratches, etc.).

[0079] Optionally, the product images of the thermoformed product to be inspected at each inspection station are overlaid, and the defect areas that overlap in all product images are marked as common defect areas.

[0080] It should be understood that perspective distortion affects defects of different sizes differently. For defects with large areas (such as large gaps or long-span scratches), perspective distortion may cause the defect to be partially obscured or deformed and difficult to identify, so targeted compensation is required.

[0081] In one alternative implementation, the method for determining the degree of compensation optimization is as follows: based on the centroid position of each common defect region, determine the offset vector of each common defect region relative to the physical marker point in each product image; based on the offset vector of each common defect region relative to the physical marker point in each product image, determine the similarity parameter of each common defect region among multiple product images; and based on the area ratio, similarity parameter, and perspective distortion degree, determine the degree of compensation optimization.

[0082] The similarity parameter is used to characterize the consistency of the positional offset of the same common defect area in different product images.

[0083] First, the vector pointing from the coordinates of the centroid of a common defect area in a product image to the coordinates of a physical marker point in the product image is determined as the offset vector.

[0084] Secondly, the cosine similarity of the offset vectors between each pair of detection stations is calculated, and the average of all the obtained cosine similarities is used to obtain the similarity parameter.

[0085] It should be understood that the larger the similarity parameter (closer to 1), the less the common defect area is affected by perspective distortion; if the similarity parameter is smaller (closer to 0), the more the common defect area is significantly shifted in position under the shooting angle of different inspection stations, and is greatly affected by perspective distortion.

[0086] Finally, the degree of compensation optimization is determined based on the area ratio, similarity parameters, and perspective distortion.

[0087] It should be understood that the larger the area ratio and the smaller the similarity parameter (i.e., the larger the defect and the more inconsistent the offset), the more severely the common defect area is affected by perspective distortion, and the higher the degree of compensation and optimization is required.

[0088] Optionally, the mean area proportion of a common defect region in all product images can be determined, and then the mean can be divided by the similarity parameter to obtain the degree of distortion influence. After averaging the degree of distortion influence of all common defect regions, the result can be multiplied by the degree of perspective distortion to obtain the degree of compensation optimization.

[0089] Optionally, the degree of compensation optimization satisfies the following formula:

[0090] in, Indicates the degree of compensation optimization. Indicates the degree of perspective distortion. Indicates the number of common defect areas ( ), Indicates the first The average area proportion of a common defective region across multiple product images Indicates the first Similarity parameters of shared defect regions This represents a normalization function, such as maximum and minimum value normalization, used to map the calculation results to the interval [0, 1].

[0091] In this formula, Reflects the first The degree of distortion impact of each shared defect area, and the average area proportion. The larger, the more The higher the potential risk of distortion affecting a shared defect region, the higher the similarity parameter. The smaller, the first The greater the offset of a shared defect region under different viewpoints, the more severe its actual distortion impact. This can be determined by summing the distortion impact of all shared defect regions and dividing by the total number of shared defect regions. The average level of influence is obtained, and then multiplied by the degree of perspective distortion to obtain the degree of compensation optimization that comprehensively considers the overall distortion level and the individual characteristics of defects.

[0092] It should be understood that the greater the degree of compensation optimization, the more significant the correction of the defective area needs to be to compensate for the distortion effect of the defective area, and a larger offset should be used; conversely, the smaller the degree of compensation optimization, the smaller the offset should be used.

[0093] In another alternative implementation, when there is no shared defect area, no compensation optimization is required, and the relative offset is directly determined as the final offset.

[0094] In one optional implementation, the method for calibrating the perspective transformation of the finished product image based on the final offset is as follows: determining the amplitude parameter of the perspective transformation based on the final offset; determining the offset trend direction based on the relative position of the physical marker points in the finished product image and the physical marker points in the finished product calibration plate image; constructing a perspective transformation matrix based on the amplitude parameter and the offset trend direction; and performing back projection resampling on the finished product image based on the perspective transformation matrix to obtain the calibrated finished product image.

[0095] The offset trend direction is used to characterize the offset direction of the thermoformed product to be tested relative to the calibration plate, and the perspective transformation matrix is ​​used to characterize the coordinate mapping relationship from the current distortion state to the standard distortion-free state.

[0096] It should be understood that the amplitude parameter characterizes the degree of deviation of the finished product image from the distortion-free standard state. The larger the value, the greater the intensity of perspective correction that needs to be performed. The amplitude parameter is positively correlated with the value of the final offset.

[0097] Alternatively, the final offset can be used directly as the amplitude parameter.

[0098] Optionally, the vector direction from the physical marker point in the finished product calibration plate image to the physical marker point in the finished product image can be determined as the offset trend direction.

[0099] Optionally, the angle value of the product in the image coordinate system is calculated based on the offset trend direction, and the four corner points of the finished product image are translated in the opposite direction of the offset trend direction and the distance represented by the amplitude parameter to obtain four target vertices; based on the coordinate mapping relationship between the four corner points and their corresponding target vertices, the perspective transformation matrix is ​​obtained by using a direct linear transformation algorithm or a calculation method based on the homography matrix.

[0100] Finally, based on the perspective transformation matrix, the coordinates of each pixel in the finished product image are transformed, and the corresponding target coordinates of each pixel after calibration are calculated.

[0101] Optionally, for each target pixel position in the calibrated finished product image, its corresponding source coordinates in the uncalibrated finished product image are calculated using the inverse of the perspective transformation matrix. When the source coordinates are non-integer coordinates, bilinear interpolation or bicubic interpolation is used to interpolate the grayscale values ​​of pixels surrounding the source coordinates to determine the grayscale value of the target pixel. By traversing all pixel positions in the calibrated finished product image, a calibrated image after perspective distortion correction is obtained.

[0102] It should be understood that in the calibrated image, the deformation of the thermoformed product area is corrected, and the geometric features of the defect are restored to normal, which facilitates accurate defect detection by subsequent defect recognition algorithms (such as edge detection, deep learning classification, etc.).

[0103] The method provided in S104 above introduces a compensation and optimization mechanism for defect areas before perspective transformation calibration. It focuses on compensating for defect areas with large areas and significant positional shifts in images at different workstations, effectively solving the problem of difficulty in identifying large defects caused by perspective distortion. By analyzing the area ratio of the defect area and the consistency of its position across workstations (similarity parameter), it accurately assesses the actual impact of distortion on the defect and adjusts the calibration offset accordingly. This ensures that the calibrated image not only eliminates overall distortion but also pays special attention to the geometric restoration of the defect area, which can help improve the accuracy of subsequent defect detection.

[0104] Based on the methods provided in S101-S104 above, this embodiment establishes a spatial reference by acquiring images from multiple inspection stations. By combining the offset relationship and morphological differences between the thermoformed product to be inspected and the calibration plate, the degree and direction of perspective distortion are assessed, and perspective transformation calibration is performed accordingly. Through data fusion from multiple inspection stations, systematic offset and distortion offset can be distinguished, improving the accuracy of distortion detection. Combined with the morphological characteristics of the thermoformed product to be inspected, the universality for thermoformed products of different shapes is improved. Finally, perspective transformation calibration eliminates the impact of distortion on defect detection, thereby improving the accuracy of defect detection when performing subsequent defect detection based on the calibrated image, and ultimately improving the production quality of thermoformed products based on industrial vision.

[0105] This application also provides an intelligent defect detection system for thermoformed products based on industrial vision, such as... Figure 2 As shown, the intelligent defect detection system 20 for thermoformed products based on industrial vision includes an image acquisition module 201, an image analysis module 202, and a calibration module 203.

[0106] The image acquisition module 201 is used to acquire product images of the thermoformed product to be inspected and calibration plate images of the calibration plate, which are taken at multiple inspection stations respectively.

[0107] The blister product to be tested and the calibration plate both contain preset physical marking points, and the multiple testing stations include the finished product testing station.

[0108] The image analysis module 202 is used to determine the relative offset of the finished product image with respect to the finished product calibration board image based on the positions of physical marker points in the product images captured at multiple inspection stations and the calibration board image.

[0109] Among them, the finished product image is a product image taken at the finished product inspection station, and the finished product calibration plate image is a calibration plate image taken at the finished product inspection station.

[0110] The image analysis module 202 is also used to assess the degree of perspective distortion in the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image.

[0111] The degree of perspective distortion is used to characterize the degree of morphological distortion of the area of ​​the thermoformed product.

[0112] The calibration module 203 is used to perform perspective transformation calibration on the finished product image based on the relative offset and the degree of perspective distortion.

[0113] Among them, the calibrated finished product images are used for intelligent defect detection.

[0114] Optionally, the industrial vision-based intelligent detection system 20 for defects in thermoformed products can also execute any of the aforementioned optional industrial vision-based intelligent detection methods for defects in thermoformed products.

[0115] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0116] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for intelligent defect detection in thermoformed products based on industrial vision, characterized in that, include: The system acquires product images of the thermoformed product to be tested and calibration plate images of the calibration plate, which are taken at multiple testing stations. Both the thermoformed product to be tested and the calibration plate contain preset physical marker points. The multiple testing stations include a finished product testing station. Based on the product images taken at multiple inspection stations and the positions of physical marker points in the calibration board image, the relative offset of the finished product image relative to the finished product calibration board image is determined. The finished product image is the product image taken at the finished product inspection station, and the finished product calibration board image is the calibration board image taken at the finished product inspection station. Based on the morphological differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, the degree of perspective distortion in the finished product image is evaluated. The degree of perspective distortion is used to characterize the degree of morphological distortion of the thermoformed product area. Based on the relative offset and the degree of perspective distortion, the finished product image is calibrated by perspective transformation, and the calibrated finished product image is used for intelligent defect detection.

2. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 1, characterized in that, The determination of the relative offset of the finished product image relative to the finished product calibration board image, based on the positions of physical marker points in product images captured at multiple testing stations and calibration board images, includes: Based on the position of physical markers in the product images captured at multiple inspection stations, the product offset vector of each other inspection station relative to the finished product image is determined. The other inspection stations are the inspection stations other than the finished product inspection station among the multiple inspection stations. Based on the positions of physical marker points in calibration plate images captured at multiple testing stations, determine the calibration plate offset vector of each other testing station's calibration plate image relative to the finished product calibration plate image; Based on the positional difference between the physical markers in the finished product image and the physical markers in the finished product calibration plate image, the offset distance of the finished product markers is determined. Based on the difference between the product offset vector and the calibration plate offset vector corresponding to each other detection station, the relative offset degree is determined. The relative offset degree is used to characterize the degree of difference in the offset patterns between the thermoformed product to be tested and the calibration plate among multiple detection stations. The relative offset is determined by weighting and correcting the offset distance of the finished product marker point based on the relative offset degree.

3. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 2, characterized in that, The determination of the relative offset degree based on the difference between the product offset vector and the calibration board offset vector corresponding to each other testing station includes: Determine the modulus of the difference between the product offset vector and the calibration plate offset vector for each other inspection station; The average mold length is obtained by averaging the mold lengths corresponding to multiple other inspection stations. The mean modulus is normalized to obtain the relative offset.

4. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 2 or 3, characterized in that, The assessment of perspective distortion in the finished product image, based on the morphological differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, includes: Determine the displacement vector between the centroid of the calibration plate area in the finished product calibration plate image and the centroid of the thermoformed product area in the finished product image, and define the direction of the displacement vector as the relative displacement direction; Construct multiple detection lines perpendicular to the direction of the relative displacement; Determine the length of the first segment of each detection line within the area of ​​the thermoformed product, and the length of the second segment within the area of ​​the calibration plate; Determine the first difference between the lengths of the first line segments corresponding to each pair of adjacent detection lines, and the second difference between the lengths of the corresponding second line segments; The degree of perspective distortion is assessed based on the difference between the first and second differences corresponding to each pair of adjacent detection lines.

5. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 4, characterized in that, The assessment of the degree of perspective distortion based on the difference between the first and second differences corresponding to every two adjacent detection lines includes: Based on the absolute value of the difference between the first difference and the second difference corresponding to each two adjacent detection lines, the degree of difference in width variation between the thermoformed product area and the calibration plate area is determined. The product of the difference in width variation and the degree of relative offset is determined as the degree of perspective distortion.

6. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 1, characterized in that, The perspective transformation calibration of the finished product image based on the relative offset and the degree of perspective distortion includes: Edge detection is performed on each product image to identify defective areas in each product image; In the case where there is at least one common defect area in multiple product images, the degree of compensation optimization is determined based on the degree of perspective distortion and the feature parameters of the common defect area in each product image. The degree of compensation optimization is used to characterize the weight of correcting the relative offset. The feature parameters include the centroid position of the common defect area and the area ratio of the common defect area to the area of ​​the thermoformed product. The relative offset is weighted and corrected based on the degree of compensation optimization to obtain the final offset. The final offset is used to perform perspective transformation calibration on the finished product image.

7. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 6, characterized in that, The determination of the compensation optimization level based on the degree of perspective distortion and the feature parameters of common defect areas in each product image includes: Based on the centroid location of each common defect region, determine the offset vector of each common defect region relative to the physical marker point in each product image; Based on the offset vector of each common defect region relative to the physical marker point in each product image, a similarity parameter of each common defect region among multiple product images is determined. The similarity parameter is used to characterize the consistency of the positional offset of the same common defect region in different product images. The degree of compensation optimization is determined based on the ratio of the area ratio to the similarity parameter and the degree of perspective distortion.

8. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 6, characterized in that, The perspective transformation calibration of the finished product image based on the final offset includes: The magnitude parameter of the perspective transformation is determined based on the final offset. The offset trend direction is determined based on the relative position of the physical marker points in the finished product image and the physical marker points in the finished product calibration plate image. The offset trend direction is used to characterize the offset direction of the thermoformed product to be tested relative to the calibration plate. A perspective transformation matrix is ​​constructed based on the amplitude parameter and the offset trend direction. The perspective transformation matrix is ​​used to characterize the coordinate mapping relationship from the current distortion state to the standard distortion-free state. The finished product image is resampled by back projection based on the perspective transformation matrix to obtain the calibrated finished product image.

9. The intelligent defect detection method for thermoformed products based on industrial vision according to claim 1, characterized in that, Before assessing the degree of perspective distortion in the finished product image based on the morphological differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image, the following steps are also included: The threshold segmentation algorithm is used to extract the thermoformed product area from the finished product image and the calibration plate area from the finished calibration plate image.

10. An intelligent defect detection system for thermoformed products based on industrial vision, characterized in that, It includes an image acquisition module, an image analysis module, and a calibration module: The image acquisition module is used to acquire product images of the thermoformed product to be inspected and calibration plate images of the calibration plate, which are taken at multiple inspection stations respectively. Both the thermoformed product to be inspected and the calibration plate contain preset physical marker points. The multiple inspection stations include a finished product inspection station. The image analysis module is used to determine the relative offset of the finished product image with respect to the finished product calibration board image based on the positions of physical marker points in the product images taken at multiple inspection stations and the calibration board image. The finished product image is the product image taken at the finished product inspection station, and the finished product calibration board image is the calibration board image taken at the finished product inspection station. The image analysis module is also used to evaluate the degree of perspective distortion of the finished product image based on the morphological feature differences between the thermoformed product area in the finished product image and the calibration plate area in the finished product calibration plate image. The degree of perspective distortion is used to characterize the degree of morphological distortion of the thermoformed product area. The calibration module is used to perform perspective transformation calibration on the finished product image based on the relative offset and the degree of perspective distortion. The calibrated finished product image is used for intelligent defect detection.

Images