An artificial intelligence vision-based automated product quality inspection method and system

Abstract

The application relates to the technical field of product quality inspection, and provides an automatic product quality inspection method and system based on artificial intelligence vision, which comprises the following steps: acquiring an image sequence of a mobile screen through an overhead camera on a production line, extracting a screen contour, a motion direction vector and a moving speed, and establishing a motion reference; after frame-by-frame extraction of a screen image, adopting multi-scale segmentation combining region growing and edge constraint to obtain a to-be-judged connected domain with a centroid coordinate, an area and a gray feature; through comparison of interframe displacement of the to-be-judged element and screen reference displacement, and by using the characteristics that a real defect moves synchronously with a screen and a reflection does not follow the movement, false images are accurately removed, and defect candidates are screened; then, a defect enhancement image is generated through motion compensation alignment and superposition, and finally, a trained classifier is used to accurately classify defects such as scratches and concave points, so that the anti-interference ability and recognition accuracy of screen quality inspection are improved.

Description

Technical Field

[0001] This invention relates to the field of product quality inspection technology, and more specifically, to an automated product quality inspection method and system based on artificial intelligence vision. Background Technology

[0002] The content in this section only provides background information related to this invention and may not constitute prior art.

[0003] With the rapid and large-scale development of industries such as display panels and smart terminals, the outgoing quality inspection of screen products has become a core link in ensuring product yield and controlling production quality. Automated quality inspection solutions based on machine vision, with their non-contact and high-efficiency technical advantages, are gradually replacing traditional manual visual inspection methods and becoming the mainstream application solution for screen production line quality inspection.

[0004] Current mainstream automated quality inspection technologies for screens mostly employ segmentation algorithms and feature extraction methods based on single-frame images to identify defects. In actual inspection scenarios with continuous production line operation, the high gloss characteristics of the screen surface easily generate artifacts such as environmental reflections and localized strong light reflections. The grayscale distribution and edge morphology features of these artifacts within a single-frame image are highly similar to real defects such as scratches, dents, and dirt on the screen. Existing technologies cannot effectively distinguish between real defects and reflective artifacts, which can easily lead to serious false positives and false negatives, significantly reducing the reliability of quality inspection results.

[0005] Meanwhile, existing technologies are ill-suited for dynamic inspection scenarios where production line screens are constantly moving. They cannot accurately match target features between frames, nor can they leverage motion information from multi-frame image sequences to enhance subtle defects or eliminate artifact interference through inter-frame motion patterns. This results in insufficient detection capabilities for low-contrast defects such as minor scratches and tiny dents, failing to meet the quality inspection accuracy requirements of high-precision screen products and severely restricting the automation level and production efficiency of production line quality inspection. Summary of the Invention

[0006] The purpose of this invention is to provide an automated product quality inspection method and system based on artificial intelligence vision, in order to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows: In a first aspect, this application provides an automated product quality inspection method based on artificial intelligence vision, comprising: By using a camera mounted above the production line to acquire image sequences containing screen movement, the rectangular outline of the screen, the screen's motion direction vector, and the screen's movement speed are extracted. Based on the rectangular outline and the movement speed, the screen position is identified frame by frame in the image sequence, and the corresponding screen image is extracted from each frame. For each frame of screen image, perform multi-scale segmentation based on region growing and edge constraints to obtain multiple independent connected components as elements to be judged, and record the centroid coordinates, connected component area and average gray value of each element to be judged. For the element to be judged in two adjacent screen images, calculate the displacement vector of the centroid coordinates in the screen image coordinate system, and compare the displacement vector with the inter-frame reference displacement vector determined by the screen's motion direction vector and movement speed; if the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, the corresponding element to be judged is marked as a candidate for following defect; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point. For all following defect candidates, inter-frame motion compensation is performed using the screen's motion direction vector and movement speed. Connected components belonging to the same following defect candidate in multiple consecutive frames are aligned and superimposed to generate a defect enhancement image. The enhanced image of the defect is input into the trained defect classifier. The defect classifier outputs a defect category judgment result that includes at least scratches, dents and dirt, based on the area of ​​connected regions, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

[0007] Further, the rectangular outline of the screen is extracted, specifically including: The image sequence is converted to grayscale, and then filtered and smoothed to eliminate random noise generated during image acquisition while preserving the edge boundary information of objects in the image. Perform edge enhancement processing on the smoothed image to amplify the contrast of areas with abrupt grayscale changes and highlight the grayscale difference between the screen edges and the surrounding background. Edge detection is performed on the image after edge enhancement to extract all closed edge contours in the image and form an initial contour set; Perform geometric feature filtering on all closed edge contours within the initial contour set, retaining contours that conform to the geometric properties of rectangles to form a candidate rectangular contour set; By combining the imaging ratio corresponding to the inherent physical size of the screen to be inspected on the production line, and the preset layout area of ​​the screen in the production line, a secondary screening is performed on the candidate rectangular contour set to remove invalid contours that do not match the inherent attributes and layout position of the screen, and the target rectangular contour is obtained.

[0008] Furthermore, screen positions are identified frame by frame in the image sequence, specifically including: Using the rectangular outline of the screen extracted from the previous frame in the image sequence as a reference, and combining the screen's motion direction vector and movement speed, the search range of the screen position in the current frame image is defined, and the rectangular outline of the screen is identified and extracted within the search range.

[0009] Furthermore, multi-scale segmentation based on region growing and edge constraints is performed, specifically including: The grayscale continuity of pixels in the screen image is used as the criterion for region growing, and the grayscale abrupt change boundary of pixels in the screen image is used as the limiting condition for edge constraint to perform multi-scale segmentation of the screen image.

[0010] Furthermore, the screen image is segmented at multiple scales, specifically including: The screen image is segmented at a preset initial scale to obtain each initial connected component and its corresponding gray mean and gray distribution dispersion. For each initially connected region, when the gray-level distribution dispersion exceeds the preset uniformity limit, the scale is reduced by a preset ratio, and the corresponding region of the connected region is further refined and segmented to correct the boundary position; when the gray-level distribution dispersion is equal to or lower than the preset uniformity limit, the current scale is maintained or the scale is expanded by a preset ratio, and the corresponding region of the connected region is merged to obtain multiple independent connected regions.

[0011] Furthermore, inter-frame motion compensation is performed using the screen's motion direction vector and movement rate, specifically including: Based on the rectangular outline of the screen in the previous frame image, the initial position of the rectangular outline of the screen in the current frame image is predicted according to the motion direction vector and the movement speed. Using the initial position as the center, a preset pixel range is expanded outward as the search window. The gradient magnitude of each pixel is calculated within the search window, and a non-maximum suppression algorithm is used to retain the local maxima of the gradient magnitude as edge candidate points. Hough transform line detection is performed on edge candidate points to filter out four lines that satisfy the rectangular geometric constraints. The intersection of the four lines is taken as the vertex of the screen rectangle outline in the current frame image, thus obtaining the accurate outline of the screen in the current frame image. Based on the precise contour, connected components belonging to the same defect candidate in multiple consecutive frames are aligned to the same coordinate system through affine transformation, and the aligned connected components are weighted and superimposed with pixel gray values ​​to generate a defect enhancement image.

[0012] Furthermore, the trained defect classifier is constructed through the following steps: Pre-collect screen defect sample images containing scratch type, dent type and dirt type, and label each screen defect sample image with its corresponding defect category label; For each screen defect sample image, the connected component area parameter, the average gray value parameter, and the edge morphology feature parameter extracted based on the gradient orientation histogram are extracted, and the connected component area parameter, the average gray value parameter, and the edge morphology feature parameter are combined to form the feature vector of the screen defect sample image. The feature vectors and defect category labels corresponding to all screen defect sample images are input into the support vector machine for supervised training. The hyperparameters of the support vector machine are adjusted until the classification prediction accuracy of the defect category meets the preset convergence condition, thus obtaining the trained defect classifier.

[0013] Furthermore, the step of pre-collecting screen defect sample images including scratch type, dent type, and dirt type also includes: Data augmentation processing is performed on all screen defect sample images. The augmentation process includes rotating the screen defect sample images, mirroring them, and adjusting their brightness to increase the number of samples and the scene coverage.

[0014] Furthermore, after outputting the defect category determination results, which include at least scratches, dents, and dirt, the following are also included: The defect category determination result is associated with and stored with the corresponding screen image and centroid coordinate information, and the defect category determination result is pushed to the preset terminal on the production line in a synchronous manner.

[0015] Secondly, this application also provides an automated product quality inspection system based on artificial intelligence vision, comprising: The data acquisition module is used to acquire image sequences containing screen movement through a camera mounted above the production line, and to extract the rectangular outline of the screen, the screen's motion direction vector, and the screen's movement speed. The screen image extraction module is used to identify the screen position frame by frame in the image sequence based on the rectangular outline and the movement speed, and extract the corresponding screen image from each frame. The segmentation module is used to perform multi-scale segmentation based on region growing and edge constraints on each frame of screen image, obtain multiple independent connected components as elements to be judged, and record the centroid coordinates, connected component area and average gray value of each element to be judged. The displacement comparison module is used to calculate the displacement vector of the centroid coordinates in the screen image coordinate system for the elements to be judged in two adjacent frames of screen images, and compare the displacement vector with the inter-frame reference displacement vector determined by the motion direction vector and the movement speed of the screen; if the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, the corresponding element to be judged is marked as a candidate for following defect; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point. The compensation module is used to perform inter-frame motion compensation for all following defect candidates using the screen's motion direction vector and movement speed, and to align and superimpose connected components belonging to the same following defect candidate in multiple consecutive frames to generate a defect enhancement image. The defect recognition module is used to input the enhanced defect image into the trained defect classifier. The defect classifier outputs a defect category judgment result that includes at least scratches, dents and dirt based on the area of ​​connected components, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

[0016] The beneficial effects of this invention are as follows: This invention acquires image sequences containing screen movement using a camera above the production line, simultaneously extracting the screen's rectangular outline, motion direction vector, and movement speed to establish a precise spatiotemporal motion benchmark for the entire inspection process. This provides a core reference for distinguishing between real defects that move with the screen and static reflective artifacts. Furthermore, based on the screen outline and movement speed, the invention identifies the screen position frame by frame and extracts the corresponding screen image, accurately defining the detection area, eliminating background interference from the production line, and ensuring the spatiotemporal consistency of the detection area between frames. Subsequently, through multi-scale segmentation based on region growing and edge constraints, independent connected components with centroid coordinates, connected component area, and average gray value are extracted from each frame of screen image. Then, the inter-frame displacement vectors of the elements to be judged in adjacent frames are compared with the reference displacement vectors corresponding to the screen movement. Taking advantage of the core difference that real defects move synchronously with the screen and artifacts such as reflections do not follow the screen movement, reflections and reflection artifacts with excessive displacement deviations are accurately eliminated, and candidates that follow the defect are selected, which completely solves the core problem of reflection interference in defect recognition. Then, inter-frame motion compensation and alignment superposition are performed on the defect candidates to generate defect enhancement images to strengthen weak defect features and reduce the false negative rate of small defects. Finally, through the trained defect classifier, combined with connected component area, average gray value, and edge morphology features, the results of defect category judgment such as scratches, dents, and dirt are accurately output, improving the accuracy of automated screen quality inspection and environmental anti-interference ability. Attached Figure Description

[0017] Figure 1 A flowchart of an automated product quality inspection method based on artificial intelligence vision provided by the present invention; Figure 2 This is a schematic diagram showing the positions of the camera and the screen in this invention; Figure 3 This is a schematic diagram illustrating the changes of the element to be judged in the previous frame and the current frame in this invention; Figure 4 This is a schematic diagram of an automated product quality inspection system based on artificial intelligence vision provided by the present invention.

[0018] In the diagram: 1. Camera; 2. Screen; 3. Conveyor belt; 201. Data acquisition module; 202. Screen image extraction module; 203. Segmentation module; 204. Displacement comparison module; 205. Compensation module; 206. Defect recognition module. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0020] like Figure 1 As shown in the embodiment of the present invention, an automated product quality inspection method based on artificial intelligence vision includes: S101, the camera 1 mounted above the production line acquires an image sequence containing the movement of the screen 2, and extracts the rectangular outline of the screen 2, the motion direction vector of the screen 2, and the movement speed of the screen 2.

[0021] Specifically, camera 1 (high-definition industrial camera) is used to collect and capture visual information from the dynamic screen 2 on the production line, laying the foundation for subsequent screen image extraction and defect detection; at the same time, camera 1 is mounted above the production line, such as... Figure 2 As shown, it can realize real-time, no-dead-angle acquisition of the continuously moving screen 2 on the production line, avoiding the subjectivity and lag of manual acquisition. At the same time, it can simultaneously extract the rectangular outline, motion direction vector and movement speed of the screen 2, quickly lock the detection target, reduce the amount of calculation for subsequent inter-frame recognition, improve quality inspection efficiency, and meet the real-time detection requirements of automated production lines.

[0022] Extracting the rectangular outline of screen 2 is the core step in achieving screen 2 positioning, and specifically includes: First, grayscale processing is performed on single-frame images in the image sequence. The principle of grayscale processing is to convert the RGB three-channel pixel values ​​of the color image into single-channel grayscale values ​​through the Gamma correction algorithm, thereby eliminating the interference of color information, reducing the amount of data for image processing, and at the same time preserving the brightness difference between screen 2 and the background, providing a clear pixel basis for subsequent edge extraction.

[0023] Second, the image after grayscale processing is filtered and smoothed to eliminate random noise generated during image acquisition while preserving the edge boundary information of objects in the image. The principle is to use a 3×3 Gaussian filter convolution kernel (standard deviation set to 1.2) and perform weighted summation on the pixels of the grayscale image through a sliding window to suppress Gaussian noise and random black and white spots caused by sensor malfunction, thereby improving the accuracy of subsequent edge extraction.

[0024] Third, edge enhancement processing is performed on the smoothed image to amplify the contrast of gray-level abrupt change areas in the image and highlight the gray-level difference between the screen edge and the surrounding background area. The principle is to use the Laplacian operator to perform convolution operation on the smoothed image to enhance the pixel difference in gray-level abrupt change areas, thereby further improving the contrast of the screen edge (where gray-level changes occur).

[0025] Fourth, edge detection is performed on the image after edge enhancement to extract all closed edge contours in the image, forming an initial contour set. The principle is to use the Canny edge detection algorithm, first performing secondary noise reduction through Gaussian filtering, then calculating the pixel gradient magnitude and direction, refining the edges through non-maximum suppression, filtering out true edges using a dual thresholding method (high threshold 180, low threshold 80), and finally connecting discrete edge points using a contour extraction algorithm to form closed edge contours. This extracts all closed contours in the image, including both screen contours and background interference contours (some components on conveyor belt 3), providing a complete contour data source for subsequent filtering and avoiding the omission of screen contours.

[0026] Fifth, geometric feature filtering is performed on all closed edge contours within the initial contour set, retaining contours that conform to the geometric properties of rectangles to form a candidate rectangular contour set. The principle is to calculate the geometric parameters of each closed contour, including the number of sides, aspect ratio (the standard screen aspect ratio is 16:9), and fill rate (the ratio of the contour area to the area of ​​the circumscribed rectangle ≥ 0.9). Contours with 4 sides, an aspect ratio within the range of 16:9 ± 0.2, and a fill rate that meets the requirements are selected. Its beneficial effects include eliminating non-rectangular interference contours such as circles and irregular polygons, narrowing the target contour range, reducing the computational load of subsequent secondary filtering, and improving the targeting of contour extraction.

[0027] Sixth, combining the imaging ratio corresponding to the inherent physical size of the screen 2 to be tested on the production line, and the preset layout area of ​​the screen 2 on the production line, a secondary screening is performed on the candidate rectangular outline set to remove invalid outlines that do not match the inherent attributes and layout position of the screen 2, thus obtaining the target rectangular outline. The principle is to pre-input the inherent physical size of the screen 2 to be tested (for example, the physical size of a 5.5-inch mobile phone screen is 121.8mm × 68.5mm), calculate the imaging ratio (1mm corresponds to 10 pixels) by combining the imaging focal length of the camera 1 (preset to 10mm), and at the same time, preset the layout area of ​​the screen 2 on the production line (such as the middle area of ​​the conveyor belt 3, with the y-axis coordinate range in the image coordinate system being 200-800 pixels), compare the imaging size and position of the candidate rectangular outlines, and remove outlines with size deviations exceeding 5% or positions exceeding the preset area, thereby completely eliminating invalid interference outlines and accurately locking the rectangular outline of the screen 2 to be tested.

[0028] The relevant calculation formulas are as follows: Let the center coordinates of the target rectangle outline in two adjacent frames be: Previous frame Current frame .

[0029] Pixel displacement at the center of adjacent frames for: (1) Unit vector of screen motion direction for: (2) In the formula, , These are the components in the x and y directions, respectively.

[0030] Screen scrolling speed for: (3) In the formula, The time interval between two adjacent frames of the camera. This refers to the camera's image ratio.

[0031] S102, based on the rectangular outline and the movement speed, identifies the screen position frame by frame in the image sequence, and extracts the corresponding screen image from each frame.

[0032] Specifically, this step is the precise locking of the target area for screen quality inspection on the dynamic production line. Following the output of S101, its core function is to achieve frame-by-frame tracking of the continuously moving screen 2 and the pure extraction of the effective detection area, thus eliminating background interference and reducing unnecessary computation for subsequent defect segmentation and identification. Specifically, identifying the screen position frame-by-frame in the image sequence includes: using the rectangular outline of screen 2 extracted from the previous frame as a reference, and combining the motion direction vector and movement speed of screen 2, defining the search range for the screen position in the current frame, and completing the identification and extraction of the rectangular outline of screen 2 within the search range. Its principle is based on the motion characteristics of the industrial conveyor belt 3 production line. The screen 2 to be inspected moves in a stable linear motion with the conveyor belt 3. The time interval between adjacent frames is uniquely determined by the fixed acquisition frame rate of the industrial camera. Therefore, based on the screen reference position verified in the previous frame, combined with the motion direction vector and the movement speed, the theoretical position of the screen 2 in the current frame can be accurately predicted by kinematic formula. The local search range is defined with this theoretical position as the core. There is no need to traverse the entire frame image. Contour recognition is only performed within the limited area. This design can reduce the calculation area of ​​single-frame contour recognition to less than 20% of the entire frame image, greatly shortening the processing time of a single frame image and matching the real-time detection requirements of high-speed automated production lines.

[0033] The calculation for predicting the theoretical position of screen 2 in the current frame is as follows: (4) S103. For each frame of screen image, perform multi-scale segmentation based on region growing and edge constraints to obtain multiple independent connected components as elements to be judged, and record the centroid coordinates, connected component area and average gray value of each element to be judged.

[0034] Specifically, multi-scale segmentation based on region growing and edge constraints is performed by using the gray-level continuity of pixels in the screen image as the criterion for region growing, and using the gray-level abrupt change boundaries of pixels in the screen image as the limiting condition for edge constraints, thus performing multi-scale segmentation on the screen image. The principle is that the pixel gray-level values ​​in normal areas of the screen have good continuity and small gray-level differences, while defective areas (such as scratches, dents, and dirt) often have obvious gray-level abrupt changes compared to normal areas, forming clear edge boundaries. Region growing can aggregate pixels with continuous gray levels into a single region, and edge constraints can effectively limit the range of region growing, avoiding excessive merging between different regions and ensuring that the segmented regions are consistent with the boundaries of the actual object.

[0035] Specifically, the determination of region growth is based on grayscale continuity because, as a standardized product, the normal area of ​​screen 2 has a uniform pixel grayscale distribution, and the grayscale difference between adjacent pixels usually does not exceed a preset threshold (based on the actual characteristics of industrial screen 2, the preset grayscale difference threshold is 5). Based on this characteristic, pixels with similar grayscale values ​​can be aggregated into connected regions, thereby quickly distinguishing normal regions from defective regions. The edge constraint is based on grayscale abrupt boundary because the grayscale abrupt change between the defective region and the normal region is the edge. By capturing this edge, the boundary range of region growth can be clearly defined, preventing normal regions and defective regions from being misjudged as the same connected domain. For example, the grayscale difference between the scratched area on the screen surface and the normal area can reach more than 20, forming a clear grayscale abrupt edge. Edge constraint can accurately limit the growth range of the scratched area, prevent it from merging with the surrounding normal area, and ensure that the scratched area is completely segmented.

[0036] Multi-scale segmentation of the screen image involves initial segmentation at a preset initial scale to obtain each initial connected component and its corresponding mean gray value and gray-level distribution dispersion. The initial scale setting needs to be determined based on the screen image resolution and actual detection requirements. Using a screen image captured by an industrial high-definition camera (1920×1080 pixels), a preset initial scale of 3×3 pixels is chosen. This scale balances segmentation accuracy and efficiency; a 3×3 neighborhood effectively captures the gray-level correlation between pixels while avoiding the omission of small defects (such as 0.1mm wide scratches) due to an excessively large initial scale, or the low segmentation efficiency and excessive redundant connected components due to an excessively small initial scale. After the initial segmentation, by calculating the mean gray value (the average gray value of all pixels within the component) and gray-level distribution dispersion (represented by standard deviation; a smaller standard deviation indicates a more uniform gray-level distribution) of each initial connected component, a preliminary judgment can be made as to whether the connected component is a uniform region, providing a basis for subsequent refinement or merging operations.

[0037] For each initially connected region, when the gray-level distribution dispersion exceeds the preset uniformity limit, the scale is reduced by a preset ratio, and the corresponding region of the connected region is further refined and segmented to correct the boundary position; when the gray-level distribution dispersion is equal to or lower than the preset uniformity limit, the current scale is maintained or the scale is expanded by a preset ratio, and the corresponding region of the connected region is merged to obtain multiple independent connected regions. The preset uniformity limit here is set based on the grayscale distribution characteristics of the normal area of ​​the screen. The preset standard deviation is 8. That is, when the standard deviation of the grayscale distribution of a certain initial connected region is greater than 8, it means that the grayscale distribution in this area is not uniform and may contain a mixed area of ​​defective and normal areas. At this time, the scale needs to be reduced (reduced by 50%, i.e., adjusted to a 2×2 pixel neighborhood) for secondary refinement segmentation. The principle is to reduce the segmentation neighborhood range, which can more accurately capture the grayscale differences in local areas, separate the defective part from the normal part in the mixed area, correct the deviation of the segmentation boundary, and ensure that the defective area is completely and independently segmented. When the standard deviation of the grayscale distribution is less than or equal to 8, it means that the grayscale distribution in this area is uniform and is a normal area or a single defective area. At this time, the initial scale is maintained or the scale is expanded (expanded by 50%, i.e., adjusted to a 4×4 pixel neighborhood) for region merging. The principle is to merge adjacent connected regions with uniform grayscale distribution into a whole, reduce the number of redundant connected regions, reduce the processing load of subsequent elements to be judged, and improve quality inspection efficiency.

[0038] S104. For the element to be judged in two adjacent screen images, calculate the displacement vector of the centroid coordinates in the screen image coordinate system, and compare the displacement vector with the inter-frame reference displacement vector determined by the screen's motion direction vector and movement speed. If the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, then the corresponding element to be judged is marked as a candidate for following defect; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point.

[0039] Specifically, the screen image coordinate system is established with the top left corner of a single frame screen image as the origin, the horizontal direction to the right as the positive x-axis, and the vertical direction downward as the positive y-axis, ensuring that the coordinate reference of adjacent frames is completely consistent. First, feature matching is used to pair the same element to be judged in adjacent frames through connected component area deviation ≤10% and average gray value deviation ≤8. Then, the inter-frame difference of the centroid coordinates of the paired elements is calculated to obtain the actual displacement vector, avoiding judgment errors caused by misalignment of different elements. The inter-frame reference displacement vector is calculated based on the screen motion direction vector and movement speed extracted by S101, combined with the time interval between adjacent frames corresponding to the fixed frame rate of the industrial camera. It represents the standard displacement feature of the actual attachment defect on screen 2. This design reuses the motion parameters extracted in the previous step, without the need for additional hardware, simplifying the algorithm link while ensuring the real-time performance of displacement calculation. If the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, the corresponding element to be judged is marked as a candidate for following defects; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflective point. The principle is that, if Figure 3 As shown, the scratches are actually attached to the screen surface. Figure 3 b) Concave point ( Figure 3 e), dirt ( Figure 3 Defects such as a) in the above are rigidly bound to screen 2, and their inter-frame displacement is completely synchronized with the reference displacement of screen 2 itself; while the lighting in the production line environment (such as...) Figure 3 c) Equipment structural components (such as Figure 3 In section d), the reflection artifacts and reflective points formed on the screen surface are spatially fixed in the production line environment. They only change in reflection angle as the screen 2 moves. In continuous frame screen images, they will produce a significant relative displacement with the screen 2 body, which can accurately distinguish between real defects and environmental artifacts.

[0040] The comparison calculation process is as follows: Let the first The set of elements to be judged in the frame is , ;No. The set of elements to be judged in frame +1 is , The pairing conditions are: (5) In the formula, For the first Frame to be judged element The area of ​​the connected region; For the first +1 frame of elements to be judged The area of ​​the connected region; For the first Frame to be judged element The average gray value; For the first +1 frame of elements to be judged The average gray value. Element pairs that meet the condition are marked as paired elements. .

[0041] Calculate the centroid displacement vector of the paired elements between adjacent frames. : (6) In the formula, For the first Frame to be judged element The coordinates of the centroid; For the first +1 frame of elements to be judged The coordinates of the centroid.

[0042] Inter-frame reference displacement vector for: (7) Deviation between the displacement vector and the inter-frame reference displacement vector for: (8) S105, for all following defect candidates, use the motion direction vector and movement speed of screen 2 to perform inter-frame motion compensation, and align and superimpose the connected components belonging to the same following defect candidate in multiple consecutive frames to generate a defect enhancement image.

[0043] Specifically, real defects such as scratches, dents, and dirt attached to the screen surface are rigidly bound to the screen body. Their motion trajectory in continuous frame images is completely synchronized with the screen motion direction vector and movement speed extracted in S101. Meanwhile, the reflection artifacts and reflective points formed on the screen surface by fixed light sources and frame structural components in the production line environment have their reflection source spatial positions fixed in the production line. They only change continuously with the movement of the screen 2 driven by the conveyor belt 3. They will have a continuous relative displacement with the screen body in the screen image coordinate system. Even if there are a few residual artifacts that are missed in S104, they will be completely removed because the inter-frame displacement does not match the screen motion parameters and cannot form an effective feature superposition in the subsequent alignment operation. This strengthens the real defect features and solves the problem of false detection of artifacts caused by ambient light interference from the root.

[0044] Inter-frame motion compensation is achieved by utilizing the motion direction vector and movement rate of screen 2. Specifically, this is accomplished through the following steps: First, based on the rectangular outline of screen 2 in the previous frame image, the initial position of the rectangular outline of the screen in the current frame image is predicted according to the motion direction vector and movement rate. The principle is that the industrial conveyor belt 3 production line is a stable, uniform linear motion, and the time interval between adjacent frames is uniquely determined by the fixed acquisition frame rate of the industrial camera. For example, when using a high-definition industrial camera at 30fps, the time interval between adjacent frames is 33.3ms. Combining the screen motion direction vector and movement rate extracted by S101, the coordinate offset of the rectangular outline of the screen in the current frame can be calculated by multiplying the speed by the time. This eliminates the need to traverse the entire frame image, significantly reducing the amount of single-frame positioning calculation while ensuring real-time positioning and matching the detection cycle of the high-speed production line. Then, using the initial position as the center, a preset pixel range is expanded outward as a search window. Within the search window, the gradient magnitude of each pixel is calculated, and a non-maximum suppression algorithm is used to retain local maxima of the gradient magnitude as edge candidate points. The preset pixel range is set to 50 pixels in combination with the motion jitter tolerance of conveyor belt 3, which can accommodate positional deviations caused by the start and stop of conveyor belt 3 and slight jitter, and avoid the loss of screen outline. The principle is that the edge of the rectangular outline of the screen is a gray-scale abrupt change area, and the corresponding pixel gradient magnitude is much higher than that of the uniform area inside screen 2 and the background area. By calculating the gradient magnitude, edge pixels can be quickly located, and non-maximum suppression can eliminate redundant pixels and retain core edge candidate points. While eliminating random noise, it reduces the amount of computation for subsequent line detection and improves the accuracy and efficiency of outline localization. Next, Hough transform line detection is performed on the edge candidate points to filter out four lines that satisfy the rectangular geometric constraints. The intersection of the four lines is taken as the vertex of the screen rectangle outline in the current frame image, thus obtaining the accurate outline of screen 2 in the current frame image. Here, the rectangular geometric constraints include parallel opposite sides, perpendicular adjacent sides, and an aspect ratio that conforms to the standard 16:9 ratio of the screen to be detected. This can effectively eliminate background interference lines. The principle is that the Hough transform can convert the lines in the image space into peak points in the parameter space, which has a very strong anti-interference ability. Even if there is local reflection or occlusion at the edge of screen 2, the complete line boundary can be detected stably. Combined with the secondary filtering of rectangular geometric constraints, background interference can be completely eliminated, and the accurate outline of screen 2 at the sub-pixel level can be obtained, providing a precise coordinate reference for subsequent affine transformation alignment.Finally, based on the precise contours, connected components belonging to the same defect candidate in multiple consecutive frames are aligned to the same coordinate system through affine transformation. The aligned connected components are then weighted and superimposed using pixel grayscale values ​​to generate a defect-enhanced image. For example, for the same minor scratch defect candidate in five consecutive frames, based on the precise contours of screen 2 in each frame, the scratch connected components in the five frames are uniformly aligned to the coordinate system of the first frame screen image through affine transformation. The pixel grayscale values ​​at the same coordinate position after alignment are then weighted and superimposed using a progressive weighting method. The principle is that the real defect appears in the same coordinate system after alignment. One coordinate position has stable grayscale characteristics. After superposition, the grayscale difference between the defect area and the normal area is continuously amplified, and the feature signal-to-noise ratio can be improved by more than 3 times. The residual reflection artifacts are not synchronized with the movement of screen 2, and the pixel positions cannot be coincident after alignment. After superposition, the grayscale value is weakened by average and cannot form effective features, so they are completely eliminated. This step not only enhances the identifiability of subtle defects and greatly reduces the false detection rate and false negative rate of subsequent defect classification, but also completes the final removal of residual reflection artifacts, ensuring the detection accuracy and environmental anti-interference ability of the entire quality inspection method.

[0045] S106, input the enhanced image of defects into the trained defect classifier. The defect classifier outputs a defect category judgment result that includes at least scratches, dents and dirt based on the area of ​​connected components, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

[0046] Specifically, the selection of connected component area, average gray value, and edge morphology features extracted based on gradient orientation histogram as input features perfectly matches the essential distinguishing characteristics of the core defects on Screen 2. The connected component area parameter can directly distinguish defects of different scales. For example, the area of ​​dot-like dents on the surface of Screen 2 is usually concentrated in the range of 10-50 pixels, while the area of ​​long strip-shaped scratches is mostly over 100 pixels. The area of ​​dirty areas fluctuates more widely, providing a basic scale judgment basis for defect classification. The average gray value parameter can distinguish the gray value difference between defects and the background. Dents often present low gray value dark spots due to light scattering, while dirt often has low gray value. In the dark areas of normal regions, some metal scratches will appear as high-grayscale bright lines due to light reflection, enabling rapid differentiation of defects with different optical properties. Based on the edge morphology features extracted from the gradient direction histogram, the contour and texture characteristics of defects can be accurately captured. Scratches have continuous linear gradient direction features, concave points have isotropic gradient features with a near-circular shape, and dirt exhibits irregular discrete gradient distribution features, providing core morphological basis for accurate determination of defect categories. The combination of these three features can cover the full-dimensional differentiation boundaries of the three core defect categories, avoiding misclassification caused by a single feature and ensuring the stability and accuracy of the classification results.

[0047] The trained defect classifier is constructed through the following steps: First, screen defect sample images containing scratches, dents, and dirt are pre-collected, and each screen defect sample image is labeled with its corresponding defect category label. The sample collection process uses a high-definition industrial camera of the same model as the production line, and the collection is completed under the same lighting, focal length, and imaging ratio conditions to ensure that the imaging characteristics of the training samples are completely consistent with those of the actual inspection images, avoiding insufficient model generalization ability due to differences in imaging environment. The labeling process is performed by professionals with screen quality inspection qualifications, using polygonal bounding boxes to accurately select defect areas and match corresponding category labels, providing an accurate ground truth benchmark for supervised training. In addition, it includes data augmentation processing on all screen defect sample images. Augmentation processing includes rotating the screen defect sample images, mirroring them, and adjusting their brightness to expand the number of samples and the scope of scene coverage. The principle is that the angle, location, and ambient brightness of defects in industrial production lines fluctuate randomly. Step-by-step angle rotation within the range of 0-360° can cover all possible paths of linear defects such as scratches. Mirroring can expand the scene of the full location distribution of defects. Brightness adjustment within the range of ±20% grayscale value can simulate the actual working conditions of production line illumination fluctuations and screen reflection differences. This processing can expand the original sample size by more than 8 times, effectively solving the industry pain point of scarce industrial defect samples, while significantly improving the classifier's generalization and adaptation capabilities to complex production line working conditions and avoiding model overfitting. Secondly, for each screen defect sample image, the connected component area parameter, the average gray value parameter, and the edge morphology feature parameter extracted based on the gradient orientation histogram are extracted. The connected component area parameter, the average gray value parameter, and the edge morphology feature parameter are combined to form the feature vector of the screen defect sample image. In specific execution, the connected component area parameter is the total number of pixels in the defect connected component, the average gray value is the arithmetic mean of the gray values ​​of all pixels in the defect connected component, and the gradient orientation histogram divides the defect region into 8×8 cell units. The pixel amplitude accumulation value in 9 gradient directions within the 0-180° range is counted in each cell. Finally, they are spliced ​​to form a 128-dimensional morphological feature vector, which is then spliced ​​with the aforementioned 2-dimensional numerical features to form a 130-dimensional comprehensive feature vector. This vector contains both the global statistical characteristics of the defect and the local edge morphological details, which can completely represent the core distinguishing features of different types of defects and provide stable feature input for the classifier.

[0048] Finally, the feature vectors and defect category labels corresponding to all screen defect sample images are input into a support vector machine for supervised training. The hyperparameters of the support vector machine are adjusted until the classification prediction accuracy of the defect category meets the preset convergence condition, thus obtaining the trained defect classifier. Support vector machines are chosen here because they possess excellent classification performance and generalization ability in scenarios with small samples and high-dimensional features, making them suitable for scenarios with a limited number of industrial defect samples and high feature dimensions. During training, a radial basis function kernel is used to map low-dimensional features to a high-dimensional space, achieving linear classification of nonlinearly separable defect features. Simultaneously, the penalty coefficient and kernel function bandwidth hyperparameters are optimized using a grid search method. The preset convergence condition is that the classifier's overall classification accuracy on the test set is ≥99.5%, the single-class defect false negative rate is ≤0.2%, and the false positive rate is ≤0.3%, meeting the high-precision quality inspection requirements of industrial production lines.

[0049] After outputting the defect category determination result, the defect category determination result is associated with and stored in conjunction with the corresponding screen image and centroid coordinate information, and the defect category determination result is simultaneously pushed to a preset terminal on the production line for subsequent review of historical defect data.

[0050] like Figure 4 As shown, based on the same inventive concept, this embodiment provides an automated product quality inspection system using artificial intelligence vision, including: The data acquisition module 201 is used to acquire an image sequence containing screen movement through a camera 1 mounted above the production line, and to extract the rectangular outline of the screen, the motion direction vector of the screen, and the movement speed of the screen. The screen image extraction module 202 is used to identify the screen position frame by frame in the image sequence based on the rectangular outline and the movement speed, and extract the corresponding screen image from each frame image. The segmentation module 203 is used to perform multi-scale segmentation based on region growing and edge constraints on each frame of screen image, obtain multiple independent connected components as elements to be judged, and record the centroid coordinates, connected component area and average gray value of each element to be judged. The displacement comparison module 204 is used to calculate the displacement vector of the centroid coordinates in the screen image coordinate system for the elements to be judged in two adjacent frames of screen images, and compare the displacement vector with the inter-frame reference displacement vector determined by the motion direction vector and the movement speed of the screen; if the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, the corresponding element to be judged is marked as a candidate for following defect; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point. The compensation module 205 is used to perform inter-frame motion compensation on all following defect candidates using the motion direction vector and movement speed of the screen, and to align and superimpose connected components belonging to the same following defect candidate in multiple consecutive frames to generate a defect enhancement image. The defect recognition module 206 is used to input the enhanced defect image into the trained defect classifier. The defect classifier outputs a defect category judgment result that includes at least scratches, dents and dirt based on the area of ​​connected components, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

[0051] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0052] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automated product quality inspection method based on artificial intelligence vision, characterized in that, include: By acquiring image sequences containing screen movement through cameras mounted above the production line, the rectangular outline of the screen, the screen's motion direction vector, and the screen's movement speed are extracted. Based on the rectangular outline and the moving speed, the screen position is identified frame by frame in the image sequence, and the corresponding screen image is extracted from each frame. For each frame of the screen image, perform multi-scale segmentation based on region growing and edge constraints to obtain multiple independent connected components as elements to be judged, and record the centroid coordinates, connected component area and average gray value of each element to be judged. Specifically, the multi-scale segmentation based on region growing and edge constraints includes: Using the grayscale continuity of pixels in the screen image as the criterion for region growing, and the grayscale abrupt change boundaries of pixels in the screen image as the limiting condition for edge constraints, the screen image is segmented at multiple scales; wherein, the multi-scale segmentation of the screen image specifically includes: The screen image is segmented at a preset initial scale to obtain each initial connected component and its corresponding gray mean and gray distribution dispersion. For each initially connected region, when the gray-level distribution dispersion exceeds a preset uniformity limit, the scale is reduced by a preset ratio, and the corresponding region of the connected region is further refined and segmented to correct the boundary position; when the gray-level distribution dispersion is equal to or lower than the preset uniformity limit, the current scale is maintained or the scale is expanded by a preset ratio, and the corresponding region of the connected region is merged to obtain multiple independent connected regions. For the element to be judged in two adjacent frames of the screen image, calculate the displacement vector of the centroid coordinates of the element to be judged in the screen image coordinate system, and compare the displacement vector with the inter-frame reference displacement vector determined by the motion direction vector of the screen and the movement speed; if the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, then the corresponding element to be judged is marked as a candidate for following defect; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point. For all the following defect candidates, inter-frame motion compensation is performed using the motion direction vector of the screen and the movement speed. Connected components belonging to the same following defect candidate in multiple consecutive frames are aligned and superimposed to generate a defect enhancement image. The enhanced image of the defect is input into a trained defect classifier, which outputs a defect category determination result that includes at least scratches, dents and dirt based on the area of ​​the connected region, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

2. The automated product quality inspection method based on artificial intelligence vision according to claim 1, characterized in that, Extracting the rectangular outline of the screen specifically includes: The image sequence is converted to grayscale, and the grayscale image is then filtered and smoothed to eliminate random noise generated during image acquisition while preserving the edge boundary information of objects in the image. Perform edge enhancement processing on the smoothed image to amplify the contrast of areas with abrupt grayscale changes and highlight the grayscale difference between the screen edges and the surrounding background. Edge detection is performed on the image after edge enhancement to extract all closed edge contours in the image and form an initial contour set; Perform geometric feature filtering on all closed edge contours within the initial contour set, retaining contours that conform to the geometric properties of rectangles to form a candidate rectangular contour set; By combining the imaging ratio corresponding to the inherent physical size of the screen to be inspected on the production line, and the preset layout area of ​​the screen in the production line, a secondary screening is performed on the candidate rectangular contour set to remove invalid contours that do not match the inherent attributes and layout position of the screen, and the target rectangular contour is obtained.

3. The automated product quality inspection method based on artificial intelligence vision according to claim 1, characterized in that, The step of identifying screen positions frame by frame in the image sequence specifically includes: Using the rectangular outline of the screen extracted from the previous frame image in the image sequence as a reference, and combining the screen's motion direction vector and the movement speed, the search range of the screen position in the current frame image is defined, and the rectangular outline of the screen is identified and extracted within the search range.

4. The automated product quality inspection method based on artificial intelligence vision according to claim 1, characterized in that, The inter-frame motion compensation using the screen's motion direction vector and the movement rate specifically includes: Based on the rectangular outline of the screen in the previous frame image, the initial position of the rectangular outline of the screen in the current frame image is predicted according to the motion direction vector and the movement speed. Using the initial position as the center, a preset pixel range is expanded outward as a search window. The gradient magnitude of each pixel is calculated within the search window, and a non-maximum suppression algorithm is used to retain the local maxima of the gradient magnitude as edge candidate points. Hough transform line detection is performed on the candidate edge points to filter out four lines that satisfy the rectangular geometric constraints. The intersection of the four lines is taken as the vertex of the screen rectangle outline in the current frame image, thereby obtaining the accurate outline of the screen in the current frame image. Based on the precise contour, connected components belonging to the same defect candidate in multiple consecutive frames are aligned to the same coordinate system through affine transformation, and the aligned connected components are weighted and superimposed with pixel gray values ​​to generate a defect enhancement image.

5. The automated product quality inspection method based on artificial intelligence vision according to claim 1, characterized in that, The trained defect classifier is constructed through the following steps: Screen defect sample images, including scratch type, dent type and dirt type, are pre-collected, and each screen defect sample image is labeled with its corresponding defect category label; For each screen defect sample image, the connected component area parameter, the average gray value parameter, and the edge morphology feature parameter extracted based on the gradient orientation histogram are extracted, and the connected component area parameter, the average gray value parameter, and the edge morphology feature parameter are combined to form the feature vector of the screen defect sample image. The feature vectors and defect category labels corresponding to all the screen defect sample images are input into a support vector machine for supervised training. The hyperparameters of the support vector machine are adjusted until the classification prediction accuracy of the defect category meets the preset convergence condition, thus obtaining the trained defect classifier.

6. The automated product quality inspection method based on artificial intelligence vision according to claim 5, characterized in that, The step of pre-acquiring screen defect sample images including scratch type, dent type, and dirt type also includes: Data augmentation processing is performed on all the screen defect sample images. The augmentation processing includes rotating the screen defect sample images, mirroring them, and adjusting their brightness, thereby increasing the number of samples and the scene coverage.

7. The automated product quality inspection method based on artificial intelligence vision according to claim 1, characterized in that, The output, after including at least the defect category determination results for scratches, dents, and dirt, also includes: The defect category determination result is associated with and stored in conjunction with the corresponding screen image and centroid coordinate information, and the defect category determination result is pushed to the preset terminal of the production line simultaneously.

8. An automated product quality inspection system based on artificial intelligence vision, wherein the automated product quality inspection method based on artificial intelligence vision as described in claim 1 is characterized in that, include: The data acquisition module is used to acquire image sequences containing screen movement through a camera mounted above the production line, and to extract the rectangular outline of the screen, the screen's motion direction vector, and the screen's movement speed. The screen image extraction module is used to identify the screen position frame by frame in the image sequence based on the rectangular outline and the movement speed, and extract the corresponding screen image from each frame image. The segmentation module is used to perform multi-scale segmentation based on region growing and edge constraints on each frame of the screen image to obtain multiple independent connected components as elements to be judged, and to record the centroid coordinates, connected component area and average gray value of each element to be judged. Specifically, the multi-scale segmentation based on region growing and edge constraints includes: Using the grayscale continuity of pixels in the screen image as the criterion for region growing, and the grayscale abrupt change boundaries of pixels in the screen image as the limiting condition for edge constraints, the screen image is segmented at multiple scales; wherein, the multi-scale segmentation of the screen image specifically includes: The screen image is segmented at a preset initial scale to obtain each initial connected component and its corresponding gray mean and gray distribution dispersion. For each initially connected component, when the grayscale distribution dispersion exceeds a preset uniformity limit, the scale is reduced by a preset ratio, and the corresponding region of the connected component is further refined and segmented to correct the boundary position; when the grayscale distribution dispersion is equal to or lower than the preset uniformity limit, the current scale is maintained or the scale is expanded by a preset ratio, and the corresponding region of the connected component is merged to obtain multiple independent connected components; the displacement comparison module is used to calculate the displacement vector of the centroid coordinates in the screen image coordinate system for the elements to be judged in two adjacent frames of the screen image, and compare the displacement vector with the inter-frame reference displacement vector determined by the motion direction vector of the screen and the movement speed; if the deviation between the displacement vector and the inter-frame reference displacement vector is less than or equal to a preset first threshold, the corresponding element to be judged is marked as a follower defect candidate; otherwise, the corresponding element to be judged is marked as a reflection artifact or a reflection point; The compensation module is used to perform inter-frame motion compensation on all the following defect candidates using the motion direction vector of the screen and the movement speed, and to align and superimpose connected components belonging to the same following defect candidate in multiple consecutive frames to generate a defect enhancement image. The defect identification module is used to input the enhanced defect image into a trained defect classifier. The defect classifier outputs a defect category determination result that includes at least scratches, dents and dirt based on the area of ​​the connected region, the average gray value and the edge morphology features extracted based on the gradient orientation histogram.

Images