Assembly line product defect detection method and system based on machine vision

Abstract

The invention discloses an assembly line product defect detection method and system based on machine vision, and relates to the technical field of industrial automation and machine vision detection, and the method comprises the following steps: aiming at each to-be-detected product model, in an offline simulation environment, based on a three-dimensional CAD model of each model of product, calculating a three-dimensional CAD model of each model of product; configuring a virtual camera and a virtual light source model and performing optical rendering to generate a standard product image; defining layout and reference parameters of a measuring tool on a standard product image, recording model parameters of the virtual camera and the virtual light source, and jointly packaging and storing the model parameters as a digital twinning detection template; the shutdown time of a production line is greatly shortened through a series of automatic operations, the overall equipment utilization rate is remarkably improved, the problem that the traditional system is low in remodeling efficiency is effectively solved, complex hardware adjustment and parameter configuration do not need to be manually conducted in the remodeling process, and the cost is reduced. And the high consistency of detection standards after model changing of different batches and different operators is ensured.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and machine vision inspection technology, specifically to a method and system for defect detection of production line products based on machine vision. Background Technology

[0002] In modern industrial manufacturing, automated inspection based on machine vision has become a key means to ensure product quality on the production line. Specifically, it involves using industrial cameras to capture product images and then using computer algorithms to analyze them, replacing manual inspection of items such as size measurement, identification of appearance defects, and assembly integrity. It has the advantages of high efficiency, good consistency, and high accuracy.

[0003] However, in actual assembly line production, as market demand evolves towards a flexible manufacturing model of small batches and multiple varieties, production lines need to frequently change product models. For each product model change, technicians need to readjust the physical poses of cameras, lenses, and light sources on-site and manually modify the area settings and threshold parameters in the detection program. The whole process is time-consuming and labor-intensive, and heavily relies on personnel experience, which can easily lead to inconsistent settings or even errors. This will cause the production line to be down for longer periods of time, or cause inconsistent or misjudged detection results. The detection system cannot be synchronized with the dynamic production cycle, which greatly restricts the overall flexibility and changeover efficiency of the production line.

[0004] Traditional visual inspection systems face the following challenges when changing production line product models: On the one hand, the hardware configuration and software parameters of existing visual inspection systems are usually fixed for a single product model. These include camera pose, light source parameters, measurement tool layout, and judgment thresholds, all of which are deeply bound to the corresponding single product model. This results in a lack of dynamic decoupling mechanism between the inspection system and the production cycle. The model change relies on offline and static configuration methods, which makes the inspection system unable to respond quickly to production changes and creates a flexibility bottleneck. On the other hand, each time a model is changed, experienced engineers must manually readjust the hardware and configure the software parameters on-site. This process is not only time-consuming and causes the production line to be shut down for a long time, but also relies heavily on human experience, resulting in low changeover efficiency and reliance on manual labor, which drags down the overall production efficiency and flexibility of the industrial production line.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a machine vision-based method and system for detecting defects in production line products. This invention achieves a fully automated online defect detection process for product changeover by establishing a digital twin detection template, combining "virtual-real mapping" to automatically match images, calculating deviations for compensation and conversion, and controlling hardware to adjust the camera state and pose in the physical environment, thereby solving the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based method for detecting defects in production line products, comprising the following steps: S1. For each product model to be inspected, in an offline simulation environment, based on the 3D CAD model of each product model, configure the virtual camera and virtual light source model and perform optical rendering to generate a standard product image; define the layout and reference parameters of the measuring tools on the standard product image, and record the model parameters of the virtual camera and virtual light source, and package and store them together as a digital twin inspection template. S2. When receiving a target product model switching instruction on the production line, automatically call the digital twin detection template matching the target model, directly parse the virtual camera and virtual light source model parameters, drive the programmable motion mechanism carrying the industrial camera and the physical light source controller, and automatically adjust the industrial camera and physical light source to the preset working state and posture consistent with the virtual environment. S3. After the hardware driver adjustment is completed, take a picture of the target model product currently being transmitted on the production line and acquire at least one actual sample image. S4. Using an image template matching algorithm, the actual sample image is registered with the standard product image in the digital twin detection template; based on the registration result, the affine transformation matrix from the coordinate system of the standard product image to the coordinate system of the actual sample image is calculated, and the average gray level difference between the two images in the overlapping area after registration is calculated. S5. Use an affine transformation matrix to perform geometric transformation on all coordinate points of the measurement tool layout defined in the digital twin inspection template, use the average gray level difference to compensate for the offset of the gray level threshold in the reference parameters, use the transformed and compensated parameters to generate an online inspection program and immediately execute online defect detection of the product.

[0008] Optionally, the configuration steps for the virtual camera and virtual light source model are as follows: The system calibrates and measures real vision hardware cameras to obtain the precise pose of physical industrial cameras in the production line coordinate system, as well as intrinsic parameters such as lens focal length and pixel size. Simultaneously measure the position, illumination angle, luminous intensity, and spectral characteristics of the physical light source LED array relative to the industrial camera and the product; A physically based ray tracing renderer is used in an offline simulation environment to create a virtual scene that corresponds to the real pipeline environment; In a virtual scene corresponding to a real production line environment, a virtual camera model is established based on the internal parameters of a physical industrial camera, and the viewpoint and angle of the virtual camera are set in combination with the precise pose of the industrial camera in the production line coordinate system. For virtual light sources, a corresponding virtual light source model is created based on the geometric position, orientation angle, light intensity distribution, and spectral data of the physical light source LED array group to ensure that the light characteristics emitted by the virtual light source model in the virtual scene are physically equivalent to those in reality.

[0009] Optionally, the steps for generating the standard product image are as follows: Based on the configured virtual camera and virtual light source model, input the product's 3D CAD model and perform optical rendering in an offline virtual simulation environment; This includes, for each pixel on the virtual camera sensor, emitting a ray of light from the camera's optical center that passes through that pixel and enters the virtual scene; After the light rays intersect with the surface of the 3D CAD model, the amount of light radiation entering the camera along the direction of the light rays is calculated based on the material properties at the intersection point and the virtual light source information of all direct lighting and indirect reflected light in the virtual scene, combined with the surface reflectivity and roughness of the 3D CAD model. The process is performed in parallel on all pixels of the image to calculate the radiance values ​​of all light rays passing through all pixels. After tone mapping, a standard product image with accurate geometric contours and realistic lighting and shadow effects is generated.

[0010] Optionally, the steps for defining the layout of the measuring tool and the reference parameters are as follows: The layout of the measurement tool includes an edge detection area, a blob analysis area, and the corresponding feature search direction; For edge detection areas, a measurement tool is defined on the generated standard product image using a script based on the detection requirements. This includes defining a rectangular region ROI at the location where the size or contour needs to be measured at the product boundary and specifying the search direction perpendicular to the edge. The corresponding benchmark parameters include: the ideal sub-pixel edge position extracted on the standard product image by using the Canny operator combined with gray-scale gradient interpolation sub-pixel edge detection algorithm, and the edge theoretical design size value based on the ideal sub-pixel edge position; For spot analysis areas, one or more rectangular or arbitrary shaped analysis areas are defined in the areas where stains, impurities, or specific textures need to be detected. The corresponding benchmark parameters include: the average gray value of qualified products within the analysis area, the statistical gray characteristics of the gray standard deviation, and the allowable spot area threshold and spot contrast threshold.

[0011] Optionally, the preset working state and pose are to make the physical imaging environment of the production line product reproduce the virtual environment when the digital twin inspection template is generated. This requires adjusting the industrial camera and physical light source to a quantified target state that is uniquely determined by all parameters in the digital twin inspection template. First, from the target model digital twin detection template, the intrinsic and extrinsic parameters of the virtual camera and virtual light source models stored in the file package are parsed. For the virtual camera model, the extrinsic parameters include the rotation matrix, translation vector, lens focal length, and pixel size obtained from the physical camera calibration measurement. For the virtual light source model, the luminance values ​​of the target position, illumination angle, and light intensity distribution are included. Based on the target pose of the virtual camera, the target angle or position to be reached by each joint on the programmable motion mechanism carrying the industrial camera is calculated by inverse kinematics, so as to drive the programmable motion mechanism to execute preset working pose instructions. At the same time, the parameters of the virtual light source are converted into voltage, PWM or linear digital instructions that can be executed by the physical light source controller.

[0012] Optionally, the registration steps between the actual sample image and the standard product image are as follows: Gaussian filtering and noise reduction preprocessing are performed on the actual sample images and the standard product images retrieved from the digital twin detection template; An image feature point matching algorithm is used to extract a set of highly discriminative key points and descriptors from two images. The descriptor is a high-dimensional vector that encodes the local appearance features of the key point neighborhood. Based on the similarity between keypoints and descriptors calculated using Euclidean distance, the most similar candidate matching point is found in the standard product image for each keypoint in the actual sample image, forming a preliminary set of matching pairs. Robust estimation is performed using the Random Sampling Consensus (RANSAC) algorithm. This involves randomly selecting a minimum sample set of at least three pairs from the set of matching pairs, calculating a candidate affine transformation matrix with a projection error less than a preset threshold, and verifying the number of interior points of the matching pairs that conform to the affine transformation matrix. The RANSAC algorithm process is repeated multiple times until the matching pairs that match the interior points of the corresponding candidate affine transformation matrix with the most interior points are selected. At the same time, a preliminary affine transformation model is obtained, and all interior point matching pairs are selected. Based on all the selected interior point matching pairs, the optimal affine transformation matrix is ​​recalculated using the least squares method on the initial affine transformation model to minimize the sum of the projection errors of all interior points, thus obtaining a high-precision and stable registration result.

[0013] Optionally, the steps for calculating the deviation between the affine transformation matrix and the average grayscale difference are as follows: Based on the registration results, all interior point matching pairs are statistically analyzed and used. An affine transformation matrix is ​​obtained by using the least squares method to solve for the virtual-real mapping relationship between the "standard product image coordinate system" and the "actual sample image coordinate system", which serves as the geometric deviation for mechanical positioning. Using the obtained affine transformation matrix, the standard product image is resampled using bilinear interpolation to generate a geometrically aligned standard image that is perfectly aligned with the actual sample image. Within the effective overlap area between the actual sample image and the aligned standard image, the grayscale difference is calculated by traversing the pixels. The arithmetic mean of all pixel grayscale differences is then used as the global brightness deviation.

[0014] Optionally, the geometric transformation and offset compensation processing steps are as follows: Geometric Transformation: Traverse the layout and reference parameters of all measurement tools defined in the digital twin inspection template, including edge detection lines, contour detection regions, and blob analysis region boxes, and extract the geometric definition points of each tool, including the start and end points of the lines, the vertex coordinates of the regions, and the center and boundary points of the boxes; By applying an affine transformation matrix to transform the coordinates of each geometrically defined point, new coordinates are obtained in the actual sample image coordinate system. Compensation offset: Traverse all the reference parameters related to absolute grayscale in the digital twin detection template, including the binarization threshold, the upper and lower limits of the grayscale range for detecting spot area, the spot contrast threshold, and the reference grayscale value for spot contrast judgment. The average grayscale difference is applied to offset the baseline parameters as a whole, generating a correction threshold suitable for the current lighting conditions to compensate for the overall brightness difference between the actual lighting conditions and the virtual standard lighting conditions.

[0015] A machine vision-based production line product defect detection system includes a digital twin inspection template management module: used to create, store, manage, and call digital twin inspection templates; Hardware-driven control module: used to analyze parameters based on the digital twin detection template matching the product model, control the programmable motion mechanism and physical light source controller, and complete the adaptive adjustment of the physical hardware; Image acquisition module: Used to control an industrial camera to photograph the actual product of the target model and obtain images of the actual sample; Visual processing and analysis engine: Embedded image registration algorithm, used to perform image registration, feature comparison, virtual-real mapping and deviation calculation, as well as generate and run online detection programs; Central control module: Used to coordinate various modules, respond to product changeover instructions on the industrial production line, and control the switching and execution of the product inspection process on the production line.

[0016] Optionally, the programmable motion mechanism is a multi-degree-of-freedom motion platform that carries and adjusts the pose of an industrial camera; The physical light source controller is an LED light source assembly whose brightness and angle can be independently programmed and adjusted; The hardware-driven control module communicates with the programmable motion mechanism and the physical light source controller via the EtherCAT bus protocol.

[0017] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention creates digital twin inspection templates for different product models in an offline simulation environment. When a target product model switching instruction is received on the production line, the matching digital twin inspection template is automatically invoked, and the programmable motion mechanism and physical light source controller are automatically controlled according to the template parameters to complete the adaptive adjustment of the physical hardware. This series of automated operations greatly reduces production line downtime, significantly improves the overall equipment utilization rate, and effectively solves the problem of low efficiency in traditional system model changeover. Moreover, during the model changeover process, there is no need for manual complex hardware adjustments and parameter configurations, avoiding experience differences and random errors caused by manual operation, completely reducing the dependence on manpower, and ensuring a high degree of consistency in inspection standards after model changeover for different batches and different operators. When switching product models on the production line, the system can quickly call up the matching digital twin inspection template, automatically adjust the working status and posture of the industrial camera and physical light source, and seamlessly adapt to the inspection needs of different product models. This enables the system to respond quickly to frequent changes in production plans, significantly enhances production flexibility, and better adapts to the modern production mode of "small batch and multiple varieties". By employing an image template matching algorithm, the actual sample image is registered with the standard product image in the digital twin inspection template. The affine transformation matrix and the average grayscale difference are calculated. The affine transformation matrix is ​​used to perform a geometric transformation on the coordinate points of the measurement tool layout defined in the digital twin inspection template. The average grayscale difference is used to compensate for the offset of the grayscale threshold in the reference parameters. The transformation of "virtual and real mapping" and the compensated parameters are used to generate an online inspection program and execute online defect detection of the product. This not only improves the inspection accuracy after model change, but also ensures the reliability of quality control, further improving the overall inspection level and production efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1This is a flowchart of the machine vision-based defect detection method for production line products according to the present invention.

[0020] Figure 2 This is a schematic diagram illustrating the principle of the virtual-real mapping and deviation calculation process of the present invention.

[0021] Figure 3 This is a schematic diagram of the digital twin template construction process of the present invention.

[0022] Figure 4 This is a block diagram of the machine vision-based production line product defect detection system of the present invention. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0024] Example 1 This invention provides, for example Figure 1-3 The machine vision-based defect detection method for production line products shown includes the following steps: S1. For each product model to be inspected, in an offline simulation environment, based on the 3D CAD model of each product model, configure a virtual camera and virtual light source model that are consistent with the optical properties of the real hardware, perform optical rendering on the 3D CAD model, and generate a standard product image with accurate geometric and lighting information; define and set the layout and reference parameters of all measuring tools used for defect detection according to the standard product image, and record the model parameters of the virtual camera and virtual light source, and package and store them together as a unique digital twin inspection template for each product model. In one embodiment, the configuration steps for the virtual camera and virtual light source model are as follows: The system calibrates and measures real vision hardware cameras to obtain the precise pose of physical industrial cameras in the pipeline coordinate system, as well as intrinsic parameters such as lens focal length, pixel size, and distortion coefficient. Simultaneously measure the position, illumination angle, luminous intensity, and spectral characteristics of the physical light source LED array relative to the industrial camera and the product; A physically based ray tracing renderer is used in an offline simulation environment to create a virtual scene that corresponds to the real pipeline environment; In a virtual scene corresponding to a real production line environment, a virtual camera model is established based on the internal parameters of a physical industrial camera. The viewpoint and angle of the virtual camera are then set using the precise pose of the industrial camera in the production line coordinate system. The expression for the virtual camera model is as follows: ,and ,as well as In the formula, Represented as a non-zero scaling factor, Represented as two-dimensional pixel coordinates projected onto a standard product image from world coordinates. Represented as an internal parameter matrix, This is represented as the extrinsic rotation matrix obtained from the physical camera calibration. This is represented as the extrinsic translation vector obtained from the physical camera calibration, used to transform world coordinates to camera coordinates. Represented as world coordinates of a point in three-dimensional space. , Expressed as focal length in pixels. This is expressed as the physical focal length of the lens. , Represented as pixel size, Represented as the pixel coordinates of the image's optical center; For virtual light sources, a corresponding virtual light source model is created based on the geometric position, orientation angle, light intensity distribution, and spectral data of the physical light source LED array. This ensures that the light characteristics emitted by the virtual light source model in the virtual scene are physically equivalent to those in reality. The expression for the virtual light source model is: In the formula, Represented as the first irradiation on the surface of the 3D CAD model Radiance at point This is represented by the light intensity along the normal direction to the center of the virtual light source. It is expressed as an exponent describing the directional attenuation of a light source, and The higher the value, the more concentrated the beam. Represented as the first The angle between the surface normal at a point and the vector pointing towards the light source. Represented as the first The distance between the point and the virtual light source.

[0025] In one embodiment, the steps for generating a standard product image are as follows: Based on the configured virtual camera and virtual light source model, the product's 3D CAD model is input, and optical rendering is performed in an offline virtual simulation environment. The expression for optical rendering is: In the formula, Represented as the surface of a 3D CAD model The point is along the exit direction of the virtual camera. The emitted radiance, and the radiance values ​​corresponding to all pixels constitute a standard product image. Represented as the surface number The point's own luminous value is usually 0. Represented as the surface number The hemispherical space centered on the point normal. Represented as the surface number The BRDF at the point is used to describe the direction of incidence. How is incoming light reflected in the direction of outgoing light? , Represented as contributions from all virtual light sources from direction Incident point Radiance It is expressed as the cosine of the angle between the incident ray and the surface normal. Represented as relative to the incident direction Differentiate; This includes, for each pixel on the virtual camera sensor, emitting a ray of light from the camera's optical center that passes through that pixel and enters the virtual scene; After the light rays intersect with the surface of the 3D CAD model, the amount of light radiation entering the camera along the direction of the light rays is calculated based on the material properties at the intersection point and the virtual light source information of all direct lighting and indirect reflected light in the virtual scene, combined with the surface reflectivity and roughness of the 3D CAD model. The process is performed in parallel on all pixels of the image to calculate the radiance value of all light rays passing through all pixels. After tone mapping, a standard product image with accurate geometric contours and realistic lighting and shadow effects is generated, which is noise-free and distortion-free.

[0026] In one embodiment, the steps for defining the measurement tool layout and reference parameters are as follows: The layout of the measurement tool includes an edge detection area, a blob analysis area, and the corresponding feature search direction; For edge detection areas, a measurement tool is defined on the generated standard product image using a script based on the detection requirements. This includes defining a rectangular region ROI at the location where the size or contour needs to be measured at the product boundary and specifying the search direction perpendicular to the edge. The corresponding benchmark parameters include: the ideal sub-pixel edge position extracted from a standard product image using a sub-pixel edge detection algorithm combining the Canny operator and grayscale gradient interpolation; and the theoretical edge design size based on the ideal sub-pixel edge position. The expression for the theoretical edge design size is as follows: In the formula, Represented as edge theory design dimension values, This is represented by the conversion factor from pixels obtained from camera calibration to actual size. , Represented as subpixel coordinates of two parallel edges on a standard product image; For spot analysis areas, one or more rectangular or arbitrary shaped analysis areas are defined in the areas where stains, impurities, or specific textures need to be detected. The corresponding benchmark parameters include: the average gray value of qualified products within the analysis area, the statistical gray-scale characteristics of the gray-scale standard deviation, and the allowable spot area threshold and spot contrast threshold. The expression for the average gray value is: In the formula, This is represented as the average gray value within the analysis area. This is expressed as the total number of pixels within the analysis area. Represented as the first in the analysis region The grayscale value of each pixel; The expression for the standard deviation of gray level is: In the formula, This is expressed as the standard deviation of grayscale values ​​within the analysis area; The formula for calculating spot contrast is: In the formula, Represented as spot contrast. The average gray level of the candidate spot analysis region is represented as . This represents the average gray level of the background region surrounding the candidate spot region.

[0027] The technical explanation and effects of this embodiment are as follows: In an offline simulation environment, optical simulation software or dedicated software integrating a physical rendering engine is used. When creating the template, the 3D CAD model of the target product is first imported, and its surface is given realistic material optical properties such as roughness and reflectivity. Combined with a virtual camera model created in the software that is completely identical to the physical industrial camera model in terms of lens focal length and distortion coefficient parameters, and a virtual light source model that matches the luminous characteristics of the actual physical light source, the geometric and optical properties of the real hardware are digitized using the projection formula in the virtual camera model and the irradiance formula in the virtual light source model. This establishes a mapping relationship between the virtual environment and the physical world. By adjusting the virtual camera pose and light source parameters, the 3D CAD model of the product is optically "rendered" in the virtual environment, generating an image similar to a real photograph. Standard product images, which are highly consistent in terms of geometric and lighting physics and are free of noise and distortion, provide a comparable basis for subsequent "virtual-real mapping." Furthermore, the use of explicit sub-pixel edge detection algorithms and statistical grayscale features on these standard product images automatically and quantitatively defines the layout of measurement tools and baseline parameters. This fundamentally avoids subjective errors and randomness inherent in manual settings, ensuring that each digital twin inspection template is the optimal and reproducible "gold standard" for each product model. This improves the accuracy and consistency of the digital twin inspection templates. When the system is used for model changes, only a small number of physical samples need to be compared and calculated to transfer the template parameters to actual working conditions with high confidence. This significantly reduces reliance on on-site debugging and manual experience, enhancing the stability and reliability of the entire inspection system when switching between different products and batches.

[0028] S2. When a target product model switching instruction is received on the production line, the digital twin detection template matching the target model is automatically called, the virtual camera and virtual light source model parameters are directly parsed, and the programmable motion mechanism and physical light source controller carrying the industrial camera are driven to automatically adjust the industrial camera and physical light source to the preset working state and pose consistent with the virtual environment created by the digital twin detection template of the corresponding target model. In one embodiment, the preset working state and pose are intended to reproduce the physical imaging environment of the production line product as the virtual environment when the digital twin inspection template is generated. This requires adjusting the industrial camera and physical light source to a quantified target state that is uniquely determined by all parameters within the digital twin inspection template, including the layout and reference parameters of all measuring tools, and recording the model parameters of the virtual camera and virtual light source. First, from the target model's digital twin detection template, the intrinsic and extrinsic parameters of the virtual camera and virtual light source models stored in the file package are parsed. For the virtual camera model, this includes the extrinsic rotation matrix, extrinsic translation vector, intrinsic lens focal length, and intrinsic pixel size obtained from physical camera calibration measurements. The extrinsic rotation matrix and extrinsic translation vector, after a homogeneous transformation matrix, determine the target pose of the camera in virtual space, including its position and orientation. The intrinsic lens focal length and intrinsic pixel size define the target's optical state. For the virtual light source model, this includes the target position, illumination angle, and brightness values ​​of the light intensity distribution. The expression for the homogeneous transformation matrix of the target pose is: In the formula, This is represented as a transformation from the camera's two-dimensional coordinate system to the world coordinate system; Based on the target pose of the virtual camera, the target angles or positions required for each joint on the programmable motion mechanism carrying the industrial camera are calculated using inverse kinematics. This drives the programmable motion mechanism to execute preset working pose commands. Simultaneously, the parameters of the virtual light source are converted into voltage, PWM, or linear digital commands executable by the physical light source controller, driving the physical light source controller to adjust preset working state commands for current or duty cycle until the feedback brightness matches the target brightness value. The inverse kinematics expression is as follows: ,and ,as well as , In the formula, This is represented as the need to calculate the target angle vectors of each joint on the programmable motion mechanism. It is represented as a fixed transformation matrix from the programmable motion mechanism coordinate system to the world coordinate system. It is represented as a fixed transformation matrix from the programmable motion mechanism coordinate system to the camera coordinate system. Represented as the first in a programmable motion mechanism Joint target angle vector This is represented as a positive integer; After the programmable motion mechanism and physical light source controller automatically adjust the industrial camera and physical light source, it is considered that the system has reached the preset working state and pose consistent with the virtual environment of the digital twin detection template corresponding to the target model.

[0029] The technical explanation and effects of this embodiment are based on the offline simulation environment in step S1, where the model parameters of the virtual camera and virtual light source are determined according to the digital twin template optimized by physical simulation for each product model. This provides a scientific and important basis for adjusting the parameters in the target digital twin template when switching models. By using inverse kinematics, involving a series of fixed coordinate transformations between the camera coordinate system, the world coordinate system, and the coordinate system of the hardware programmable motion mechanism, the camera pose in the virtual environment is accurately mapped to the spatial pose of the programmable motion mechanism in the real physical environment. This allows the hardware to be accurately reproduced to the optimal working state in the shortest time, improving the efficiency of model switching detection. This completely replaces the process of manually using measuring tools and repeatedly taking test shots to adjust the camera and light source in traditional model switching, greatly improving the stability of the entire detection system and the traceability of product quality control.

[0030] S3. After the hardware driver adjustment is completed, the industrial camera will capture the actual product of the target model currently being transmitted on the production line and acquire at least one actual sample image containing complete product features. In one embodiment, the steps for obtaining the actual sample image are as follows: Once the hardware driver adjustment is complete, an image acquisition signal is triggered and sent to the industrial camera; The target product moves along the conveyor belt. When it passes a preset, fixed physical shooting position, the industrial camera completes the exposure and readout of a frame image in a very short exposure time to obtain the original image. The acquired raw images are transmitted to the visual processing and analysis engine via a high-speed interface for rapid quality checks of complete product features, including image integrity, overall sharpness, and average brightness. An original image is validly marked as an actual sample image only if the quality of the original image containing complete product features is assessed as passing. If the quality of the original image does not contain complete product features, the acquisition of the next frame is immediately retried until at least one quality-compliant image is obtained.

[0031] The technical explanation and effects of this embodiment are as follows: After the hardware driver adjustment is completed, the acquisition of the actual product image is triggered, which ensures that the relative spatial position of the product and the industrial camera is strictly fixed at the time of acquisition. This can eliminate the random timing error caused by the free operation sampling of products on the production line, making the images acquired each time comparable. Furthermore, the visual processing and analysis engine performs a quality pre-evaluation on the acquired original images, ensuring the quality of the actual sample images containing complete product features. This improves the accuracy and stability of the subsequent "virtual-real mapping" calculation, as well as the practicality and robustness of the overall system process, making it more adaptable to real and non-ideal industrial site environments.

[0032] S4. Using an image feature point matching algorithm, the actual sample image is registered with the standard product image in the digital twin detection template with high precision. Based on the registration result, the affine transformation matrix from the coordinate system of the standard product image to the coordinate system of the actual sample image is calculated as the mechanical positioning geometric deviation, and the average gray level difference between the two images in the overlapping area after registration is calculated as the global brightness deviation. In one embodiment, the registration steps between the actual sample image and the standard product image are as follows: Gaussian filtering and noise reduction preprocessing are performed on the actual sample images and the standard product images retrieved from the digital twin detection template; An image feature point matching algorithm is used to extract a set of highly discriminative key points and descriptors from two images. The descriptor is a high-dimensional vector that encodes the local appearance features of the key point neighborhood. The similarity between keypoints and descriptors is calculated using Euclidean distance. For each keypoint in the actual sample image, the most similar candidate matching point is found in the standard product image, forming a preliminary set of matching pairs. The expression for the similarity between a keypoint and a descriptor is: In the formula, Represented as the first image in the actual sample image and the standard product image The similarity between key points and descriptors Represented as the first in the actual sample image Descriptor vectors of key points Represented as the first in the standard product image Descriptor vectors of key points Represented as the Euclidean distance calculation symbol. Represented as an actual sample image, Represented as a standard product image, Represented as an index of key points; Robust estimation using the Random Sample Consensus (RANSAC) algorithm involves randomly selecting a minimum sample set of at least three pairs from the set of matched pairs. A candidate affine transformation matrix with a projection error less than a preset threshold is calculated, and the number of interior points in the matched pairs that conform to the affine transformation matrix is ​​checked. The projection error of the matched pair is calculated using the following formula: ,and , In the formula, This is expressed as projection error, used to describe the pixel distance of a matching pair. This is represented as the coordinates of a point in a standard product image within a matching pair. Let it be the calculated candidate affine transformation matrix to be tested. This is represented as the coordinates of a point in the actual sample image within a matching pair; The expression for interior point determination is: In the formula, This is represented as a preset threshold, such as 2 pixels. When the projection error of a matching pair is less than the preset threshold, the matching pair is determined to be an interior point under the current affine transformation matrix. The RANSAC algorithm process is repeated multiple times until the matching pairs that match the interior points of the corresponding candidate affine transformation matrix with the most interior points are selected. At the same time, a preliminary affine transformation model is obtained, and all interior point matching pairs are selected. Based on all the selected interior point matching pairs, the optimal affine transformation matrix is ​​recalculated using the least squares method on the initial affine transformation model to minimize the sum of the projection errors of all interior points, thus obtaining a high-precision and stable registration result.

[0033] In one embodiment, the steps for calculating the deviation between the affine transformation matrix and the average grayscale difference are as follows: Based on the registration results, all interior point matching pairs are statistically analyzed and used. An affine transformation matrix, representing the virtual-to-real mapping relationship between the "standard product image coordinate system" and the "actual sample image coordinate system," is then calculated using the least squares method. This matrix serves as the geometric deviation for mechanical positioning. The expression for the affine transformation matrix is ​​as follows: In the formula, Represented as transformation matrix elements for scaling, rotation, and shearing. The vector elements are represented as translations, and the optimal transformation matrix elements and translation vector elements are obtained by using the least squares method through all interior point matching pairs. The obtained element parameters satisfy the condition that the mapping error of all interior point matching pairs is minimized. Using the obtained affine transformation matrix, the standard product image is resampled using bilinear interpolation to generate a geometrically aligned standard image that is perfectly aligned with the actual sample image. Within the effective overlap area between the actual sample image and the aligned standard image, the grayscale difference is calculated by traversing all pixels. The arithmetic mean of all pixel grayscale differences is then used as the global brightness deviation, where the expression for the global brightness deviation is: In the formula, This is represented as global brightness deviation. This represents the total number of pixels within the effective overlap area between the actual sample image and the aligned standard image. Represented as the actual sample image at the Pixel position grayscale value at that location Represented as an aligned standard image in the 1st... Pixel position grayscale value at that location Represented as the first The grayscale value at each pixel location Represented as a standard product image in the first Pixel position The grayscale value at that location.

[0034] The technical explanation and effects of this embodiment, by employing an image feature point matching algorithm combined with the Random Sample Consensus (RANSAC) algorithm, enable the system to automatically derive the geometric correspondence between two images and resist common noise, local occlusion, or non-rigid deformation interference in on-site images. It robustly estimates the affine transformation matrix representing the overall mechanical positioning geometric deviation, and then uses the affine transformation matrix to resample the standard product image to obtain an aligned standard image. The average value is calculated using the grayscale difference of all pixels within the effective overlapping area of ​​the actual sample image and the aligned standard image, representing the global brightness deviation. By decoupling the difference between virtual and reality into two independent variables with clear physical meaning: the affine transformation matrix and the average grayscale difference, where the affine transformation matrix represents the pure geometric deviation caused by mechanical positioning and installation errors, and the average grayscale difference represents the global photometric deviation caused by light source attenuation and changes in light intensity, this decoupling makes subsequent compensation more targeted. It breaks away from the traditional on-site calibration work that requires manual use of calibration objects or repeated trial and error, and is able to adapt to subtle mechanical and lighting changes after each model change, ensuring the real-time accuracy of the detection benchmark and significantly improving the system's intelligence and robustness.

[0035] S5. Use an affine transformation matrix to perform geometric transformation on all coordinate points of the measurement tool layout defined in the digital twin inspection template. Use the average gray level difference to compensate for the offset of the gray level threshold in the reference parameters. Use the transformed and compensated parameters to generate an online inspection program that is applied to the current real physical environment, and immediately run the online inspection program to perform online defect detection of products on the production line.

[0036] In one embodiment, the geometric transformation and offset compensation processing steps are as follows: Geometric Transformation: Traverse the layout and reference parameters of all measurement tools defined in the digital twin inspection template, including edge detection lines, contour detection regions, and blob analysis region boxes, and extract the geometric definition points of each tool, including the start and end points of the lines, the vertex coordinates of the regions, and the center and boundary points of the boxes; The affine transformation matrix obtained in step S4 is used to perform coordinate transformation on each geometrically defined point to obtain new coordinates in the actual sample image coordinate system. The expression for the coordinate transformation is: In the formula, Represented as the measurement tool in the digital twin inspection template The homogeneous coordinates of a geometrically defined point in the standard product image coordinate system. Represented as an affine transformation matrix controlling rotation, scaling, and translation. Represented as the first The corresponding homogeneous coordinates of each geometrically defined point in the actual sample image coordinate system after transformation; Compensation offset: Traverse all the reference parameters related to absolute grayscale in the digital twin detection template, including the binarization threshold, the upper and lower limits of the grayscale range for detecting spot area, the spot contrast threshold, and the reference grayscale value for spot contrast judgment. The average grayscale difference obtained in step S4 is used to offset the baseline parameters overall, generating a correction threshold suitable for the current lighting conditions to compensate for the overall brightness difference between the actual lighting conditions and the virtual standard lighting conditions. The expression for the correction threshold is: In the formula, This represents the corrected grayscale threshold used for the current physical environment after compensation. This represents the original grayscale threshold set in the digital twin detection template.

[0037] In one embodiment, the steps for generating the online detection program are as follows: Program skeleton cloning: Loading the logical skeleton of the detection program from the digital twin detection template, including the sequence of the detection process, the types of measurement tools used and their calling relationships, defect judgment logic, etc. The expression of the detection program is as follows: In the formula, This is represented as the function representation of the detection program. Represented as the input image, This is represented as the test result. Represented as the original geometric and grayscale parameter set; Parameter injection: The new set of coordinate points after geometric transformation and the new set of corrected grayscale thresholds after grayscale compensation are injected into the corresponding variable positions in the program skeleton according to the predefined mapping relationship; Dynamic program compilation / configuration: Depending on the system architecture, the updated program may be enabled in two ways: one is by dynamically compiling a new executable kernel from the visual processing and analysis engine; the other is by dynamically configuring the properties of existing detection tools through the engine's API. Integrity Verification and Activation: The system performs a rapid verification of the newly generated testing program, including checking whether all measuring tools have been correctly assigned values ​​and whether the parameters are within reasonable ranges. After successful verification, the new program is marked as the currently active online testing program and is in an executable, ready state. The expression for the new testing program is: ,and ,as well as , In the formula, This represents the function representation of the new detection program. This can be represented as applying an affine transformation matrix to all geometric points. This is represented by adding a global brightness offset to all absolute grayscale thresholds. It is represented as an affine transformation matrix that controls rotation, scaling, and translation.

[0038] The technical explanation and effects of this embodiment are as follows: Through the processing results of steps S1-S4, step S5 generates a directly executable detection program for online migration and immediate activation. The detection program is designed to be highly parameterized, which allows a new program adapted to a new environment to be quickly generated by batch replacing parameters without rewriting or redesigning the detection logic. The geometric transformation strictly follows the affine transformation model derived from image registration, ensuring precise spatial alignment between the measuring tool and the actual product. Grayscale compensation is based on the average brightness deviation calculated from physical imaging differences, enabling photometric criteria to track changes in illumination. This allows the entire generation and activation process to be completed automatically at the software level, achieving detection program switching within milliseconds. Thus, after the physical hardware adjustment is completed, an instantaneous switch from "debugging state" to "production detection state" is achieved, without requiring any additional interruption to the production line.

[0039] Example 2 This invention provides, for example Figure 4 The machine vision-based production line product defect detection system shown includes a digital twin inspection template management module: used to create, store, manage, and call digital twin inspection templates. Creation: For each product model to be inspected, in an offline simulation environment, based on the 3D CAD model of that model, a virtual camera and virtual light source model are configured and optically rendered to generate a standard product image. The layout and reference parameters of the measuring tools are defined according to the standard product image, and the model parameters of the virtual camera and virtual light source are recorded. Finally, this information is packaged into a digital twin inspection template file. Storage: The created digital twin inspection templates corresponding to each product model are securely and systematically stored for later retrieval. Management: The stored digital twin inspection templates are managed. Retrieval: When a target product model switching instruction is received on the production line, the matching digital twin inspection template can be quickly and accurately called. Hardware driver control module: Used to receive the matching product model digital twin inspection template called by the digital twin inspection template management module, parse the virtual camera and virtual light source model parameters, control the programmable motion mechanism and physical light source controller, and automatically adjust the industrial camera and physical light source to the preset working state and pose consistent with the virtual environment, thus completing the adaptive adjustment of the physical hardware; Image acquisition module: After the hardware driver control module completes the adaptive adjustment of the physical hardware, it is used to control the industrial camera to capture the actual product of the target model currently being transported on the production line, and to obtain the actual sample image, providing a data basis for subsequent image processing and analysis; Visual processing and analysis engine: Embedded image registration algorithm, it registers the acquired actual sample image with the standard product image provided by the digital twin inspection template management module, finds the correspondence between the two images, and compares image features based on the registration result. It calculates the affine transformation matrix from the coordinate system of the standard product image to the coordinate system of the actual sample image, and calculates the average gray-level difference between the two images in the overlapping area after registration, realizing virtual-real mapping and deviation calculation. It also uses the affine transformation matrix to perform geometric transformation on all coordinate points of the measurement tool layout defined in the digital twin inspection template, uses the average gray-level difference to compensate for the gray-level threshold in the reference parameters, and uses the transformed and compensated parameters to generate an online inspection program and immediately execute online defect detection of the product. It is used to perform image registration, feature comparison, virtual-real mapping and deviation calculation, as well as generate and run online inspection programs. Central control module: Used to coordinate and control various modules, respond to product changeover instructions from the industrial production line, and control the switching and execution of the product testing process on the production line, so as to ensure that the testing work can smoothly adapt to the testing needs of different product models.

[0040] In one embodiment, the programmable motion mechanism is a multi-degree-of-freedom motion platform that carries and adjusts the pose of an industrial camera; The physical light source controller is an LED light source assembly whose brightness and angle can be independently programmed and adjusted; The driving process involves converting the three-dimensional spatial coordinates and angle parameters in the digital twin detection template into motion control commands for the programmable motion mechanism and the physical light source controller. The hardware-driven control module communicates with the programmable motion mechanism and the physical light source controller via the EtherCAT bus protocol.

[0041] The technical explanation and effects of this embodiment are as follows: In this system, the central control module sends a product change command to the digital twin inspection template management module, controlling it to call the corresponding model of digital twin inspection template; it sends a hardware adjustment start signal to the hardware driver control module, enabling it to complete adaptive adjustments of the physical hardware according to the parameters of the digital twin inspection template; it sends an image acquisition command to the image acquisition module, controlling the industrial camera to photograph the actual product; it sends a processing and analysis command to the vision processing and analysis engine, initiating image registration, feature comparison, virtual-real mapping and deviation calculation, and online inspection program operation; simultaneously, the central control module receives feedback information from each module, performing real-time monitoring and adjustment of the entire inspection process; the digital twin inspection template management module displays standard product images... The image acquisition module provides the actual sample image to the vision processing and analysis engine for subsequent image registration and feature comparison. The vision processing and analysis engine uses the actual sample image to perform registration, feature comparison, and other operations with the standard product image to complete the online defect detection of the product. After the hardware drive control module completes the adaptive adjustment of the physical hardware of the programmable motion mechanism and physical light source controller to the industrial camera, it sends a hardware adjustment completion signal to the image acquisition module, triggering the image acquisition module to control the industrial camera to shoot the target model product and obtain the actual sample image. Due to the synergy between the modules, the entire production line product defect detection system can efficiently and accurately complete the defect detection tasks of different models of production line products.

[0042] The machine vision-based assembly line product defect detection system provided in this embodiment of the invention is implemented through the above-described machine vision-based assembly line product defect detection method. For details of the specific methods and processes of the machine vision-based assembly line product defect detection system, please refer to the embodiments of the above-described machine vision-based assembly line product defect detection method, which will not be repeated here.

[0043] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0044] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0045] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0046] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0047] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A machine vision-based method for detecting defects in production line products, characterized in that, Includes the following steps: S1. For each product model to be inspected, in an offline simulation environment, based on the 3D CAD model of each product model, configure the virtual camera and virtual light source model and perform optical rendering to generate a standard product image; define the layout and reference parameters of the measuring tools on the standard product image, and record the model parameters of the virtual camera and virtual light source, and package and store them together as a digital twin inspection template. S2. When receiving a target product model switching instruction on the production line, automatically call the digital twin detection template matching the target model, directly parse the virtual camera and virtual light source model parameters, drive the programmable motion mechanism carrying the industrial camera and the physical light source controller, and automatically adjust the industrial camera and physical light source to the preset working state and posture consistent with the virtual environment. S3. After the hardware driver adjustment is completed, take a picture of the target model product currently being transmitted on the production line and acquire at least one actual sample image. S4. Use an image template matching algorithm to register the actual sample image with the standard product image in the digital twin detection template; Based on the registration results, the affine transformation matrix from the standard product image coordinate system to the actual sample image coordinate system is calculated, and the average gray level difference between the two images in the overlapping area after registration is calculated. S5. Use an affine transformation matrix to perform geometric transformation on all coordinate points of the measurement tool layout defined in the digital twin inspection template, use the average gray level difference to compensate for the offset of the gray level threshold in the reference parameters, use the transformed and compensated parameters to generate an online inspection program and immediately execute online defect detection of the product.

2. The machine vision-based defect detection method for production line products according to claim 1, characterized in that, The configuration steps for the virtual camera and virtual light source model are as follows: The system calibrates and measures real vision hardware cameras to obtain the precise pose of physical industrial cameras in the production line coordinate system, as well as intrinsic parameters such as lens focal length and pixel size. Simultaneously measure the position, illumination angle, luminous intensity, and spectral characteristics of the physical light source LED array relative to the industrial camera and the product; A physically based ray tracing renderer is used in an offline simulation environment to create a virtual scene that corresponds to the real pipeline environment; In a virtual scene corresponding to a real production line environment, a virtual camera model is established based on the internal parameters of a physical industrial camera, and the viewpoint and angle of the virtual camera are set in combination with the precise pose of the industrial camera in the production line coordinate system. For virtual light sources, a corresponding virtual light source model is created based on the geometric position, orientation angle, light intensity distribution, and spectral data of the physical light source LED array group to ensure that the light characteristics emitted by the virtual light source model in the virtual scene are physically equivalent to those in reality.

3. The machine vision-based defect detection method for production line products according to claim 2, characterized in that, The steps for generating the standard product image are as follows: Based on the configured virtual camera and virtual light source model, input the product's 3D CAD model and perform optical rendering in an offline virtual simulation environment; This includes, for each pixel on the virtual camera sensor, emitting a ray of light from the camera's optical center that passes through that pixel and enters the virtual scene; After the light rays intersect with the surface of the 3D CAD model, the amount of light radiation entering the camera along the direction of the light rays is calculated based on the material properties at the intersection point and the virtual light source information of all direct lighting and indirect reflected light in the virtual scene, combined with the surface reflectivity and roughness of the 3D CAD model. The process is performed in parallel on all pixels of the image to calculate the radiance values ​​of all light rays passing through all pixels. After tone mapping, a standard product image with accurate geometric contours and realistic lighting and shadow effects is generated.

4. The machine vision-based defect detection method for production line products according to claim 3, characterized in that, The steps for defining the layout of the measuring tool and the reference parameters are as follows: The layout of the measurement tool includes an edge detection area, a blob analysis area, and the corresponding feature search direction; For edge detection areas, a measurement tool is defined on the generated standard product image using a script based on the detection requirements. This includes defining a rectangular region ROI at the location where the size or contour needs to be measured at the product boundary and specifying the search direction perpendicular to the edge. The corresponding benchmark parameters include: the ideal sub-pixel edge position extracted on the standard product image by using the Canny operator combined with gray-scale gradient interpolation sub-pixel edge detection algorithm, and the edge theoretical design size value based on the ideal sub-pixel edge position; For spot analysis areas, one or more rectangular or arbitrary shaped analysis areas are defined in the areas where stains, impurities, or specific textures need to be detected. The corresponding benchmark parameters include: the average gray value of qualified products within the analysis area, the statistical gray characteristics of the gray standard deviation, and the allowable spot area threshold and spot contrast threshold.

5. The machine vision-based defect detection method for production line products according to claim 1, characterized in that, The preset working state and pose are to make the physical imaging environment of the production line product reproduce the virtual environment when the digital twin detection template is generated. It is necessary to adjust the industrial camera and physical light source to a quantified target state that is uniquely determined by all parameters in the digital twin detection template. First, from the target model digital twin detection template, the intrinsic and extrinsic parameters of the virtual camera and virtual light source models stored in the file package are parsed. For the virtual camera model, the extrinsic parameters include the rotation matrix, translation vector, lens focal length, and pixel size obtained from the physical camera calibration measurement. For the virtual light source model, the luminance values ​​of the target position, illumination angle, and light intensity distribution are included. Based on the target pose of the virtual camera, the target angle or position to be reached by each joint on the programmable motion mechanism carrying the industrial camera is calculated by inverse kinematics, so as to drive the programmable motion mechanism to execute preset working pose instructions. At the same time, the parameters of the virtual light source are converted into voltage, PWM or linear digital instructions that can be executed by the physical light source controller.

6. The machine vision-based defect detection method for production line products according to claim 1 or 5, characterized in that, The registration steps between the actual sample image and the standard product image are as follows: Gaussian filtering and noise reduction preprocessing are performed on the actual sample images and the standard product images retrieved from the digital twin detection template; An image feature point matching algorithm is used to extract a set of highly discriminative key points and descriptors from two images. The descriptor is a high-dimensional vector that encodes the local appearance features of the key point neighborhood. Based on the similarity between keypoints and descriptors calculated using Euclidean distance, the most similar candidate matching point is found in the standard product image for each keypoint in the actual sample image, forming a preliminary set of matching pairs. Robust estimation is performed using the Random Sampling Consensus (RANSAC) algorithm. This involves randomly selecting a minimum sample set of at least three pairs from the set of matching pairs, calculating a candidate affine transformation matrix with a projection error less than a preset threshold, and verifying the number of interior points of the matching pairs that conform to the affine transformation matrix. The RANSAC algorithm process is repeated multiple times until the matching pairs that match the interior points of the corresponding candidate affine transformation matrix with the most interior points are selected. At the same time, a preliminary affine transformation model is obtained, and all interior point matching pairs are selected. Based on all the selected interior point matching pairs, the optimal affine transformation matrix is ​​recalculated using the least squares method on the initial affine transformation model to minimize the sum of the projection errors of all interior points, thus obtaining a high-precision and stable registration result.

7. The machine vision-based method for detecting defects in production line products according to claim 6, characterized in that, The steps for calculating the deviation between the affine transformation matrix and the average grayscale difference are as follows: Based on the registration results, all interior point matching pairs are statistically analyzed and used. An affine transformation matrix is ​​obtained by using the least squares method to solve for the virtual-real mapping relationship between the "standard product image coordinate system" and the "actual sample image coordinate system", which serves as the geometric deviation for mechanical positioning. Using the obtained affine transformation matrix, the standard product image is resampled using bilinear interpolation to generate a geometrically aligned standard image that is perfectly aligned with the actual sample image. Within the effective overlap area between the actual sample image and the aligned standard image, the grayscale difference is calculated by traversing the pixels. The arithmetic mean of all pixel grayscale differences is then used as the global brightness deviation.

8. The machine vision-based method for detecting defects in production line products according to claim 7, characterized in that, The geometric transformation and compensation offset processing steps are as follows: Geometric Transformation: Traverse the layout and reference parameters of all measurement tools defined in the digital twin inspection template, including edge detection lines, contour detection regions, and blob analysis region boxes, and extract the geometric definition points of each tool, including the start and end points of the lines, the vertex coordinates of the regions, and the center and boundary points of the boxes; By applying an affine transformation matrix to transform the coordinates of each geometrically defined point, new coordinates are obtained in the actual sample image coordinate system. Compensation offset: Traverse all the reference parameters related to absolute grayscale in the digital twin detection template, including the binarization threshold, the upper and lower limits of the grayscale range for detecting spot area, the spot contrast threshold, and the reference grayscale value for spot contrast judgment. The average grayscale difference is applied to offset the baseline parameters as a whole, generating a correction threshold suitable for the current lighting conditions to compensate for the overall brightness difference between the actual lighting conditions and the virtual standard lighting conditions.

9. A machine vision-based assembly line product defect detection system, implemented by the machine vision-based assembly line product defect detection method according to any one of claims 1-8, characterized in that, Includes a digital twin detection template management module: used for creating, storing, managing, and retrieving digital twin detection templates; Hardware-driven control module: used to analyze parameters based on the digital twin detection template matching the product model, control the programmable motion mechanism and physical light source controller, and complete the adaptive adjustment of the physical hardware; Image acquisition module: Used to control an industrial camera to photograph the actual product of the target model and obtain images of the actual sample; Visual processing and analysis engine: Embedded image registration algorithm, used to perform image registration, feature comparison, virtual-real mapping and deviation calculation, as well as generate and run online detection programs; Central control module: Used to coordinate various modules, respond to product changeover instructions on the industrial production line, and control the switching and execution of the product inspection process on the production line.

10. The machine vision-based assembly line product defect detection system according to claim 9, characterized in that, The programmable motion mechanism is a multi-degree-of-freedom motion platform that supports and adjusts the pose of an industrial camera. The physical light source controller is an LED light source assembly whose brightness and angle can be independently programmed and adjusted; The hardware-driven control module communicates with the programmable motion mechanism and the physical light source controller via the EtherCAT bus protocol.

Images