Automobile part surface defect online detection method and system based on machine vision

Abstract

This invention discloses an online detection method and system for surface defects of automotive parts based on machine vision, belonging to the field of computer vision. The method includes: acquiring spatial point cloud data of the part surface; performing local plane fitting on the point cloud to extract the normal vector of the center position of the area to be measured; synchronously adjusting the imaging pose of the industrial camera and the illuminance distribution of the multi-array light source system according to the normal vector, aligning the imaging principal axis with the normal vector, and constructing a spatial non-uniform illuminance distribution model; distinguishing between specular reflection paths and large-angle illumination paths based on the angle between the emission direction of the light-emitting unit and the normal vector, and performing differentiated brightness adjustment; capturing image sequences with different exposure parameters, performing feature alignment and semantic fusion through a two-stage neural network to reconstruct a high-fidelity defect texture image; and inputting the image into a deep residual network for defect detection and classification. This invention suppresses specular reflection and shadow blind spots from the source of optical imaging, improving the accuracy of surface defect detection for complex-shaped, highly reflective parts.

Description

Technical Field

[0001] This application belongs to the field of computer vision, specifically relating to an online detection method and system for surface defects of automotive parts based on machine vision. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, machine vision plays a key role in the quality monitoring of automotive parts. Surface defect detection of parts with complex geometric shapes (such as deep holes, protrusions, and freeform surfaces) and highly reflective material properties is a current technical challenge in the field of industrial vision. Such detection usually uses industrial cameras to collect surface images and algorithms to identify defects such as micro-cracks and pores.

[0003] However, the optical systems and detection methods commonly used in existing technologies are insufficient to effectively address imaging interference caused by the complex morphology and high reflectivity of components. For example, patent CN110632087A discloses a surface defect detection device that uses multiple light sources illuminated at different angles and times to enhance the contrast between defects and the background; another example is patent CN114397304A, which discloses a component surface defect detection system that adjusts the camera orientation based on the three-dimensional information of the component to be detected. Although these methods attempt to improve image quality through multi-angle illumination or camera pose adjustment, their control strategies remain at the macroscopic level, lacking precise perception and utilization of the local geometric features of the component surface, and unable to dynamically adjust the optical path according to the orientation of each point on the surface.

[0004] Specifically, due to the lack of geometric information extraction of local surface normals, existing technologies cannot ensure that the imaging axis of industrial cameras is always precisely aligned with the normal of the measured surface. They also cannot perform differentiated brightness adjustment for each unit in the light source array based on the angular relationship between each light source unit and the surface normal. As a result, when facing highly reflective metal surfaces, fixed or coarse optical adjustments cannot systematically suppress specular reflections and shadow blind spots from the source. Specular reflections will cause local overexposure of the image, while concave areas will form dark areas due to insufficient lighting. This results in the loss of bright information and blurring of dark details in the obtained image, which seriously limits the accuracy of subsequent defect detection. Summary of the Invention

[0005] The purpose of this invention is to provide an online detection method and system for surface defects of automotive parts based on machine vision, which can effectively solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The first aspect is a machine vision-based online detection method for surface defects in automotive parts, including: Spatial point cloud data of component surfaces are acquired through structured light 3D measurement while the components are in motion. Local plane fitting is performed on the point cloud within the test area in the spatial point cloud data to extract the normal vector at the center position of the test area; Based on the normal vector of the center position of the area under test, the imaging pose of the industrial camera and the illuminance distribution of the multi-array light source system are simultaneously adjusted, including: controlling the movement of the industrial camera to align the imaging principal axis of the industrial camera with the normal vector of the center position of the area under test; and constructing a spatial non-uniform illuminance distribution model. For each light-emitting diode (LED) unit in the multi-array light source system, based on the angle between the light-emitting principal axis direction of the LED unit and the normal vector of the center position of the area under test, the LED unit is determined to be in a mirror reflection path or a large tilt angle illumination path, and differentiated brightness adjustment is performed on the LED units determined to be in different paths to suppress specular reflection on the surface of the component and compensate for insufficient illuminance in the shadow area. During the illuminance distribution adjustment process of the multi-array light source system, a sequence of images of the test area under different exposure parameters are continuously captured, and a high-fidelity defect texture image is reconstructed by performing feature alignment and semantic fusion on the sequence of images through a two-stage neural network model. The high-fidelity defect texture image is input into a deep residual convolutional neural network detection model to detect and classify surface defects of parts and output the detection results.

[0007] Secondly, a machine vision-based online inspection system for surface defects in automotive parts is provided to implement the aforementioned method. The system includes a central controller, a 3D sensor, a six-axis robotic arm, an industrial camera, a multi-array light source system, and a high-performance computing unit. The central controller is used to perform optical pose coordination control, and according to the normal vector of the center position of the area to be measured calculated by the high-performance computing processing unit, control the six-axis robotic arm to adjust the imaging pose of the industrial camera so that the imaging main axis of the industrial camera is aligned with the normal vector of the center position of the area to be measured; and control the multi-array light source system to construct a spatial non-uniform illuminance distribution model and perform differentiated brightness adjustment. The high-performance computing processing unit is used to process the spatial point cloud data acquired by the three-dimensional sensor to calculate the normal vector of the center position of the area to be measured, to perform feature alignment and semantic fusion of the sequence images captured by the industrial camera using a two-stage neural network model, and to detect and classify surface defects of parts using a deep residual convolutional neural network detection model.

[0008] In summary, this application includes at least one of the following beneficial technical effects: 1. This application acquires spatial point cloud data of the component surface through structured light three-dimensional measurement, and performs local plane fitting on the point cloud in the area to be measured in the spatial point cloud data to extract the normal vector of the center position of the area to be measured. This enables the optical imaging system to accurately perceive the local geometric orientation of the component surface, providing a geometric reference for the precise adjustment of the subsequent imaging posture and illumination distribution, and overcoming the defect in the prior art that it is impossible to dynamically adjust the optical path according to the orientation of each point on the surface.

[0009] 2. This application simultaneously adjusts the imaging pose of the industrial camera and the illuminance distribution of the multi-array light source system according to the normal vector of the center position of the area under test, so that the imaging principal axis of the industrial camera is aligned with the normal vector of the center position of the area under test, thus eliminating perspective distortion caused by the surface undulations of the parts. At the same time, by constructing a spatial non-uniform illuminance distribution model, the light-emitting diode unit is determined to be in a mirror reflection path or a large tilt angle illumination path according to the angle between the light-emitting principal axis direction of each light-emitting diode unit and the normal vector of the center position of the area under test. Differentiated brightness adjustment is performed on the light-emitting diode units determined to be in different paths, which suppresses specular reflection on the surface of the parts from the source of optical imaging and compensates for insufficient illuminance in the shadow areas of the bottom of deep holes and the backlight side of the protrusion, thus solving the problem of coexistence of overexposure in bright areas and loss of detail in dark areas when imaging complex curved surfaces with high reflectivity.

[0010] 3. This application continuously captures a sequence of images of the test area under different exposure parameters during the illumination distribution adjustment process, and performs feature alignment and semantic fusion on the sequence images through a two-stage neural network model. The feature alignment network is used to eliminate spatial misalignment between different exposure frames, and the semantic mask generated by the semantic fusion network is used to intelligently guide the pixel-level fusion weight allocation of each frame sequence image. This achieves a balance between bright texture details and dark contrast in high dynamic range scenes. The reconstructed high-fidelity defect texture image effectively preserves the texture boundary and shape features of physical defects such as micro-cracks and pores.

[0011] 4. This application achieves high-precision identification of multi-scale defects on the surface of parts by inputting high-fidelity defect texture images into a deep residual convolutional neural network detection model for detection and classification. Furthermore, by spatially associating the identified defect texture regions with spatial point cloud data, it enables quantitative measurement of defect depth and area, providing a process-executable numerical basis for surface defect judgment and directly driving the sorting execution mechanism to automatically reject unqualified parts. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the overall scheme of the online detection method for surface defects of automotive parts based on machine vision in this application; Figure 2 This is a schematic diagram of the core principle framework for performing optical attitude coordination control in this application; Figure 3 This is a flowchart illustrating the logical flow of multi-level semantic fusion performed by this application using a two-stage neural network model. Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow between the central controller, 3D sensor and industrial camera in this application; Figure 5 This is a flowchart of the online defect classification and recognition process performed by the deep residual convolutional neural network detection model in this application. Detailed Implementation

[0013] The following will be combined with the appendix Figures 1 to 5 The technical solution of the present invention is clearly and completely described below. Obviously, the following embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0014] In the online detection method for surface defects of automotive parts based on machine vision provided by this invention, the entire detection process is integrated into a high-precision industrial automated production line. Through deep collaboration between the hardware layer, driver layer, algorithm layer and application layer, sub-millimeter level defect capture is achieved for automotive parts with complex geometric shapes and high reflectivity.

[0015] In the existing technology, when performing surface defect detection on automotive parts with complex geometries, there is still a prominent problem: when traditional imaging systems acquire images of moving parts on high-speed production lines, the lack of a high-precision spatiotemporal synchronization mechanism makes it easy for motion blur to cause spatial distortion in the acquired point cloud data or image data. This will cause errors in the subsequent normal vector calculation based on the point cloud, and thus affect the accuracy of the entire optical attitude coordination control link.

[0016] Therefore, how to achieve "frozen" spatial sampling under high-speed motion of components and obtain high-fidelity three-dimensional point cloud data has become the primary prerequisite for achieving accurate normal vector calculation and subsequent adaptive optics control.

[0017] In the above method, step S1 involves synchronously triggering the acquisition of spatial point cloud data of the component surface. This data is used to generate a dense spatial point cloud of the area to be measured under high-speed motion of the component through high-precision time synchronization and structured light 3D measurement, providing a geometric reference for subsequent normal vector calculation and imaging attitude control. Specifically, this includes the following sub-steps: In step S101, the central controller, acting as an EtherCAT or Profinet master, establishes a real-time communication link with the 3D sensors acting as slaves by reading the slave device description files and configuring process data object mapping. The clock synchronization module inside the central controller is based on the IEEE 1588 protocol. It calculates the clock offset and transmission delay of each slave through periodic synchronization frame interaction, compensates the local clock of the slave, and enables all nodes on the bus to share the same time reference, with synchronization jitter controlled within 1μs.

[0018] In step S102, the digital input module of the central controller connects to the signal of the proximity switch or photoelectric encoder to form a workpiece arrival detection loop. When the part enters the station to be measured at a speed of 1m / s to 3m / s, the arrival signal triggers the clock synchronization module to generate a hard trigger pulse with a synchronization accuracy better than 1 microsecond. The hard trigger pulse is output in the form of RS-422 or LVDS differential signal. The equal amplitude and opposite phase voltage on the twisted pair cancels the common mode interference and is sent to the trigger input terminal of the three-dimensional sensor so that the three-dimensional sensor starts to collect data at a precise time.

[0019] The aforementioned S101 and S102 are synchronized with a precision clock through hard triggering, enabling "frozen" spatial sampling of moving parts on a high-speed production line, thus avoiding point cloud distortion caused by motion blur.

[0020] In step S103, the three-dimensional sensor adopts the principle of line laser triangulation measurement. The laser projector built into the three-dimensional sensor projects a structured light strip with a spectral line width controlled within 0.02mm onto the surface of the component. The laser wavelength is selected between 450nm and 650nm according to the absorption characteristics of the metal surface. The pixel size of the high-speed CMOS photosensitive device inside the three-dimensional sensor is 5μm to 7μm. The photosensitive surface resolution of the high-speed CMOS photosensitive device is not less than 2048×1024 pixels. The high-speed CMOS photosensitive device synchronously receives the light strip reflected back by the surface of the component. The degree of deformation of the light strip directly reflects the undulation of the surface.

[0021] Step S104: For each line of light stripe image acquired by the high-speed CMOS image sensor, the processor built into the 3D sensor, based on the principle of triangle similarity, determines the pixel offset of the light stripe on the photosensitive surface of the high-speed CMOS image sensor. Calculate the height of the corresponding point relative to the pre-calibrated reference plane. The calculation formula is: in, The working distance of the 3D sensor is the distance from the point where the optical axis of the laser projector intersects the reference plane along the optical axis to the laser projector. This is the triangular baseline distance, which is the distance between the light-emitting point of the laser projector and the optical center of the high-speed CMOS image sensor. This is a triangulation measurement angle, specifically the angle between the optical axis of the laser projector and the optical axis of the high-speed CMOS image sensor. , , All values ​​are factory calibrated and fixed in the firmware of the 3D sensor.

[0022] The three-dimensional sensor achieves a measurement repeatability of 0.005 mm in the Z-axis direction and sets the sampling interval in the horizontal XY plane to 0.05 mm. The X and Y coordinates of each spatial point cloud are obtained by trigonometric scaling based on the row and column numbers of the spatial point cloud on the photosensitive surface of the high-speed CMOS photosensitive device and the pixel size of the high-speed CMOS photosensitive device. This generates a high-density spatial point cloud, in which the coordinates of each point contain geometric information in the three dimensions of X, Y, and Z.

[0023] In step S105, the 3D sensor encapsulates the generated 3D point cloud coordinate set into a binary data stream. The encapsulation order is frame header, timestamp, number of points, X coordinate array, Y coordinate array, Z coordinate array and checksum. The binary data stream is transmitted in real time to the high-performance computing processing unit through a 10 Gigabit Ethernet interface conforming to the IEEE 802.3ae standard. After the 3D point cloud coordinate set enters the memory buffer, it is mapped to the application memory space through direct memory access, without going through the file system, thus controlling the delay of triggering the availability of the acquired data to the millisecond level.

[0024] The above-mentioned S103 to S105, with the help of sub-millimeter-level structured light projection and high-resolution CMOS reception, combined with the triangulation principle, reconstruct a dense spatial point cloud with measurement-level precision for complex-shaped metal surfaces, and completely preserve the spatial information of key features such as deep holes and protrusions.

[0025] The high-fidelity 3D point cloud data obtained in step S1 provides a solid geometric reference for the accurate calculation of the normal vector of the center position of the area to be tested in the subsequent step S2, thereby supporting the adaptive control of optical imaging attitude in step S3 and the fusion of multi-exposure image sequences in step S4, and finally realizing the online identification of surface defects of highly reflective complex parts.

[0026] In the above method, step S2 calculates the normal vector of the center position of the area to be measured. This vector is used to denoise and simplify the 3D point cloud coordinate set obtained in step S1, and the normal vector is extracted through local plane fitting to provide a geometric reference for subsequent optical imaging attitude adjustment and illumination parameter configuration. Specifically, it includes the following sub-steps: In step S201, after receiving the set of three-dimensional point cloud coordinates in the memory buffer, the high-performance computing processing unit starts the preprocessing thread. The preprocessing thread calls the statistical filter to process the set of three-dimensional point cloud coordinates point by point.

[0027] The statistical filter searches for the K nearest neighbors in the spatial neighborhood of each point cloud coordinate, where K is an integer value between 6 and 20. The average Euclidean distance between the point cloud coordinate and the K nearest neighbors is calculated as the local average distance. After traversing all point cloud coordinates in the three-dimensional point cloud coordinate set, the global average distance and standard deviation are obtained. When the local average distance of a point cloud coordinate is greater than the sum of the global average distance and 3 times the standard deviation, the statistical filter determines that the point cloud coordinate is an outlier and removes it from the three-dimensional point cloud coordinate set. Outliers are mainly caused by diffuse reflection from metal surfaces or ambient stray light.

[0028] In step S202, the high-performance computing processing unit performs geometric contour recognition on the three-dimensional point cloud coordinate set after removing outliers and performs voxel filtering downsampling. The voxel filtering divides the three-dimensional space where the three-dimensional point cloud coordinate set is located into regular cubic grids according to the preset cube side length. The centroid position is calculated for all point cloud coordinates contained in each cubic grid, and the centroid position represents all the original point cloud coordinates in this cubic grid. After voxel filtering, the point cloud density is reduced, and the computational complexity of subsequent fitting operations is reduced accordingly.

[0029] The above steps S201 and S202 clean and simplify the original three-dimensional point cloud coordinate set, reduce the point cloud density while retaining the surface geometric features of the area to be measured, and speed up the subsequent normal vector calculation.

[0030] In step S203, the high-performance computing processing unit performs least squares fitting on the point cloud distribution in the region to be measured in the three-dimensional point cloud coordinate set after voxel filtering, and constructs the infinitesimal plane equation.

[0031] The neighborhood search radius for local plane fitting is set to a preset value within the range of 3mm to 10mm. The covariance matrix is ​​calculated using the coordinates of all point clouds within the neighborhood search radius as input. covariance matrix The calculation formula is: in, It is the covariance matrix; This represents the total number of point cloud coordinates within the neighborhood search radius; For the search radius of the neighborhood, the first A column vector of point cloud coordinates, a column vector Including the The X, Y, and Z coordinates of a point cloud; Let be the column vector of the geometric centers of all point cloud coordinates within the neighborhood search radius. The X-coordinate component is the arithmetic mean of the X-coordinate values ​​of all point cloud coordinates within the neighborhood search radius; the Y-coordinate component is the arithmetic mean of the Y-coordinate values ​​of all point cloud coordinates within the neighborhood search radius; and the Z-coordinate component is the arithmetic mean of the Z-coordinate values ​​of all point cloud coordinates within the neighborhood search radius. (Superscript) This represents the matrix transpose operation. For the first The offset vector of each point cloud coordinate relative to the geometric center, and the covariance matrix. Characterizes the distribution characteristics of a local point set in three-dimensional space.

[0032] In step S204, the high-performance computing processing unit extracts the covariance matrix using the Jacobian rotation method or singular value decomposition method. The eigenvalues ​​are sorted by numerical value, and the eigenvector corresponding to the smallest eigenvalue is extracted. The extracted eigenvector is then normalized by dividing each component of the eigenvector by its magnitude, which is the square root of the sum of the squares of the eigenvector components. The resulting unit vector after normalization is the normal vector at the center of the region to be measured. normal vector The direction is perpendicular to the micro-element plane fitted in step S203.

[0033] The above steps S203 and S204 construct the covariance matrix and perform eigenvalue decomposition to stably extract the normal vector representing the surface orientation from the local point cloud. The direction corresponding to the smallest eigenvalue of the covariance matrix is ​​the direction in which the point cloud distribution is flattest, which is also the normal direction of the surface of the area to be measured. This overcomes the interference of residual noise in the point cloud data on the accuracy of normal vector estimation.

[0034] Step S205, the high-performance computing processing unit will process the normal vector. The spatial coordinates of the center position of the area to be measured are written into the shared memory region. The shared memory region is accessed by both the high-performance computing processing unit and the central controller. When the central controller performs the optical attitude coordination control in step S3, it reads the normal vector from the shared memory region. Spatial coordinates and normal vector of the center position of the region to be measured It serves as an attitude reference for the six-axis robotic arm to adjust the imaging axis direction of the industrial camera, and also as a geometric reference benchmark for constructing a model of non-uniform illumination distribution in the spatial domain.

[0035] Through step S2 above, the normal vector of the center position of the area to be measured is accurately calculated from the original three-dimensional point cloud coordinate set. normal vector This provides direct geometric input for the optimal imaging pose calculation of the industrial camera and the partitioned brightness adjustment of the multi-array light source system in step S3, enabling the optical imaging components to actively adjust their posture and illumination parameters according to the actual concave and convex directions of the component surface, thereby specifically suppressing specular reflections and shadow blind spots generated by highly reflective metal surfaces.

[0036] In the above method, step S3, performing optical pose coordination control, is used to synchronously adjust the imaging pose of the industrial camera and the illuminance distribution of the multi-array light source system based on the normal vector calculated in step S2, so that the imaging principal axis is aligned with the normal vector of the surface under test, and to suppress specular reflections and shadow blind spots on the metal surface. Specifically, it includes the following sub-steps: Step S301, the central controller reads the normal vector from the shared memory region. The spatial coordinates of the center position of the region to be measured are used to convert the normal vector into a coordinate vector using an inverse kinematics algorithm. The target posture is represented and converted into the angle increments of each joint of the six-axis robotic arm, generating motion control commands. The end-effector repeatability of the six-axis robotic arm is better than 0.02mm.

[0037] In step S302, the central controller establishes a 4×4 affine transformation model from the point cloud coordinate system to the robot base coordinate system. The affine transformation model includes a 3×3 rotation matrix R and a 3×1 translation vector t. The rotation matrix R is composed of the direction cosines of the three coordinate axes of the point cloud coordinate system in the robot base coordinate system, and the translation vector t is composed of the coordinates of the origin of the point cloud coordinate system in the robot base coordinate system. The central controller converts the direction of the normal vector n in the point cloud coordinate system into the target direction in the robot base coordinate system through the rotation matrix R, and converts the center position of the area to be measured into the target position in the robot base coordinate system through the translation vector t, thereby calculating the optimal imaging pose of the industrial camera.

[0038] In step S303, during the movement of the six-axis robotic arm carrying the industrial camera toward the optimal imaging pose, the robot controller reads the values ​​of the encoders of each joint in real time through a closed-loop feedback mechanism to monitor the actual pointing of the imaging main axis of the industrial camera. When the angle between the imaging main axis and the normal vector n deviates from the preset angle range, the robot controller drives the servo motor to perform angle compensation, controlling the angle between the imaging main axis and the normal vector n within ±2°, thereby reducing the geometric distortion caused by oblique shooting and providing a highly consistent original image for the image sequence acquisition in the subsequent step S4.

[0039] The above-mentioned S301 to S303, through inverse kinematics calculation, affine coordinate transformation and closed-loop feedback compensation, ensure that the optical axis of the industrial camera is always aligned with the normal direction of the surface to be measured, thereby eliminating perspective distortion caused by the undulation of the curved surface of the parts.

[0040] In step S304, while performing industrial camera pose adjustment, the central controller uses a predictive trajectory smoothing algorithm to pre-calculate the arrival time of the next part at the test station based on the real-time feedback of the conveyor belt speed from the production line encoder. The motion program of the six-axis robotic arm is started 100ms to 300ms before the arrival of the next part, compressing the time for the six-axis robotic arm to move from the end pose of the previous part to the starting pose of the next part into a predetermined time period, so that the idle movement of the six-axis robotic arm overlaps with the continuous flow of the production line.

[0041] In step S305, the central controller generates an illumination modulation command to drive the multi-array light source system. The multi-array light source system is arranged in a hemispherical or ring shape and integrates 128 to 256 independently controlled light-emitting diode units. Each light-emitting diode unit is independently driven by a pulse width modulation module at a modulation frequency of 50kHz or higher. The modulation frequency is higher than the reciprocal of the frame rate during high-speed exposure of an industrial camera to avoid flickering stripes during imaging.

[0042] The aforementioned S304 and S305 ensure the detection cycle and imaging environment from the two dimensions of time scheduling and lighting drive, respectively, so that the movement of the robotic arm does not become a bottleneck of the production line, and at the same time, the light source modulation frequency does not conflict with the camera exposure sequence.

[0043] Step S306: The central controller constructs a spatial non-uniform illuminance distribution model, specifically by calculating the angle between the principal axis of light emission of each LED unit and the normal vector n.

[0044] The main light-emitting axis of each LED unit is the spatial direction of the optical axis of the encapsulating lens, represented by a unit vector in the light source coordinate system. For each LED unit, the central controller transforms the main light-emitting axis vector through the transformation matrix from the light source coordinate system to the point cloud coordinate system, and then calculates the angle between the vector and the normal vector n.

[0045] Based on the cosine effect and distance attenuation law, it is determined whether each LED unit is in a mirror reflection path or a large-angle illumination path relative to the area under test. The light emitted by the LED unit in the mirror reflection path is reflected by the metal surface and directly enters the industrial camera lens, producing mirror reflection. The light emitted by the LED unit in the large-angle illumination path illuminates the surface of the area under test at a grazing angle, forming a shadow area at the bottom of the deep hole or on the raised backlight side.

[0046] Step S307: For the light-emitting diode unit located on the mirror reflection path, the spatial non-uniform illuminance distribution model reduces the pulse width modulation duty cycle of the corresponding light-emitting diode unit to 10% to 40% of the original duty cycle in order to suppress the specular reflection on the metal surface.

[0047] For LED units located on large-angle lighting paths, the spatial non-uniform illuminance distribution model increases the pulse width modulation duty cycle of the corresponding LED unit. The increase in pulse width modulation duty cycle is proportional to the cosine attenuation compensation of the included angle, in order to compensate for insufficient illuminance in the shadow areas at the bottom of deep holes or on the raised backlight side.

[0048] In step S308, the pulse width modulation module monitors the branch current of each LED unit in real time through the current sensor. When the current of a branch exceeds the preset normal range or an open circuit is detected, the pulse width modulation module determines that the corresponding LED unit is damaged. The central controller automatically reallocates the brightness weights of other adjacent normal LED units in the spatial non-uniform illuminance distribution model. The original brightness contribution of the damaged LED unit is weighted according to spatial proximity and distributed to the normal LED units in the same zone, so that the uniformity deviation of the lighting field is maintained within 5%.

[0049] The above-mentioned S306 to S308 construct a spatial non-uniform illuminance model based on normal vector feedback, perform zoned differential dimming on the multi-array light source system, actively reduce brightness in the specular reflection direction, actively supplement light in the shadow area, and have the adaptive compensation capability for damaged units, thereby obtaining uniform illumination conditions on complex morphological metal surfaces.

[0050] Through the above step S3, the normal vector calculated in step S2 is transformed in real time into a dual adjustment of the industrial camera's imaging posture and the illuminance distribution of the multi-array light source. This enables the optical imaging components to actively adapt to the geometric undulations and reflection characteristics of the component surface, suppressing the impact of specular reflection and shadow blind spots on image quality from the source, and creating controllable imaging conditions for step S4 to capture high dynamic range sequence images with different exposure characteristics.

[0051] Furthermore, although the aforementioned optical pose coordination control suppresses specular reflection and shadows at the source, the difference in reflectivity of different areas on the surface of the components means that images acquired under a single exposure parameter may still exhibit overexposure in bright areas or underexposure in dark areas. While the industry has seen the emergence of technologies that combine multi-exposure imaging with deep learning for defect detection, in actual industrial environments, slight vibrations of robotic arms or positional deviations of components on conveyor belts can cause pixel-level misalignment between different exposure frames. Existing methods often lack specialized processing for spatial alignment of multi-exposure sequence images. At the same time, general image synthesis methods struggle to intelligently distinguish between bright and dark areas and differentiate the fusion weights when fusing different exposure frames, easily leading to the loss of texture details in the fused image. This is especially problematic in high dynamic range scenarios, where the contradiction between overexposure and saturation in bright areas and insufficient contrast in dark areas is difficult to resolve.

[0052] Therefore, how to perform adaptive semantic fusion on multi-exposure sequence images while ensuring spatial alignment accuracy in order to reconstruct high-fidelity defect texture images and thus improve the accuracy of subsequent defect classification is a key challenge.

[0053] In the above method, step S4, image sequence acquisition and multi-level semantic fusion, is used to capture a sequence of images of the same test area under different exposure parameters under the controllable imaging conditions established in step S3. High-fidelity defect texture images are then reconstructed through feature alignment and semantic fusion, eliminating the interference of overexposed and underexposed areas of highly reflective surfaces on defect feature extraction. Specifically, it includes the following sub-steps: In step S401, the industrial camera continuously captures 3 to 5 frames of original sequence images during the process of switching illuminance parameters according to the spatial non-uniform illuminance distribution model of the multi-array light source system.

[0054] Each frame of the original image sequence corresponds to a different set of exposure times, which are dynamically adjusted in real time according to the reflectivity of the surface material of the area to be tested.

[0055] For areas where the material reflectivity is higher than the preset high reflectivity threshold, the exposure time is adjusted to a lower value to preserve the bright texture. For areas where the material reflectivity is lower than the preset low reflectivity threshold, the exposure time is adjusted to a higher value to improve the contrast of the dark areas.

[0056] In step S402, the industrial camera transmits the original sequence images to the high-performance computing processing unit via Camera Link or CoaXPress high-speed interface. After the original sequence images enter the graphics processor memory of the high-performance computing processing unit, they are directly loaded into the computation graph of the preset two-stage neural network model.

[0057] Step S403: The two-stage neural network model is built on a deep learning framework. The network structure of the two-stage neural network model includes a first-stage feature alignment network and a second-stage semantic fusion network. The output feature map of the feature alignment network serves as the input of the semantic fusion network. The two-stage neural network model runs faster in forward inference mode on a graphics processor.

[0058] The above-mentioned S401 to S403, through multi-exposure sequence acquisition and high-speed data transmission, send the complete brightness information of the same test area under different exposure conditions into the two-stage neural network model, providing input data for subsequent feature alignment and semantic fusion.

[0059] Step S404: In the first stage, the feature alignment network adopts a feature transformation algorithm based on optical flow estimation.

[0060] For two adjacent frames of original sequence images, the feature alignment network extracts the feature map of each frame of the original sequence image through convolutional layers, and calculates the motion vector at each pixel position between the two feature maps. The motion vector reflects the pixel displacement caused by the slight vibration of the robotic arm or the slight deviation of the position of the parts on the conveyor belt.

[0061] The feature alignment network constructs a 3×3 homography transformation matrix H based on all pixel motion vectors. The homography transformation matrix H aligns the original sequence images under different exposures in spatial pixels.

[0062] The homography transformation matrix H is constructed as follows: four sets of matching point pairs are randomly sampled from all pixel motion vectors, the eight degrees of freedom parameters of the homography transformation matrix H are solved using the direct linear transformation method, and the final homography transformation matrix H is obtained after removing mismatched point pairs through the random sampling consistency algorithm.

[0063] The formula for alignment transformation is: in, This represents the pixel value at pixel coordinates (x, y) in the aligned image. H represents the original sequence image before alignment, and H represents the 3×3 homography transformation matrix. Represents a homogeneous column vector of pixel coordinates (x, y). This indicates the pixel position in the aligned image obtained after spatial mapping of pixel coordinates through a homography transformation matrix. The superscript T indicates the matrix transpose operation.

[0064] The above-mentioned S404 eliminates spatial misalignment between multi-exposure image sequences through optical flow estimation and homography transformation, ensuring that subsequent semantic fusion is performed under pixel-level spatial consistency.

[0065] In step S405, in the second stage, the semantic fusion network extracts the statistical features of the high illumination region and the low illumination region in each frame of the aligned image through a lightweight convolutional layer.

[0066] The statistical characteristics of overexposed areas are the average brightness and pixel percentage of areas with pixel values ​​greater than the preset overexposure threshold of 240 in the aligned image. The statistical characteristics of underexposed areas are the average darkness and pixel percentage of areas with pixel values ​​less than the preset underexposure threshold of 15 in the aligned image.

[0067] The semantic fusion network generates semantic masks for bright and dark regions based on two types of statistical features. The semantic mask for bright regions marks the locations of overexposed and saturated pixels in the image, while the semantic mask for dark regions marks the locations of underexposed pixels that cause texture loss.

[0068] In step S406, the semantic fusion network merges the semantic mask of the bright area and the semantic mask of the dark area into a bidirectional semantic mask.

[0069] Bidirectional semantic masking is used to guide the pixel weight allocation between sequential images. It reduces the fusion weight of corresponding pixels in low-exposure frames in bright areas and increases the fusion weight of corresponding pixels in normal or high-exposure frames, so that the fused high-fidelity defect texture image retains more than 95% of the texture details in bright areas without saturation.

[0070] By reducing the fusion weight of corresponding pixels in high-exposure frames and increasing the fusion weight of corresponding pixels in normal or low-exposure frames in dark areas, the contrast of dark areas can be improved by more than 200%.

[0071] The formula for calculating the fusion weight is: in, Indicates the first The fusion weights of the frame image at pixel coordinates (x, y). The bidirectional semantic mask represents the first... The mask value at pixel coordinates (x, y) of the frame image, where N represents the total number of frames in the image sequence. To prevent the smallest constant with a denominator of zero from being taken as 10 -6 .

[0072] The above-mentioned S405 and S406 adaptively select the best exposure information of each pixel position in the multi-exposure sequence image for fusion by using pixel-level weight allocation based on semantic mask, avoiding the loss of brightness and darkness areas caused by manually setting fusion parameters in traditional high dynamic range synthesis methods.

[0073] Through step S4 above, the multi-exposure sequence image obtained by optical pose coordination control in step S3 is transformed into a high-fidelity defect texture image without overexposure or underexposure. The high-fidelity defect texture image eliminates the destruction of the defect texture integrity by the high reflectivity of the metal surface, making the texture boundary and shape features of physical defects such as cracks and pores clearly distinguishable, providing high-quality texture input for the high-precision defect classification and recognition of the deep residual convolutional neural network detection model in step S5.

[0074] In the above method, step S5 involves performing online defect classification and recognition. This is used to perform multi-scale defect detection and classification on the high-fidelity defect texture image output in step S4, and triggers re-inspection for low-confidence results through a confidence assessment mechanism. Finally, a detection report and quality rating information are output. Specifically, this includes the following sub-steps: Step S501: The deep residual convolutional neural network detection model is pre-trained in the automotive parts defect database. The automotive parts defect database contains more than 500,000 labeled samples. The labeled samples cover real defects generated in different process stages such as casting, processing, and transportation. Each labeled sample is marked with the category label and bounding box position coordinates of at least one type of defect, such as cracks, pores, sand holes, or scratches.

[0075] Step S502: The deep residual convolutional neural network detection model adopts a residual structure of 101 or 152 layers. In the residual structure, each residual block adds the input feature map to the output feature map after convolution transformation through a skip connection, so that the gradient can be directly backpropagated to the shallow network along the skip connection during backpropagation.

[0076] The deep residual convolutional neural network detection model internally constructs a feature pyramid structure, which generates feature maps from layers P2 to P5 by lateral connection and top-down fusion of feature maps at different depths of the backbone network. The feature map of layer P2 has the highest resolution and is used to detect tiny defects smaller than 0.1 mm, while the feature map of layer P5 has the lowest resolution and is used to detect larger defects larger than 1 mm.

[0077] For each spatial location on the feature maps of layers P2 to P5, multiple anchor boxes with different scales and aspect ratios are preset. The scale of the anchor boxes increases progressively from 16×16 pixels in layer P2 to 256×256 pixels in layer P5, with aspect ratios of 1:1, 1:2, and 2:1. The deep residual convolutional neural network detection model performs defect category prediction and bounding box regression on all the above anchor boxes, enabling concurrent detection of defects at different scales.

[0078] The aforementioned S501 and S502, through large-scale labeled data training and a multi-scale feature pyramid structure, enable the deep residual convolutional neural network detection model to cover extremely small targets such as micro-cracks and slender targets such as scratches.

[0079] In step S503, the high-fidelity defect texture image output in step S4 is input into the trained deep residual convolutional neural network detection model. The deep residual convolutional neural network detection model extracts the physical defect features of the component surface in the high-fidelity defect texture image, identifies and locates cracks, pores, sand holes and scratches. For micro-cracks with a size greater than or equal to 0.05 mm, the recognition accuracy reaches more than 99.5%.

[0080] Step S504: While outputting the classification and bounding box positions, the deep residual convolutional neural network detection model calculates a confidence score for each detection result using a Softmax layer. The calculation formula for the Softmax layer is as follows: in, This represents the probability that the detection result belongs to the j-th type of defect. This represents the raw score of the j-th type of defect output by the fully connected layer. This represents the original score of the c-th type of defect output by the fully connected layer, where M represents the total number of defect categories, e is a natural constant, and the maximum value among all M probability values ​​is taken as the confidence score.

[0081] In step S505, when the confidence score of a certain detection result is lower than 0.85, the central controller automatically triggers a re-shooting command. The industrial camera is equipped with an electric zoom lens. The central controller sends a switching command to the electric zoom lens through the lens control interface to switch the lens focal length to the macro lens focal length. The suspected area corresponding to the detection result is locally re-examined with a spatial resolution higher than that of the normal shooting mode. The re-examined image is then sent to the deep residual convolutional neural network detection model for secondary recognition.

[0082] In step S506, the central controller summarizes all detection results with a confidence score of not less than 0.85 and the results confirmed after re-inspection, and generates an online detection report. The online detection report includes the defect category, defect location, defect size and confidence score. At the same time, it outputs the quality rating information of the corresponding parts according to the preset quality rating rules, and the false alarm rate is controlled below 0.1%.

[0083] The above-mentioned S503 to S506 combine high-precision defect identification with low false alarm rate quality judgment through multi-scale detection of deep residual networks, Softmax confidence assessment and low-score re-inspection mechanism, ensuring that only detection results with confidence meeting the threshold requirements are included in the final report.

[0084] Through step S5 above, the high-fidelity defect texture image reconstructed in step S4 is transformed into structured defect information with category labels, spatial location and confidence scores. The online detection report and quality rating information can directly drive the subsequent sorting execution mechanism to remove unqualified parts, and provide defect detection records for each part to the quality traceability database of the central monitoring and management platform.

[0085] Furthermore, existing surface defect detection systems often stop at outputting the defect type and location in a two-dimensional image. However, in actual automotive parts production quality control, determining whether a defect leads to part scrap depends not only on the type of defect but also on whether its physical dimensions exceed the tolerance range allowed by the process. Simple two-dimensional detection cannot provide key three-dimensional quantitative information such as defect depth, area, or volume, resulting in a lack of accurate numerical basis for sorting decisions.

[0086] Therefore, the key is to accurately correlate the two-dimensional defect textures identified in the aforementioned steps with the high-precision three-dimensional point cloud obtained in step S1, thereby advancing defect analysis from two-dimensional qualitative identification to three-dimensional quantitative measurement, and driving automatic sorting based on quantification parameters to construct a complete online detection closed loop.

[0087] In the above method, step S6, defect 3D morphology reconstruction and quantization measurement, is used to spatially correlate the defect texture region identified in step S5 with the 3D point cloud obtained in step S1. This involves mapping pixels to spatial points through coordinate transformation and timestamp alignment, extracting the depth, area, and volume parameters of the defect, and driving the sorting mechanism to remove unqualified parts based on a preset threshold. Specifically, it includes the following sub-steps: In step S601, when the central controller triggers the acquisition of data from the 3D sensor and the industrial camera, it marks each frame of point cloud data and each frame of image data with the same hardware trigger sequence number as a timestamp.

[0088] The hardware trigger sequence number is synchronously written by the clock synchronization module when generating the hard trigger pulse. The hardware trigger sequence number is an auto-incrementing count value. The count value is incremented by 1 for each hard trigger pulse generated. The count value is reset to zero and starts counting again after reaching the preset upper limit.

[0089] The central controller appends hardware trigger sequence numbers to the headers of the 3D point cloud data frames and the industrial camera image data frames, respectively. The high-performance computing processing unit at the receiving end establishes a high degree of synchronization between the point cloud data and the image data in terms of timestamps by matching the hardware trigger sequence numbers.

[0090] Step S602: Construct a spatial coordinate transformation matrix using the pre-calibrated industrial camera intrinsic parameter matrix K and the 4×4 extrinsic parameter transformation matrix T from the industrial camera coordinate system to the three-dimensional sensor coordinate system.

[0091] The intrinsic parameter matrix K of the industrial camera is a 3×3 matrix that includes the focal length. , and principal point coordinates , Each parameter is in pixels, and the extrinsic transformation matrix T contains rotation and translation components, which are obtained through hand-eye calibration.

[0092] For a pixel coordinate u,v within the defect bounding box, the depth estimate d is obtained from the corresponding position in the 3D point cloud generated in step S1. The pixel coordinate u,v is back-projected onto the 3D point in the industrial camera coordinate system using the industrial camera intrinsic parameter matrix K and the depth estimate d. Then, it is mapped to the 3D sensor coordinate system through the extrinsic parameter transformation matrix T. The direction vector from the spatial position of the industrial camera optical center in the 3D sensor coordinate system to the mapped 3D point is the spatial ray direction of the corresponding pixel.

[0093] Step S603: Perform a nearest neighbor search on the spatial ray and the set of three-dimensional point cloud coordinates.

[0094] Centered on the mapped 3D point at a distance d from the optical center of the industrial camera along the spatial ray direction, search for neighboring points in the 3D point cloud coordinate set within a search tolerance range of 0.1mm to 0.15mm. The search tolerance is 2 to 3 times the lateral sampling interval of the 3D sensor.

[0095] The three-dimensional spatial point with the shortest perpendicular distance to the spatial ray direction within the search tolerance range is determined as the corresponding spatial point of the corresponding pixel. After traversing all pixels within the defect bounding box, a subset of the defect three-dimensional point cloud corresponding one-to-one with the defect texture region is obtained.

[0096] The above-mentioned S601 to S603 achieve pixel-level precise correlation between two-dimensional defect images and three-dimensional point clouds through hardware timestamp synchronization and calibration matrix mapping, providing a data foundation for the measurement of defect geometric parameters.

[0097] Step S604: Calculate the geometric parameters of the defect 3D point cloud subset.

[0098] Using the normal surface point cloud within a 1mm to 2mm neighborhood extending outward from the defect edge as input, a random sampling consensus algorithm is used to fit the reference plane. The random sampling consensus algorithm randomly selects 3 points from the neighborhood point cloud to calculate the plane equation, counts the distances from the remaining points in the neighborhood to the plane equation, and points with a distance less than a preset distance threshold are identified as interior points. The plane model with the most interior points is iteratively selected as the reference plane.

[0099] The defect depth is the maximum value of the vertical distance from each point in the defect point cloud subset to the reference plane.

[0100] The defect area is calculated by projecting each point in the defect point cloud subset onto the reference plane along the normal vector direction of the reference plane to obtain the projection point set. The two-dimensional convex hull of the projection point set is then calculated, and the area of ​​the two-dimensional convex hull is the defect area.

[0101] The defect volume is calculated by performing Delaunay triangulation on a subset of the defect point cloud to generate a set of triangular facets. For each triangular facet, the volume of the triangular prism enclosed by the triangular facet and the reference plane is calculated. The total volume of the triangular prism is then summed to obtain the defect volume.

[0102] Step S605: When the calculated defect depth exceeds 0.2 mm, or the defect area exceeds 0.5 mm... 2 At this time, the central controller sends a rejection instruction to the sorting execution mechanism, which pushes the defective parts from the current assembly line station or puts them into the scrap area by means of cylinder pushing or robotic arm grabbing.

[0103] The above-mentioned S604 and S605 quantify the three-dimensional features of defects by fitting the reference plane and calculating geometric parameters, and compare them with the preset process threshold to realize automatic sorting decision.

[0104] Through step S6 above, the two-dimensional defect detection results obtained in step S5 are upgraded to three-dimensional quantitative information, so that the determination of the depth and area of ​​surface defects has a numerical basis that can be executed by the process, and directly triggers the physical sorting action of the production line.

[0105] This embodiment also provides an online inspection system for surface defects of automotive parts based on machine vision, which includes a central monitoring and management platform.

[0106] The central monitoring and management platform communicates with the central controller, sorting actuators and display terminals via EtherNet / IP or Profinet industrial Ethernet protocol. It reads detection statistics, defect distribution heatmaps and production line operating status at a fixed refresh cycle and renders them in real time on the display terminal.

[0107] The central monitoring and management platform records a unique identification code for each component, generated by laser marking. The identification code adopts the Data Matrix or QR code format and is marked at the loading station before the component enters the inspection station.

[0108] The central monitoring and management platform writes the online inspection report generated in step S506, along with the timestamp and identification code, into a MySQL or PostgreSQL relational database. The data table establishes a batch number field, a process parameter field, and a defect image storage path field, forming a complete quality traceability file covering the batch number, process parameters, and defect images.

[0109] The system is also equipped with a remote maintenance module, which establishes a secure communication link between the diagnostic terminal in the R&D center and the local network on the production line through an industrial gateway that supports IPsec or TLS encrypted tunnels.

[0110] Researchers can remotely access the operation logs recorded by the central monitoring and management platform through the diagnostic terminal. The operation logs include the execution timestamps of each step, abnormal event records, and hardware status parameters.

[0111] Researchers can view historical curves of imaging parameters through a diagnostic terminal. These parameters include the PWM duty cycle of each LED unit in the multi-array light source system, the exposure time of the industrial camera, and the gain value.

[0112] Researchers push the updated neural network weight file to the high-performance computing unit via an encrypted tunnel through the diagnostic terminal. After receiving the neural network weight file, the high-performance computing unit completes the hot loading of weights for the two-stage neural network model and the deep residual convolutional neural network detection model in the next detection interval, realizing continuous iterative optimization of the algorithm model.

[0113] To improve imaging quality, a pneumatic cleaning device is integrated into the end effector of the six-axis robotic arm.

[0114] Before the 3D sensor performs the structured light scanning in step S103, the central controller sends an opening command to the solenoid valve of the pneumatic cleaning device. The pneumatic cleaning device sprays clean compressed air at a pressure of 0.6MPa onto the surface of the component to be tested for a duration of 200ms to 500ms to remove residual cutting fluid, metal shavings or dust from the surface. After the spraying is completed, the 3D sensor starts the structured light scanning.

[0115] The multi-array light source system adopts a multi-wavelength composite light source, integrating a red light-emitting diode unit with a center wavelength of 630nm, a green light-emitting diode unit with a center wavelength of 525nm, and a blue light-emitting diode unit with a center wavelength of 465nm.

[0116] The central controller automatically switches the lighting mode according to the material properties of the components. For cast iron, it switches to a single-wavelength blue light lighting mode with a center wavelength of 450nm to 470nm. For aluminum alloy, it switches to a three-color white light mode composed of red, green, and blue light. For chrome-plated parts, it switches to a single-wavelength red light lighting mode with a center wavelength of 620nm to 640nm. By utilizing the differences in reflectivity and scattering characteristics of different metal surfaces at specific wavelengths, the contrast between defect textures and background areas is enhanced.

[0117] The online detection system for surface defects of automotive parts based on machine vision provided by the present invention includes a central controller, a three-dimensional sensor, a six-axis robotic arm, an industrial camera, a multi-array light source system, and a high-performance computing processing unit.

[0118] The central controller, as the control core of the system, is responsible for performing optical attitude coordination control. The central controller is communicatively connected to the high-performance computing processing unit and obtains the normal vector of the center position of the measured area from the high-performance computing processing unit. The central controller is electrically connected to the six-axis robotic arm and controls the six-axis robotic arm to adjust the imaging pose of the industrial camera according to the normal vector of the center position of the measured area, so that the imaging main axis of the industrial camera is aligned with the normal vector of the center position of the measured area. The central controller is electrically connected to the multi-array light source system and controls the multi-array light source system to construct a spatial non-uniform illuminance distribution model and perform differentiated brightness adjustment on each light-emitting diode unit in the multi-array light source system.

[0119] The three-dimensional sensor is used to project structured light stripes onto the surface of the component while it is in motion, and to receive the deformation image of the light stripes reflected back from the surface of the component. Based on the deformation image of the light stripes, spatial point cloud data is generated and transmitted to a high-performance computing processing unit.

[0120] An industrial camera is mounted on the end effector of the six-axis robotic arm. The six-axis robotic arm receives motion control commands from the central controller and moves the industrial camera to the target imaging pose.

[0121] Industrial cameras are used to continuously capture a sequence of images of the area under test under different exposure parameters during the illuminance distribution adjustment process of a multi-array light source system, and then transmit the sequence of images to a high-performance computing processing unit.

[0122] The multi-array light source system is arranged in a hemispherical or ring shape, and integrates multiple independently controlled light-emitting diode units. Each light-emitting diode unit is independently driven by a pulse width modulation module. The multi-array light source system receives lighting modulation commands from the central controller and performs differentiated brightness adjustment.

[0123] The high-performance computing processing unit is used to perform statistical filtering, voxel filtering downsampling, and local plane fitting on the spatial point cloud data acquired by the 3D sensor to calculate the normal vector of the center position of the area to be measured; to perform feature alignment and semantic fusion on the sequence images captured by the industrial camera using a two-stage neural network model to reconstruct high-fidelity defect texture images; and to use a deep residual convolutional neural network detection model to detect and classify surface defects of parts and output the detection results.

[0124] The online inspection system for surface defects of automotive parts based on machine vision provided in this embodiment works in a manner corresponding to steps S1 to S6 in the above method embodiment. The central controller and the high-performance computing processing unit exchange data via industrial Ethernet. The central controller and the three-dimensional sensor and industrial camera achieve time synchronization through hard trigger signals and high-speed data interfaces, which will not be elaborated here.

[0125] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0126] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A machine vision-based online detection method for surface defects in automotive parts, characterized in that, include: Spatial point cloud data of component surfaces are acquired through structured light 3D measurement while the components are in motion. Local plane fitting is performed on the point cloud within the test area in the spatial point cloud data to extract the normal vector at the center position of the test area; Based on the normal vector of the center position of the area to be measured, the imaging pose of the industrial camera and the illuminance distribution of the multi-array light source system are adjusted synchronously, including: controlling the movement of the industrial camera so that the imaging principal axis of the industrial camera is aligned with the normal vector of the center position of the area to be measured. Furthermore, a spatial non-uniform illuminance distribution model is constructed. For each light-emitting diode unit in the multi-array light source system, based on the angle between the light-emitting principal axis direction of the light-emitting diode unit and the normal vector of the center position of the area to be measured, the light-emitting diode unit is determined to be in a mirror reflection path or a large tilt angle illumination path. Differentiated brightness adjustment is performed on the light-emitting diode units determined to be in different paths to suppress mirror reflection on the surface of the component and compensate for insufficient illuminance in the shadow area. During the illuminance distribution adjustment process of the multi-array light source system, a sequence of images of the test area under different exposure parameters are continuously captured, and a high-fidelity defect texture image is reconstructed by performing feature alignment and semantic fusion on the sequence of images through a two-stage neural network model. The high-fidelity defect texture image is input into a deep residual convolutional neural network detection model to detect and classify surface defects of parts and output the detection results.

2. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The step of performing local plane fitting on the point cloud within the test area in the spatial point cloud data and extracting the normal vector at the center position of the test area includes: The high-performance computing processing unit sequentially performs statistical filtering and voxel filtering downsampling processing on the spatial point cloud data; For the set of three-dimensional point cloud coordinates after voxel filtering downsampling, the covariance matrix of the point cloud coordinates in the neighborhood is calculated with the center of the region to be measured as the neighborhood. The eigenvalues ​​of the covariance matrix are extracted using the Jacobi rotation method or singular value decomposition method. The eigenvectors corresponding to the smallest eigenvalues ​​are then normalized to obtain the normal vector at the center of the region to be measured. The direction of the normal vector at the center of the region to be measured is perpendicular to the fitted infinitesimal plane.

3. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The differential brightness adjustment of the light-emitting diode units located on different paths includes: For LED units determined to be on the mirror reflection path, the pulse width modulation duty cycle of the LED units determined to be on the mirror reflection path is reduced to suppress mirror reflection on the surface of the component. For LED units determined to be on a large-angle lighting path, increase the pulse width modulation duty cycle of the LED units determined to be on a large-angle lighting path to increase the illuminance of the shadow area at the bottom of the deep hole or on the raised backlight side.

4. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The step of reconstructing a high-fidelity defect texture image by performing feature alignment and semantic fusion on the sequence of images using a two-stage neural network model includes: In the first stage, the pixel motion vectors between adjacent frames in the sequence of images are calculated through a feature alignment network, and a homography transformation matrix is ​​constructed based on the pixel motion vectors. The homography transformation matrix is ​​then used to align the original sequence of images under different exposure parameters in terms of spatial pixels. In the second stage, a two-way semantic mask is generated by a semantic fusion network to mark overexposed saturated areas and underexposed texture loss areas in the image. The two-way semantic mask is then used to guide the fusion weight allocation of each pixel in each frame sequence image. The pixel information of different exposure frames is weighted and fused to obtain the high-fidelity defect texture image.

5. The online detection method for surface defects of automotive parts based on machine vision according to claim 4, characterized in that, The homography transformation matrix is ​​constructed as follows: Multiple sets of matching point pairs are randomly sampled from all pixel motion vectors. The degree of freedom parameters of the homography transformation matrix are solved using the direct linear transformation method. Finally, the homography transformation matrix is ​​obtained by eliminating mismatched point pairs using the random sampling consensus algorithm.

6. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The step of controlling the movement of the industrial camera to align the imaging principal axis of the industrial camera with the normal vector of the center position of the area to be measured includes: An affine transformation model is established from the point cloud coordinate system to the robot base coordinate system. The affine transformation model includes a rotation matrix and a translation vector. The normal vector of the center position of the region to be measured in the point cloud coordinate system and the center position of the region to be measured in the robot base coordinate system are transformed through the affine transformation model. The inverse kinematics algorithm is used to solve the target pose represented by the normal vector of the center position of the transformed test area into the angle increment of each joint of the six-axis robot arm, and to generate motion control commands to drive the six-axis robot arm to move the industrial camera to the target pose. During the movement, the values ​​of the encoders of each joint of the six-axis robotic arm are read in real time through a closed-loop feedback mechanism to monitor the actual pointing of the imaging spindle of the industrial camera. When the angle between the imaging spindle of the industrial camera and the normal vector of the center position of the area to be measured deviates from the preset angle range, the robot servo motor is driven to perform angle compensation.

7. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The process of constructing the spatial non-uniform illumination distribution model also includes: The central controller monitors the branch current of each light-emitting diode unit in the multi-array light source system in real time through a current sensor; When a damaged LED unit is detected, the original brightness contribution of the damaged LED unit is automatically distributed to the LED units in normal working condition in the same area according to spatial proximity, so as to maintain the uniformity of the lighting field.

8. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, The deep residual convolutional neural network detection model internally constructs a feature pyramid structure. The feature pyramid structure generates P2 layer feature maps, P3 layer feature maps, P4 layer feature maps, and P5 layer feature maps by lateral connection and top-down fusion of feature maps at different depths of the backbone network. The P2 layer feature map is used to detect tiny defects with a size smaller than a preset value, and the P5 layer feature map is used to detect larger defects with a size larger than the preset value, so as to achieve concurrent detection of defects of different scales.

9. The online detection method for surface defects of automotive parts based on machine vision according to claim 1, characterized in that, After outputting the detection results, it also includes: The identified defect texture region is spatially associated with the spatial point cloud data. Through hardware timestamp synchronization and a pre-calibrated spatial coordinate transformation matrix, the two-dimensional pixels within the defect bounding box are mapped to three-dimensional space to obtain a defect three-dimensional point cloud subset corresponding to the defect texture region. The random sampling consensus algorithm is used to fit the reference plane with the normal surface point cloud in the neighborhood of the defect edge as input. Project each point in the defect 3D point cloud subset onto the reference plane along the normal vector direction of the reference plane, calculate the 2D convex hull area of ​​the projected point set as the defect area, and triangulate the defect 3D point cloud subset to calculate the sum of the volumes of the triangular facets and the reference plane as the defect volume. When the defect depth or the defect area exceeds a preset process threshold, a rejection instruction is sent to the sorting execution mechanism.

10. An online inspection system for surface defects of automotive parts based on machine vision, characterized in that, For implementing the machine vision-based online detection method for surface defects of automotive parts as described in any one of claims 1 to 9, the system includes a central controller, a three-dimensional sensor, a six-axis robotic arm, an industrial camera, a multi-array light source system, and a high-performance computing processing unit; wherein... The central controller is used to perform optical pose coordination control, and according to the normal vector of the center position of the area to be measured calculated by the high-performance computing processing unit, control the six-axis robotic arm to adjust the imaging pose of the industrial camera so that the imaging main axis of the industrial camera is aligned with the normal vector of the center position of the area to be measured; and control the multi-array light source system to construct a spatial non-uniform illuminance distribution model and perform differentiated brightness adjustment. The high-performance computing processing unit is used to process the spatial point cloud data acquired by the three-dimensional sensor to calculate the normal vector of the center position of the area to be measured, to perform feature alignment and semantic fusion of the sequence images captured by the industrial camera using a two-stage neural network model, and to detect and classify surface defects of parts using a deep residual convolutional neural network detection model.

Images