A valve plate defect automatic detection method and system based on three-dimensional vision

Abstract

The application discloses a kind of valve piece defect automatic detection method and system based on three-dimensional vision, belong to machine vision and precision detection technical field.The method includes: through intelligent sensing system sequentially obtains original three-dimensional point cloud and multi-view polarization sub-image sequence, and calculates polarization angle image;Original three-dimensional point cloud is preprocessed, and valve piece standard point cloud is obtained;Valve piece standard point cloud is input into cross-modal pre-fusion network with polarization angle image, with the direction relationship deviation of three-dimensional normal vector projection direction and polarization ellipse major axis direction as gate signal, to generate cross-modal fusion feature map;Cross-modal fusion feature map is input into defect segmentation network, and pixel-level defect segmentation mask is output;After three-dimensional points in defect area are excluded, the measured value of flatness is calculated by the reference plane of iterative fitting of robust least square method.The application can simultaneously realize the high-precision segmentation of multiple defects on valve piece surface and the accurate evaluation of flatness, meet the requirement of online detection rhythm.

Description

Technical Field

[0001] This invention relates to the field of machine vision and precision inspection technology, specifically to an automatic detection method and system for valve plate defects based on three-dimensional vision. Background Technology

[0002] Valve plates are critical components in compressors, and their surface quality and flatness directly affect the compressor's energy efficiency and service life. During mass production of valve plates, each plate must undergo surface defect inspection and flatness assessment to ensure product consistency. Currently, commonly used valve plate inspection methods include manual visual inspection, traditional two-dimensional machine vision inspection, and contact or laser scanning coordinate measuring machines.

[0003] Manual visual inspection relies on operator experience, is labor-intensive, and lacks consistency in judging minute cracks and shallow pits. Traditional two-dimensional machine vision solutions segment defects based on grayscale or color images, and have some detection capability for texture defects such as scratches and edge chipping, but are not sensitive to depth information. The detection rate for shallow pits and subsurface microcracks caused by residual stamping stress needs improvement. Furthermore, these solutions typically cannot directly obtain flatness data, requiring the introduction of additional flatness measurement equipment, resulting in a fragmented inspection process and low efficiency.

[0004] Line laser contour scanning acquires three-dimensional topography through line-by-line scanning, providing depth information and outperforming two-dimensional methods in pit detection and flatness calculation. However, the trade-off between its scanning speed and inspection cycle time is significant, making it difficult to match the production cycle time of online full inspection. Furthermore, vibrations of the motion platform during scanning and errors in splicing multiple contour segments introduce measurement uncertainties, negatively impacting the repeatability accuracy of flatness measurements.

[0005] Therefore, there is a need in this field for a technical solution that can simultaneously complete high-precision segmentation of valve plate surface defects and accurate flatness measurement at a single inspection station, while meeting the requirements of online inspection cycle time. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic valve plate defect detection method and system based on three-dimensional vision, which can simultaneously achieve high-precision segmentation of multiple types of defects on the valve plate surface and accurate assessment of flatness, meeting the requirements of online detection cycle time.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An automatic defect detection method for valve plates based on three-dimensional vision includes: Step 1: Collect the original 3D point cloud of the valve plate under test based on the intelligent sensing system; Step 2: Preprocess the original 3D point cloud to output the standard point cloud of the valve plate; Step 3: Acquire auxiliary image sequences of the valve plate under test from various perspectives based on the intelligent sensing system, and calculate the polarization angle image corresponding to each perspective based on the Stokes parametric method. Step 4: Input the standard point cloud of the valve plate and the polarization angle images of all views into the cross-modal pre-fusion network, project each three-dimensional point in the standard point cloud of the valve plate onto the image plane of each polarization angle, extract the corresponding polarization angle value, and use the deviation of the direction relationship between the projection direction of the three-dimensional point normal vector in the image plane and the direction of the principal axis of the polarization ellipse as the gating signal, and concatenate the polarization angle value with the three-dimensional point normal vector to form a cross-modal fusion feature map; Step 5: Input the cross-modal fused feature map into the defect segmentation network and output a pixel-level segmentation mask for the valve plate surface defects; Step 6: Fit a reference plane using the standard point cloud of the valve plate. For the effective points in the standard point cloud of the valve plate that are not located within the defect segmentation mask, use the robust least squares method to iteratively solve the plane equation. Use this plane as the ideal reference plane, calculate the directed distance from all effective points to the ideal reference plane, and use the sum of the absolute values ​​of the maximum positive deviation and the maximum negative deviation as the measured flatness value.

[0008] Furthermore, in step 3, the auxiliary image sequence includes four polariton images.

[0009] Further, in step 3, the polarization angle images corresponding to each viewpoint are calculated based on the Stokes parametric method, specifically as follows: For any viewing angle, let the gray values ​​of the polarizer image at the pixel location corresponding to its four transmission azimuth angles be the first gray value, the second gray value, the third gray value, and the fourth gray value, respectively. Calculate the first Stokes parameter as the difference between the first and third gray values, and calculate the second Stokes parameter as the difference between the second and fourth gray values. Then, the polarization angle at that pixel location is half the arctangent function value, where the independent variable of the arctangent function is the ratio of the second Stokes parameter to the first Stokes parameter. When the first Stokes parameter is zero, the polarization angle is assigned a value of π / 4 or 3π / 4 depending on the sign of the second Stokes parameter. When the second Stokes parameter is also zero, the pixel is marked as having an invalid polarization angle.

[0010] Furthermore, in step 4, the specific steps for the cross-modal pre-fusion network to generate the cross-modal fusion feature map are as follows: For any point in the standard point cloud of the valve plate, calculate its local normal vector, and project the point onto the image plane of the kth polarization angle through the projection matrix. Then, project the local normal vector onto the image plane through the rotation component of the projection matrix and normalize it to obtain the two-dimensional normal vector projection. Extract the polarization angle value at the projection position from the polarization angle image; if the polarization angle value is marked as invalid, skip the viewpoint. Construct the polarization angle expression vector and its orthogonal vector respectively, calculate the first value of the absolute value of the inner product of the two-dimensional normal vector projection and the polarization angle expression vector, and the second value of the absolute value of the inner product of the two-dimensional normal vector projection and the orthogonal vector, take the maximum value between the first value and the second value, and use the result of subtracting the maximum value from the unit as the gate signal; When the gate signal is less than the preset gate threshold, the local normal vector and the polarization angle expression vector are concatenated to form the fused feature vector of the point under the viewpoint; otherwise, all elements of the fused feature vector are set to zero. After traversing all viewpoints, the effective fused feature vectors of each 3D point are written into the 2D feature map according to their projection positions to obtain the cross-modal fused feature map.

[0011] Furthermore, in step 5, the backbone of the defect segmentation network adopts an adaptive deformable convolution module, wherein the sampling point offset of the deformable convolution is generated by the cross-modal fusion feature map through an offset prediction branch including at least two convolutional layers.

[0012] Furthermore, the adaptive deformable convolution module also includes a modulation factor branch, which has the same structure as the offset prediction branch and whose output is mapped to a modulation factor in the range of zero to one by the Sigmoid function. The feature value of each sampling point of the deformable convolution is multiplied by the corresponding modulation factor before weighted summation.

[0013] Furthermore, the offset prediction branch consists of a cascaded 1×1 convolutional layer, a modified linear unit activation function, and a 3×3 convolutional layer, wherein the number of output channels of the 3×3 convolutional layer is twice the number of sampling points of the offset tensor.

[0014] Furthermore, the loss function of the defect segmentation network consists of two parts: a hybrid segmentation loss and an offset regularization loss. The hybrid segmentation loss includes cross-entropy loss and multi-class Dice loss, and the offset regularization loss constrains the magnitude and spatial gradient of the offset.

[0015] The present invention also provides an automatic valve plate defect detection system based on three-dimensional vision, which is applied to the above-mentioned automatic valve plate defect detection method based on three-dimensional vision, including: an intelligent sensing system, a point cloud preprocessing module, a cross-modal pre-fusion network, a defect segmentation network, and a reference plane fitting and flatness calculation module. The intelligent sensing system includes a structured light projector, a multi-view industrial camera group, and intelligent sensing elements. The multi-view industrial camera group includes N polarization industrial cameras, where N is an integer greater than or equal to 3. Each polarization industrial camera has a rotatable linear polarizer mounted in front of its lens. The structured light projector, the multi-view industrial camera group, and the intelligent sensing elements together constitute the intelligent sensing system, which is used to sequentially acquire structured light stripe images and polarization images when the valve plate under test is stationary. The point cloud preprocessing module is used to perform outlier filtering and smoothing resampling on the original 3D point cloud and output the valve plate standard point cloud; The cross-modal pre-fusion network is used to perform gated fusion of the standard point cloud of the valve plate with the polarization angle image calculated based on the Stokes parametric method to generate a cross-modal fusion feature map. The backbone of the defect segmentation network adopts an adaptive deformable convolutional module that includes an offset prediction branch, which is used to output a pixel-level defect segmentation mask. The reference plane fitting and flatness calculation module is used to perform robust plane fitting using valid points after excluding defect areas and to calculate the measured flatness value.

[0016] In summary, the present invention has at least one of the following beneficial technical effects: Firstly, by sequentially acquiring the three-dimensional point cloud and multi-view polarization images of the valve plate under test within a single static acquisition cycle, the three-dimensional geometric information and polarization optical information are naturally registered in space, avoiding cross-modal spatial deviations introduced by object motion, which is conducive to improving the accuracy of subsequent fusion and segmentation.

[0017] Secondly, the cross-modal pre-fusion network adopts a bidirectional gating mechanism, which uses the deviation between the directional relationship between the projection direction of the three-dimensional normal vector and the principal axis of the polarization ellipse to determine the consistency between geometric information and polarization information. It can adapt to the changes in polarization characteristics of valve plates of different materials under different lighting conditions, and suppress unreliable polarization signals while retaining effective complementary information, thus providing robust cross-modal fusion features for the defect segmentation network.

[0018] Third, the defect segmentation network introduces an adaptive deformable convolution module, whose sampling point offset is generated by the cross-modal fusion feature map through an offset prediction branch containing the spatial receptive field. This structure allows the receptive field of the convolution kernel to adaptively deform according to the combination mode of the normal vector and polarization angle of each region. For thin scratches, the receptive field can be stretched along the extension direction, and for point-like pits, the receptive field can be shrunk. This helps to capture various defect types with significant morphological differences and has a positive impact on the segmentation sharpness of defect boundaries.

[0019] Fourth, in the flatness calculation stage, a defect segmentation mask is used to pre-exclude three-dimensional points within the defect area, and a robust least squares method based on Tukey's double-weight function is used to iteratively fit the reference plane for non-defect valid points. This method can reduce the interference of local defects such as dents and scratches on the fitting of the ideal reference plane, while suppressing the influence of residual non-Gaussian error points, so that the measured flatness value more realistically reflects the morphological deviation of the valve plate's non-defect surface, thus improving the accuracy and repeatability of flatness evaluation.

[0020] Fifth, the entire inspection method completes three-dimensional reconstruction, polarization acquisition, defect segmentation and flatness calculation in one go at a single inspection station. The system has a high degree of integration and the inspection cycle can meet the production requirements of online full inspection. It provides a multi-modal and high-precision automated solution for the quality control of precision thin sheet parts such as compressor valve plates. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 A bar chart comparing mIoU in ablation experiments; Figure 3 A scatter plot comparing the flatness measurements of valve plates of different specifications. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0023] This invention provides an automatic valve plate defect detection system based on three-dimensional vision, comprising: an intelligent sensing system, a point cloud preprocessing module, a cross-modal pre-fusion network, a defect segmentation network, and a reference plane fitting and flatness calculation module, which will be described in detail below: The intelligent sensing system includes a structured light projector, a multi-view industrial camera assembly, and intelligent sensing elements. The structured light projector uses a digital light processing projection unit to project four sets of sinusoidal phase-shifted fringes of different frequencies onto the valve plate under test. The multi-view industrial camera assembly includes three polarization industrial cameras. Each polarization camera has a linear polarizer mounted in front of its lens, which can be rotated by a servo motor. The linear polarizer is fixed during the structured light acquisition phase. The transmission azimuth angle is rotated sequentially to the polarization acquisition stage. , , and The transmission azimuth angle. The multi-view industrial camera group, together with the structured light projector and intelligent sensing element, constitutes an intelligent sensor. The intelligent sensing element is an embedded processor, which is electrically connected to the structured light projector and the multi-view industrial camera group respectively, to uniformly control the projection and acquisition timing, and to deploy a multi-frequency heterodyne phase decoding algorithm and a stereo phase matching algorithm to generate the original three-dimensional point cloud of the valve plate under test.

[0024] The input of the point cloud preprocessing module is connected to the output of the intelligent sensing element to receive the raw 3D point cloud. This module has a built-in outlier filtering unit based on local neighborhood covariance eigenvalue analysis and a smoothing resampling unit based on moving least squares. After sequential processing, it outputs the standard point cloud of the valve plate.

[0025] The input of the cross-modal pre-fusion network is connected to the output of the point cloud preprocessing module to receive the standard point cloud of the valve plate, and to a multi-view industrial camera group to receive the polariton image sequence. Internally, the network includes a normal vector calculation submodule, a polarization angle calculation submodule based on Stokes parameters, and a multi-view projection and bidirectional gating fusion submodule, outputting a cross-modal fusion feature map.

[0026] The input of the defect segmentation network is connected to the output of the cross-modal pre-fusion network. The defect segmentation network employs an encoder-decoder architecture, with the encoder backbone composed of multiple cascaded adaptive deformable convolutional modules. Each adaptive deformable convolutional module contains an offset prediction branch and a modulation factor branch. The offset prediction branch is constructed by... Convolutional layers, modified linear unit activation functions and The network consists of cascaded convolutional layers, with the modulation factor branch structure identical to the offset prediction branch, followed by a sigmoid activation function. The defect segmentation network outputs a pixel-level defect segmentation mask.

[0027] The first input of the reference plane fitting and flatness calculation module is connected to the point cloud preprocessing module to receive the standard point cloud of the valve plate, and the second input is connected to the defect segmentation network to receive the defect segmentation mask. After excluding 3D points within the defect area, this module uses a robust least squares method based on Tukey's double-weight function to iteratively fit the reference plane and calculate the measured flatness value.

[0028] The system also includes a conveyor belt and a sorting actuator. Under the control of the intelligent sensing element, the conveyor belt pauses at the inspection station to keep the valve piece under test stationary. The sorting actuator is either a pneumatic air-blowing rejection device or a mechanical lever rejection device, and its control end is electrically connected to the intelligent sensing element. When the measured flatness value exceeds a preset flatness tolerance threshold or the area of ​​any defect in the defect segmentation mask exceeds a preset area threshold, the intelligent sensing element outputs a trigger signal to control the sorting actuator to remove the valve piece currently under test, after which the conveyor belt resumes operation.

[0029] like Figure 1As shown, the present invention also provides an automatic valve plate defect detection method based on three-dimensional vision, applied to the above-mentioned automatic valve plate defect detection system based on three-dimensional vision, comprising: Step 1: Collect the original 3D point cloud of the valve plate under test based on the intelligent sensing system; Step 2: Preprocess the original 3D point cloud to output the standard point cloud of the valve plate; Step 3: Acquire auxiliary image sequences of the valve plate under test from various perspectives based on the intelligent sensing system, and calculate the polarization angle image corresponding to each perspective based on the Stokes parametric method. Step 4: Input the standard point cloud of the valve plate and the polarization angle images of all views into the cross-modal pre-fusion network, project each three-dimensional point in the standard point cloud of the valve plate onto the image plane of each polarization angle, extract the corresponding polarization angle value, and use the deviation of the direction relationship between the projection direction of the three-dimensional point normal vector in the image plane and the direction of the principal axis of the polarization ellipse as the gating signal, and concatenate the polarization angle value with the three-dimensional point normal vector to form a cross-modal fusion feature map; Step 5: Input the cross-modal fused feature map into the defect segmentation network and output a pixel-level segmentation mask for the valve plate surface defects; Step 6: Fit a reference plane using the standard point cloud of the valve plate. For the effective points in the standard point cloud of the valve plate that are not located within the defect segmentation mask, use the robust least squares method to iteratively solve the plane equation. Use this plane as the ideal reference plane, calculate the directed distance from all effective points to the ideal reference plane, and use the sum of the absolute values ​​of the maximum positive deviation and the maximum negative deviation as the measured flatness value.

[0030] Next, the above steps will be explained in detail with reference to the system and specific parameter settings: In step 1, the original 3D point cloud of the valve plate under test is acquired based on the intelligent sensing system, specifically as follows: After the control conveyor belt transports the valve piece under test to the testing station, it pauses its operation to keep the valve piece stationary throughout the subsequent data acquisition process. Multi-frequency phase-shifting fringes are projected onto the valve piece using a structured light projector, and the linear polarizer of the multi-view industrial camera group is fixed to... Under the condition of the vibration transmission azimuth angle, a multi-view industrial camera group synchronously acquires a sequence of multi-view fringe images of the valve plate. Based on the multi-view fringe image sequence, intelligent sensing elements are used to perform multi-frequency heterodyne phase decoding and stereo phase matching to generate the original three-dimensional point cloud of the valve plate under test.

[0031] The specific structure of the intelligent sensing system will be described in detail in the subsequent system section. Here, we will explain its specific usage: Each polarizing industrial camera has a linear polarizer mounted in front of its lens, which can be rotated by a servo motor. The linear polarizer is fixed in step 1. Vibration transmission azimuth. The value of is not less than an integer. This ensures geometric redundancy in subsequent multi-view point cloud fusion and polarization angle calculation. The multi-view industrial camera group, along with the structured light projector and intelligent sensing elements, constitutes an intelligent sensor. This intelligent sensor is used to simultaneously complete structured light projection acquisition and polarization image acquisition under a single trigger.

[0032] The structured light projector uses a digital light processing projection unit to project four sets of sinusoidal phase-shifted fringes of different frequencies onto a stationary valve plate located on a conveyor belt. Each set of fringes contains a four-step phase-shift pattern. The frequencies of the four sets of fringes are as follows: , , , The multi-view industrial camera group and structured light projector are controlled by intelligent sensing elements to sequentially complete fringe projection and synchronous acquisition within one trigger cycle, obtaining a multi-frequency phase-shifted fringe image sequence from each camera's viewpoint. The linear polarizer is fixed during step 1. The azimuth angle is such that the intensity of the unpolarized light projected by the structured light projector is reduced by about 50% after passing through the linear polarizer. This is compensated for by extending the camera exposure time or increasing the optical power of the structured light projector to ensure that the four-step phase-shifted image has a sufficient grayscale dynamic range.

[0033] The intelligent sensing element is an embedded processor or industrial computer, which internally deploys a multi-frequency heterodyne phase decoding algorithm and a stereo phase matching algorithm. First, for each frequency stripe image acquired by each camera, the position of each pixel is extracted. The wrap phase value The expression is as follows: ; in to The four-step phase-shifting images are respectively in grayscale value at that location For the range of values The wrapping phase.

[0034] Then, the multi-frequency heterodyne method is used to unwrap the wrapped phase at four frequencies step by step. Starting with the lowest frequency... The wrapping phase Based on this, its absolute phase .right ,use Assisted deployment get And so on, until the highest frequency is obtained. The absolute phase diagram is shown below. The formula for calculating the absolute phase is as follows.

[0035] ; in , This is the rounding function.

[0036] After obtaining the absolute phase maps at the highest frequencies of all cameras, stereo phase matching is performed on corresponding pixels between any two cameras using pre-calibrated internal and external parameters of the intelligent sensing system. For the pixels in the first polarization industrial camera... and its absolute phase value In the second polarization industrial camera, the absolute phase value is searched along the polar line. corresponding pixels The three-dimensional spatial coordinates are calculated using triangulation to generate an original 3D point cloud. Different camera pairs are then sequentially matched and fused to obtain a more complete original 3D point cloud. In step 2, the original 3D point cloud is preprocessed to output the standard point cloud of the valve plate, specifically as follows: The original 3D point cloud is input into a pre-built point cloud preprocessing module. The point cloud preprocessing module sequentially performs statistical outlier filtering based on local neighborhood covariance eigenvalue analysis and smoothing resampling based on moving least squares on the original 3D point cloud, and outputs a dense and denoised valve plate standard point cloud.

[0037] The original 3D point cloud inevitably contains outliers caused by phase mismatch and ambient stray light, and the point cloud density may be uneven in the valve edge region. This step obtains a high-quality standard point cloud for the valve through two-step processing.

[0038] First, statistical outlier filtering is performed based on local neighborhood covariance eigenvalue analysis. This is done for each point in the original 3D point cloud. With search radius Constructing a local neighborhood point set The number of neighboring points is The covariance matrix of the neighborhood point set is calculated as follows.

[0039] ; in Let be the centroid coordinate vector of the neighborhood point set. For the covariance matrix... Eigenvalue decomposition yields three non-negative eigenvalues. , , And satisfy .

[0040] Define neighborhood curvature factor as follows.

[0041] ; It reflects the degree of change of the local surface in the neighborhood along the minimum principal direction. For a point located on the smooth surface of the valve plate, much smaller and , close to For outliers, the three eigenvalues ​​are similar in magnitude. close to Set curvature threshold ,when At that time, the point Mark them as outlier candidates and remove them from the original 3D point cloud. The preferred value range is to It was determined through experiments. It can filter out the vast majority of outliers while preserving the effective edge points.

[0042] After obtaining the preliminary filtered point cloud, smooth resampling based on moving least squares is performed. For each point in the preliminary filtered point cloud... In radius Within a spherical neighborhood, a set of neighborhood points is selected to construct a local quadratic surface fitting function. The moving least squares method is then used to find the local quadratic surface that minimizes the weighted fitting residual. Then, the current point... Move along the normal direction of the local quadratic surface to the projection position on the quadratic surface, iterate this process until convergence, and obtain the standard point cloud of the valve plate. Radius Based on the point cloud density setting, the preferred setting is the average point cloud spacing. to This process doubles the previous one. After processing, the standard point cloud of the valve plate achieves uniform sampling density and smooth surface characterization while maintaining the true geometric shape of the valve plate.

[0043] In step 3, auxiliary image sequences of the valve plate under test are acquired from various perspectives using the intelligent sensing system, and the polarization angle image corresponding to each perspective is calculated based on the Stokes parametric method. Specifically: With the valve under test remaining stationary, the structured light projector is turned off and the uniform diffuse auxiliary light source is turned on. The linear polarizers of the multi-view industrial camera group are then rotated sequentially to... , , and The transmission azimuth angle is determined, and an image is simultaneously acquired by a multi-view industrial camera group at each azimuth angle to obtain a multi-view auxiliary image sequence for the valve plate. Each view in the auxiliary image sequence contains four polariton images corresponding to the four polarization azimuth angles. The polarization angle image corresponding to each view is calculated based on the Stokes parametric method.

[0044] Steps 1 and 3 are executed sequentially within the same static cycle of the valve plate under test. There is no spatial offset between the 3D point cloud and the polarization angle image caused by the movement of the object, which ensures the spatial registration accuracy of the projection fusion in step 4.

[0045] For any viewpoint, let , , , These represent the polarizer images at pixel positions corresponding to the four transmission azimuth angles from this viewpoint. The grayscale value at that location. First, calculate the Stokes parameter. and as follows.

[0046] ; ; The polarization angle image at this viewpoint is located polarization angle value at The definition is as follows.

[0047] ; in The return value is mapped to a range. .when At that time, if but ,like but ,like If the polarization angle of the pixel is marked as invalid, it means that the polarization degree at that location is below the resolvable level, and the pixel will be skipped in subsequent fusion steps.

[0048] polarization angle This characterizes the orientation angle of the polarization ellipse of the scattered light at the pixel location. For subsurface microcracks on the valve plate surface caused by residual stress from stamping, the polarization angle exhibits a significant directional abrupt change in the crack edge region, making the polarization angle image an effective source of information for detecting subsurface defects. For the flat, normal area of ​​the valve plate, the degree of polarization is low under near-orthogonal camera viewing conditions, and the polarization angle of some pixels may be marked as invalid. This does not affect the overall detection logic because defect segmentation mainly relies on the complementarity of polarization information from defect edges and beveled regions with three-dimensional geometric information.

[0049] In step 4, the standard point cloud of the valve plate and polarization angle images from all viewpoints are input into the cross-modal pre-fusion network. Each 3D point in the standard point cloud of the valve plate is projected onto its respective polarization angle image plane, and the corresponding polarization angle value is extracted. The deviation between the projection direction of the 3D point normal vector in the image plane and the principal axis of the polarization ellipse is used as a gating signal. The polarization angle value and the 3D point normal vector are concatenated to form a cross-modal fusion feature map. Specifically: This section provides a detailed explanation of the specific operations performed by the cross-modal pre-fusion network: For any point in the standard point cloud of the valve plate Calculate the local normal vector at that point. Take the radius in the standard point cloud of the valve plate with the center as the center. Given a set of neighborhood points within a given area, fit a local plane using this set of neighborhood points. The normalized normal vector of this plane is denoted as... .radius Preferably, the average spacing of the point cloud This is multiplied by a factor of 1 to ensure the robustness of the normal vector estimation.

[0050] Then, using the pre-calibrated intrinsic and extrinsic parameters of the intelligent sensing system, the three-dimensional points are... Project to the Each polarization angle image plane. The projection relationship is determined by the projection matrix. Description: Image coordinates are obtained through homogeneous coordinate transformation. . The three-dimensional normal vector Through projection matrix The rotation component (i.e., the upper left) Projecting the submatrix onto the image plane, taking the first two components and normalizing them, yields the two-dimensional normal vector projection. ,satisfy .

[0051] Extracting from polarization angle images using bilinear interpolation polarization angle value at If the pixel position is marked as having an invalid polarization angle in step 3, then that viewpoint is skipped and will not participate in the subsequent fusion processing of that 3D point.

[0052] Constructing a polarization angle representation vector and its orthogonal vectors as follows.

[0053] ; ; The purpose of using double-width characters is to eliminate exist and The discontinuity in representation caused by the periodicity of angles nearby means that continuously changing angles also change continuously in vector space. Pointing towards the major axis of the polarization ellipse, Pointing towards the minor axis of the polarization ellipse.

[0054] To evaluate the relationship between the three-dimensional geometric direction and the polarization direction in the first... Consistency under different perspectives, constructing gating signals as follows.

[0055] ; in Represents the vector dot product. Represents absolute value. The absolute value of the inner product. The degree of parallelism between the projection direction of the normal vector and the major axis of the polarization ellipse is equal to... When it indicates strict parallelism, it is equal to The time interval indicates strict perpendicularity. The absolute value of the inner product. This measures the degree of parallelism between the projection direction of the normal vector and the minor axis of the polarization ellipse. Because... and Orthogonal, at least one of them is perpendicular to... The absolute value of the inner product is greater than or equal to When the projection direction of the normal vector is aligned with any principal axis of the polarization ellipse, Item close , Approaching This design does not presuppose that the projection of the normal vector and the polarization direction must be parallel or perpendicular, but rather accommodates both possible physical correspondences, thus adapting to the changes in polarization characteristics of valve plates made of different materials under different lighting conditions.

[0056] Preset gate threshold Preferred .when and When valid, the geometric and polarization information is considered reliably consistent from that perspective. and The fused feature vector is formed by splicing the features together from this perspective. as follows.

[0057] ; when or If invalid, it is assumed that there is an abnormal geometric-polarization mismatch or that the polarization signal is unreliable; therefore, no fusion feature is generated from this perspective. All elements are set to zero.

[0058] right After traversing the above process from each perspective, the effective fused feature vectors of each 3D point are arranged according to their projection positions. Including the depth value of that point Several layers of two-dimensional feature maps are written, and all two-dimensional feature maps together constitute a cross-modal fusion feature map. When multiple 3D point projections exist at a certain spatial location, the fused feature vector with the smallest depth value is retained.

[0059] In step 5, the cross-modal fused feature map is input into the defect segmentation network, and the output is a pixel-level segmentation mask for the valve plate surface defects, specifically: The defect segmentation network employs an encoder-decoder architecture. The encoder backbone consists of multiple cascaded adaptive deformable convolutional modules, with a stride of [value missing] between each pair of adaptive deformable convolutional modules. The downsampling layer is used to achieve multi-scale feature extraction.

[0060] The adaptive deformable convolution module contains three parallel branches: the main convolution branch, the offset prediction branch, and the modulation factor branch.

[0061] The offset prediction branch consists of two convolutional layers. The first layer is... Convolution compresses the number of channels in the input feature map to a smaller number. , Ideally, it should be one-quarter of the number of input channels. This is followed by a modified linear unit activation function. The second layer is... Convolution, with the number of output channels being The offset tensor, where For the number of sampling points of the deformable convolution kernel, Convolution kernel, . Each channel corresponds to Each sampling point is and Offset in both directions. Convolutional layers provide offset prediction The spatial receptive field allows for the reference of the geometric and polarization context information of its neighborhood when predicting the offset of each sampling point. The modulation factor branch has the same structure as the offset prediction branch, and the number of output channels is... The modulation factor tensor is mapped to its range by the Sigmoid activation function. Interval.

[0062] Let the set of offsets output by the offset prediction branch be... The set of modulation scalars output by the modulation factor branch is The output value of adaptive deformable convolution is... The calculation is as follows.

[0063] ; in For the input feature map, For the convolution kernel at the standard sampling position The weight of the position, Traversal Grid Standard Position .because Typically, these are non-integer coordinates, and the corresponding feature values ​​are obtained from the feature map through bilinear interpolation. Modulation factor exist Within the interval, the contribution weight of each sampling point is adaptively adjusted.

[0064] In this invention, the offset is generated by the cross-modal fusion feature map through an offset prediction branch, enabling the spatial offset of the convolution kernel sampling points to be self-consistently adjusted according to the combination mode of the normal vector and polarization angle of each region. For the scratch region, the change in normal vector is concentrated in the scratch extension direction. After the convolutional layer senses the gradient direction of the normal vector in the neighborhood, the offset prediction branch generates a large offset along the scratch direction, stretching the receptive field to fully capture the slender defect. For the pit region, the local normal vector changes more uniformly in all directions, the offset remains small and symmetrical, and the receptive field shrinks to near the standard mesh, avoiding interference from surrounding normal textures. The modulation factor automatically reduces the weight of unreliable sampling points at the defect boundary, further improving the sharpness of the segmentation boundary.

[0065] The decoder of the defect segmentation network fuses multi-scale features from various layers of the encoder via skip connections. After several upsampling layers and standard convolutional layers, the final output channel count is... The segmentation prediction map, This is equal to the number of defect categories plus one. The defect categories include three types: dents, scratches, and edge chips, plus the background category, for a total of four categories. Segmentation uses pixel-wise Softmax classification, outputting the probability of each pixel belonging to each defect category and the background. The category label is taken as the one with the highest probability, resulting in a pixel-level segmentation mask.

[0066] The loss function of the defect segmentation network consists of two parts: The first part is a hybrid segmentation loss based on Dice coefficients and cross-entropy, which is applied to the final output layer of the decoder; The second part is the offset regularization loss of adaptive deformable convolution, which is applied to the offset prediction branch of all adaptive deformable convolution modules. Total loss The calculation formula is as follows: ; in, To predict the segmentation probability map, This is a true-to-life annotation of the unique heat encoding diagram. This is the pixel-wise cross-entropy loss. For multi-class Dice loss, the mean area of ​​overlap between the predicted region and the ground truth labeled region for each class is calculated. As the balance coefficient, the preferred one is... . This represents the total number of spatial locations involved in the calculation. Constrain the offset amplitude to prevent the sampling point from deviating excessively from the standard position. The spatial gradient of the constraint offset, where It is a spatial gradient operator, which is approximated by the difference between adjacent pixels. The gradient penalty coefficient is preferred. . The total loss balancing weight coefficient is the preferred one. . The number of sampling points for deformable convolution kernels In step 6, a reference plane is fitted using the standard point cloud of the valve plate. For valid points in the standard point cloud that are not located within the defect segmentation mask area, the plane equation is iteratively solved using the robust least squares method. This plane is used as the ideal reference plane. The directed distances from all valid points to the ideal reference plane are calculated. The sum of the absolute values ​​of the maximum positive deviation and the maximum negative deviation is used as the measured flatness value. Specifically: Step 6 aims to accurately calculate the flatness of the valve plate while eliminating the influence of defect areas. First, the defect segmentation mask is back-projected from the 2D image plane to the 3D space containing the valve plate's standard point cloud using a projection matrix. For each 3D point in the valve plate's standard point cloud, it is projected using a projection matrix... Project the image onto the image plane that generated the mask, and check if the projected position falls within the defect mask area. If it does, mark the 3D point as a defect point and exclude it; the remaining points are the valid point set.

[0067] For the effective point set, a robust least squares method is used to iteratively fit the reference plane. Let the plane equation be... Let the parameter vector be... . No. The coordinates of the valid points are .

[0068] In each iteration of robust least squares, the directed distance from each valid point to the current estimated plane is calculated. as follows.

[0069] ; according to The weight of each point is calculated using the Tukey double-weight function, whose expression is: when... Time weight ,when Time weight .in For all in the current iteration The robust estimate of the median absolute deviation is calculated as follows: , It is a constant, preferably This corresponds to a 99.7% confidence interval under a Gaussian distribution.

[0070] Update parameters using weighted least squares That is, to solve for Minimize the parameter value. Iterate until the parameter changes by a certain amount. Norm less than convergence threshold Or reach the maximum number of iterations .

[0071] After iterative convergence, the equation of the ideal reference plane is obtained. The directed distances from all valid points in the valve plate's standard point cloud to this ideal reference plane are then recalculated. Let the maximum positive deviation be... The maximum negative deviation is Then the measured flatness value for.

[0072] ; Measured flatness values It is the minimum distance between two parallel ideal planes that encompass all valid points, conforming to the flatness definition specified in GB / T 11337-2004.

[0073] Since points in defective areas such as pits and scratches have been excluded when fitting the reference plane, and the robust least squares method further suppresses the influence of residual non-Gaussian error points on the fitting results through the Tukey double weight function, and It reflects the true morphological deviation of the non-defective surface of the valve plate.

[0074] When the measured flatness value Exceeding the preset flatness tolerance threshold When, or when the area of ​​any defect in the defect segmentation mask exceeds a preset area threshold. At that time, the intelligent sensing system outputs a defect alarm signal and controls the sorting actuator to remove the current valve piece to be tested from the conveyor belt. Afterwards, the conveyor belt resumes operation, transporting the next valve piece to be tested to the inspection station. Based on the valve plate tolerance grade, for compressor valve plates, Set as . Set as .

[0075] The present invention also provides several embodiments to further illustrate the above method: Example 1: This embodiment is used to verify the curvature threshold in the point cloud preprocessing module. The influence of the selection of parameters on the outlier filtering effect and the effective edge point retention rate was investigated. The experiment was conducted on a batch of 200 compressor valve plates containing known manually labeled defects, covering three defect types: pits, scratches, and edge chipping, as well as qualified products. The original 3D point cloud was acquired within 0.4 seconds using a structured light array system, with each point cloud containing approximately 500,000 points. Curvature thresholds were set separately. The values ​​were 0.10, 0.15, 0.20, 0.25, and 0.30. The number of outliers removed and the false removal rate of valid points on the valve edge were calculated (based on manual visual confirmation). The results are shown in Table 1.

[0076] Table 1. Outlier filtering effect under different curvature thresholds

[0077] As shown in Table 1, when At this threshold, the number of outliers removed was moderate (7643), and the false removal rate of valid edge points was the lowest (0.04%), achieving the best balance between noise reduction and preservation of true geometric information. When the threshold was too low, some outliers were not removed, interfering with subsequent normal vector and flatness calculations; when the threshold was too high, valid points in the slight notches or arc transition areas of the valve edge were incorrectly removed, resulting in a decrease in the recall rate of edge chipping defect detection.

[0078] Example 2: This embodiment compares the performance of the method of the present invention with existing two-dimensional vision inspection schemes and line laser profilometry schemes in defect detection and planarity measurement. 200 valve pieces were selected as the test set, including 85 samples with various defects and 115 qualified samples. Three schemes were used for inspection: Scheme A was a two-dimensional vision inspection system based on a 5-megapixel industrial camera and a ring light source, utilizing ResNet50 feature extraction and a U-Net segmentation network; Scheme B was a line scanning scheme using a single-line laser profilometry sensor and a displacement platform, with a scanning speed of 5 mm / s and a dot pitch of 0.05 mm, using the traditional elevation threshold method to detect pits and calculate flatness; Scheme C was the method of the present invention. All three schemes were run in the same hardware environment (Intel i7-13700 processor, 32GB memory). The accuracy, recall, F1 score of defect detection and the repeatability of planarity measurement (standard deviation of 20 repeated measurements of the same valve piece) were statistically analyzed, and the inspection cycle time per piece was recorded. The results are shown in Table 2.

[0079] Table 2 Performance Comparison of Three Detection Schemes

[0080] As shown in Table 2, the method of this invention significantly outperforms traditional two-dimensional vision solutions in the detection rate of all three types of defects, especially for shallow pits with a depth of less than 10 μm, where the recall rate increases from 58.2% to 96.4%. In planar measurement, the line laser solution is affected by vibration of the moving platform and cumulative errors from multiple stitching operations, resulting in a repeatability of only 3.8 μm. In contrast, this invention achieves a repeatability of 1.2 μm through robust fitting of single static acquisition and defect area exclusion, meeting the 15 μm tolerance detection requirement for compressor valve plates. The single-piece inspection cycle is 1.6 seconds, including 0.4 seconds of structured light acquisition, 0.3 seconds of polarization acquisition, and 0.9 seconds of algorithm processing, fully meeting the production line's 3-second / piece cycle requirement.

[0081] Example 3: This embodiment is used to verify the contributions of the bidirectional gating mechanism, adaptive deformable convolution module, and modulation factor branch in the cross-modal pre-fusion network to the final segmentation performance. Based on the 200 test sets in Embodiment 2, four ablation models were constructed: Model 1 removed the polarization branch and used only the 3D normal vector features for segmentation; Model 2 retained the polarization branch but adopted the single-direction gating in the original scheme (only calculating the inner product of the normal vector projection and the major axis direction); Model 3 replaced the adaptive deformable convolution with a standard convolution with the same number of parameters; Model 4 removed the modulation factor branch and retained only the offset prediction. The remaining settings of each model were the same as the complete scheme of this invention, and the mIoU (mean intersection-over-union ratio) of the segmentation results was statistically analyzed. The results are shown in Table 3 and... Figure 2 As shown.

[0082] Table 3 Ablation Experiment Results

[0083] Table 3 shows that removing the polarization branch reduces mIoU by 6.7 percentage points, demonstrating the significant complementary effect of polarization information on subsurface defect detection. When using unidirectional gating (Model 2), the segmentation performance is lower than bidirectional gating due to the erroneous suppression of some normal regions, resulting in a 2.8 percentage point decrease in mIoU. Replacing it with standard convolution reduces mIoU by 4.5 percentage points, indicating that adaptive deformable convolution effectively captures the morphological differences of fine scratches and pits through its deformation receptive field. Removing the modulation factor reduces mIoU by 1.4 percentage points, demonstrating the improvement effect of the modulation mechanism on fine boundary segmentation.

[0084] Example 4: This embodiment is used to verify the adaptability of the method of the present invention to valve plates of different materials and thicknesses, and the accuracy of flatness measurement. Three specifications of valve plates were selected: specification A is a 304 stainless steel annular valve plate (thickness 0.20mm), specification B is a spring steel 60Si2Mn butterfly valve plate (thickness 0.35mm), and specification C is a 7A04 aluminum alloy valve plate (thickness 0.15mm). Thirty qualified samples of each specification were taken, and their flatness was measured using the method of the present invention and a high-precision coordinate measuring machine (ZEISS CONTURA G2, accuracy 0.6μm). The coordinate measuring machine measurement value was used as the reference true value to calculate the measurement error of the method of the present invention. The results are shown in Table 4 and... Figure 3 As shown.

[0085] Table 4. Measurement error of flatness of valve plates of different materials

[0086] As shown in Table 4, the average error of the flatness measurement of valve plates of the present invention using the method of the present invention is less than 1.5 μm, and the maximum error does not exceed 2.5 μm. The flatness measurement is highly linearly correlated with the coordinate measuring machine (CMM) measurements (R0). 2 All values ​​are greater than 0.99, fully meeting the accuracy requirements for industrial testing.

[0087] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. An automatic defect detection method for valve plates based on three-dimensional vision, characterized in that, include: Step 1: Collect the original 3D point cloud of the valve plate under test based on the intelligent sensing system; Step 2: Preprocess the original 3D point cloud to output the standard point cloud of the valve plate; Step 3: Acquire auxiliary image sequences of the valve plate under test from various perspectives based on the intelligent sensing system, and calculate the polarization angle image corresponding to each perspective based on the Stokes parametric method. Step 4: Input the standard point cloud of the valve plate and the polarization angle images of all views into the cross-modal pre-fusion network, project each three-dimensional point in the standard point cloud of the valve plate onto the image plane of each polarization angle, extract the corresponding polarization angle value, and use the deviation of the direction relationship between the projection direction of the three-dimensional point normal vector in the image plane and the direction of the principal axis of the polarization ellipse as the gating signal, and concatenate the polarization angle value with the three-dimensional point normal vector to form a cross-modal fusion feature map; Step 5: Input the cross-modal fused feature map into the defect segmentation network and output a pixel-level segmentation mask for the valve plate surface defects; Step 6: Fit a reference plane using the standard point cloud of the valve plate. For the effective points in the standard point cloud of the valve plate that are not located within the defect segmentation mask, use the robust least squares method to iteratively solve the plane equation. Use this plane as the ideal reference plane, calculate the directed distance from all effective points to the ideal reference plane, and use the sum of the absolute values ​​of the maximum positive deviation and the maximum negative deviation as the measured flatness value.

2. The automatic valve plate defect detection method based on three-dimensional vision according to claim 1, characterized in that, In step 3, the auxiliary image sequence includes four polariton images.

3. The automatic valve plate defect detection method based on three-dimensional vision according to claim 2, characterized in that, In step 3, the polarization angle images corresponding to each viewpoint are calculated based on the Stokes parametric method, specifically as follows: For any viewing angle, let the gray values ​​of the polarizer image at the pixel location corresponding to its four transmission azimuth angles be the first gray value, the second gray value, the third gray value, and the fourth gray value, respectively. Calculate the first Stokes parameter as the difference between the first and third gray values, and calculate the second Stokes parameter as the difference between the second and fourth gray values. Then, the polarization angle at that pixel location is half the arctangent function value, where the independent variable of the arctangent function is the ratio of the second Stokes parameter to the first Stokes parameter. When the first Stokes parameter is zero, the polarization angle is assigned a value of π / 4 or 3π / 4 depending on the sign of the second Stokes parameter. When the second Stokes parameter is also zero, the pixel is marked as having an invalid polarization angle.

4. The automatic valve plate defect detection method based on three-dimensional vision according to claim 3, characterized in that, In step 4, the specific steps for the cross-modal pre-fusion network to generate the cross-modal fusion feature map are as follows: For any point in the standard point cloud of the valve plate, calculate its local normal vector, and project the point onto the image plane of the kth polarization angle through the projection matrix. Then, project the local normal vector onto the image plane through the rotation component of the projection matrix and normalize it to obtain the two-dimensional normal vector projection. Extract the polarization angle value at the projection position from the polarization angle image; if the polarization angle value is marked as invalid, skip the viewpoint. Construct the polarization angle expression vector and its orthogonal vector respectively, calculate the first value of the absolute value of the inner product of the two-dimensional normal vector projection and the polarization angle expression vector, and the second value of the absolute value of the inner product of the two-dimensional normal vector projection and the orthogonal vector, take the maximum value between the first value and the second value, and use the result of subtracting the maximum value from the unit as the gate signal; When the gate signal is less than the preset gate threshold, the local normal vector and the polarization angle expression vector are concatenated to form the fused feature vector of the point under the viewpoint; otherwise, all elements of the fused feature vector are set to zero. After traversing all viewpoints, the effective fused feature vectors of each 3D point are written into the 2D feature map according to their projection positions to obtain the cross-modal fused feature map.

5. The automatic valve plate defect detection method based on three-dimensional vision according to claim 4, characterized in that, In step 5, the backbone of the defect segmentation network adopts an adaptive deformable convolution module, wherein the sampling point offset of the deformable convolution is generated by the cross-modal fusion feature map through an offset prediction branch including at least two convolutional layers.

6. The automatic valve plate defect detection method based on three-dimensional vision according to claim 5, characterized in that, The adaptive deformable convolution module further includes a modulation factor branch, which has the same structure as the offset prediction branch and whose output is mapped to a modulation factor in the range of zero to one by the Sigmoid function. The feature value of each sampling point of the deformable convolution is multiplied by the corresponding modulation factor before weighted summation.

7. The automatic valve plate defect detection method based on three-dimensional vision according to claim 6, characterized in that, The offset prediction branch consists of a cascaded 1×1 convolutional layer, a modified linear unit activation function, and a 3×3 convolutional layer. The 3×3 convolutional layer outputs an offset tensor with twice the number of sampling points.

8. The automatic valve plate defect detection method based on three-dimensional vision according to claim 7, characterized in that, The loss function of the defect segmentation network consists of two parts: a hybrid segmentation loss and an offset regularization loss. The hybrid segmentation loss includes cross-entropy loss and multi-class Dice loss, while the offset regularization loss constrains the magnitude and spatial gradient of the offset.

9. An automatic valve plate defect detection system based on three-dimensional vision, applied to the automatic valve plate defect detection method based on three-dimensional vision as described in any one of claims 1-8, characterized in that, include: Intelligent sensing system, point cloud preprocessing module, cross-modal pre-fusion network, defect segmentation network, and reference plane fitting and flatness calculation module; The intelligent sensing system includes a structured light projector, a multi-view industrial camera group, and intelligent sensing elements. The multi-view industrial camera group includes N polarization industrial cameras, where N is an integer greater than or equal to 3. Each polarization industrial camera has a rotatable linear polarizer mounted in front of its lens. The structured light projector, the multi-view industrial camera group, and the intelligent sensing elements together constitute the intelligent sensing system, which is used to sequentially acquire structured light stripe images and polarization images when the valve plate under test is stationary. The point cloud preprocessing module is used to perform outlier filtering and smoothing resampling on the original 3D point cloud and output the valve plate standard point cloud. The cross-modal pre-fusion network is used to perform gated fusion of the valve plate standard point cloud and the polarization angle image calculated based on the Stokes parametric method to generate a cross-modal fusion feature map. The backbone of the defect segmentation network adopts an adaptive deformable convolutional module that includes an offset prediction branch, which is used to output a pixel-level defect segmentation mask. The reference plane fitting and flatness calculation module is used to perform robust plane fitting using valid points after excluding defect areas and to calculate the measured flatness value.

Images