Umbrella stand size defect detection system and method based on machine vision

Abstract

The present application relates to the field of machine vision detection and intelligent manufacturing, in particular to a system and method for detecting size defects of umbrella frames based on machine vision; the system clamps and applies a calibrated micro-vibration to a half-opened or fully opened umbrella frame through a jig; within a preset delay after the release, an industrial camera and a pulse light source are triggered synchronously, and 5-12 frames of images are continuously collected; after grayscale normalization, local contrast enhancement, candidate edge extraction and centroid tracking, the second-order differential sequence of the centroid and the displacement direction sequence are input into the timing branch, and the edge normal local grayscale graph is input into the spatial branch; after attention fusion, the effective edge is output; combined with the camera calibration parameters and the standard geometric model of the type, the umbrella rib width, joint gap, skeleton straightness and node offset are measured, and the defect results and abnormal positions are output according to the size qualified range; the present application avoids the measurement fluctuation caused by inconsistent single-frame collection time.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection and intelligent manufacturing, specifically to a machine vision-based system and method for detecting dimensional defects in umbrella frames. Background Technology

[0002] With the increasing automation of umbrella frame manufacturing, machine vision-based dimensional inspection has become an important part of umbrella frame quality control. Under the inspection standards of key dimensional accuracy such as umbrella rib width, joint gap, frame straightness and node misalignment, the system needs to have the technical capability to stably acquire and accurately measure images of the umbrella frame in its unfolded state.

[0003] However, the umbrella frame is usually accompanied by a short period of micro-vibration after release, and the metal edges are prone to reflective interference under light, which makes the effective edges in a single frame image unstable, thus affecting subsequent edge recognition and size conversion. At the same time, there are various umbrella frame models, and the structural benchmarks and qualified ranges corresponding to different models are different. If the calibration parameters and judgment rules that match the model are lacking, measurement deviations or misjudgments of defects are likely to occur.

[0004] Therefore, there is a need for a system and method that enables continuous imaging under controlled micro-vibration conditions, accurately filters out reflective interference by combining edge dynamic features and local grayscale information, and completes the static position determination, size calculation and defect judgment of different types of umbrella frames, so as to improve the stability and consistency of umbrella frame size detection. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based umbrella frame size defect detection system and method, which avoids the instability of effective edges, measurement deviations or defect misjudgments caused by short-term micro-vibrations and metal edge reflection interference after the umbrella frame is released. It can also accurately screen out reflection interference by combining edge dynamic features and local grayscale information, and realize defect judgment of different models of umbrella frames, thereby improving the stability and consistency of umbrella frame size detection.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A machine vision-based umbrella frame size defect detection system and method is proposed. The system uses a fixture support module to limit and clamp the umbrella frame in a semi-open or fully open state, and causes the umbrella frame to generate micro-vibration along a pre-calibrated main vibration direction.

[0008] After the external robotic arm completes its release, the detection start time is determined based on the release completion signal and the preset delay. Within the preset trigger window after the detection start time, the industrial camera and pulse light source are synchronously controlled to continuously acquire more than five frames and less than twelve frames of images.

[0009] The acquired images are normalized in grayscale, enhanced in local contrast, and candidate edges are extracted. The centroid changes of the same candidate edge are tracked to obtain temporal dynamic features for characterizing the micro-vibration state and spatial distribution features for characterizing the intensity of reflective interference.

[0010] Then, a multi-branch neural network containing a temporal feature extraction network, a spatial feature extraction network, and an attention fusion layer is used to fuse and distinguish the two types of features, and output the effective edge results;

[0011] Based on camera calibration parameters, standard geometric models, and acceptable dimensional ranges, static position and dimensional analysis of umbrella rib width, joint gaps, frame straightness, and node misalignment is performed, and defect conclusions and abnormal locations are output.

[0012] Furthermore, the fixed limiting component is provided with limiting surfaces that abut against the reference part of the umbrella frame, which are used to limit the translation and rotation of the umbrella frame in the measuring plane;

[0013] The elastic clamping component includes elastic clamping arms that are arranged opposite each other along the main vibration direction, apply a preload to the umbrella frame, and form a gap between the clamping surface that allows vibration. The micro-vibration amplitude is obtained and stored by standard umbrella frame calibration.

[0014] Furthermore, the visual acquisition and light source control process ensures that the first frame exposure start time is within a preset trigger delay after the detection start time. The industrial camera outputs an exposure trigger signal to the pulse light source at a preset frequency, and the exposure time is set to the microsecond level to reduce motion blur and improve edge stability.

[0015] Furthermore, the average coordinates of the candidate edge point set are used to determine the centroid, and a displacement sequence is constructed based on the centroid displacement of adjacent frames. The centroid coordinates that change along the frame sequence are subjected to a second difference to obtain temporal features that reflect the vibration change trend. At the same time, local gray-scale regions are truncated along the normal direction with the candidate edge as the center to form spatial feature input.

[0016] Furthermore, the multi-branch neural network model can be pre-trained using calibrated samples, and supervised learning of the network can be performed using effective edge samples and reflective interference samples, enabling the model to adaptively distinguish between effective edges and interference edges.

[0017] Furthermore, the static position analysis process combines sharpness weights and displacement differences between adjacent frames to perform weighted fusion of the edge positions of each frame, outputting static positions that meet accuracy standards, thereby optimizing the size conversion accuracy.

[0018] Furthermore, the model under test can retrieve the corresponding standard geometric model, standard coordinate system, and upper and lower dimensional limits from the model database, compare each measured value with the corresponding range, output the conclusion of whether the whole part is qualified or unqualified, and at the same time give the number, measurement coordinates, measurement value, and deviation information of the abnormal item.

[0019] The beneficial effects of this invention are:

[0020] 1. This invention uses a jig-bearing module to limit and elastically clamp the umbrella frame, and forms controlled micro-vibration under the preset amplitude micro-vibration obtained by calibration. Combined with the detection start time corresponding to the release completion signal and the preset trigger delay, continuous acquisition is performed, so that the first frame exposure can fall within the effective observation range. With the synchronous emission of pulse light source and short exposure continuous shooting, the imaging stability of the umbrella frame in the mechanical vibration decay period after release is improved, and the measurement fluctuation caused by the inconsistent timing of single frame acquisition is avoided.

[0021] 2. This invention performs grayscale normalization, local contrast enhancement, and candidate edge extraction on continuous images, and performs centroid tracking, second-order difference analysis, and displacement direction analysis on the same candidate edge. This allows for the simultaneous filtering of reflective interference edges using dynamic edge change information and local grayscale distribution information. Furthermore, by combining temporal feature extraction network, spatial feature extraction network, and attention fusion layer for adaptive weighting, the accuracy of effective edge recognition and the robustness of the system are optimized.

[0022] 3. This invention introduces camera calibration parameters, model standard geometric model and size acceptable range in static position determination, and calculates frame weights based on the clarity of local images and displacement changes of adjacent frames. The effective edge positions of continuous frames are weighted and obtained, thereby reducing the impact of local blur, edge discontinuity and instantaneous reflection on the results, and establishing the reliability of the conversion of umbrella rib width, joint gap, skeleton straightness and node offset size. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application and the prior art, the accompanying drawings used in the description of the embodiments and the prior art will be briefly introduced below.

[0024] Figure 1 This is a module architecture diagram of the machine vision-based umbrella frame size defect detection system in an embodiment of this application.

[0025] Figure 2 This is a schematic diagram of the steps of the machine vision-based umbrella frame size defect detection method of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] Please see Figure 1 A machine vision-based umbrella frame dimensional defect detection system; the system includes a controller, and connected to the controller are a fixture support module, a feeding and triggering module, a vision acquisition and light source control module, an image processing module, a neural network analysis module based on feature attention mechanism, a calibration and modeling module, a static position and size analysis module, a defect judgment and result output module, and a model training module; its specific implementation is as follows:

[0028] The fixture bearing module limits and clamps the umbrella frame in a semi-open or fully open state through fixed limiting components and elastic clamping components, and determines the main vibration direction of the umbrella frame, so that the umbrella frame generates a preset amplitude micro-vibration along this direction, which has been calibrated and stored.

[0029] After the external robotic arm is released, the feeding and triggering module adds the release completion signal time to the preset delay stored in the controller to obtain the detection start time.

[0030] The visual acquisition and light source control module acquires the umbrella frame image at the start of the first frame exposure and simultaneously triggers the pulse light source to emit light during the exposure, continuously acquiring 5 to 12 frames of images;

[0031] The image processing module performs grayscale normalization, local contrast enhancement, and candidate edge extraction on each frame of the image, and tracks the centroid position changes of the same candidate edge.

[0032] The neural network analysis module internally deploys a multi-branch neural network model, including a temporal feature extraction network, a spatial feature extraction network, and an attention fusion layer. It is used to input the centroid coordinate quadratic difference sequence and displacement direction sequence into the temporal feature extraction network to extract temporal dynamic features, and input the local grayscale image sequences on both sides of the edge normal to the spatial feature extraction network to extract spatial distribution features. Then, the attention fusion layer adaptively weights the two types of features and outputs the effective edge classification results.

[0033] The calibration and model module provides camera calibration parameters, model standard geometric model, and dimensional acceptable range; the static position and size analysis module determines the effective edge static position and converts the length to obtain umbrella rib width, joint gap, frame straightness, and node offset; the defect judgment and result output module compares the measured values ​​with the corresponding dimensional acceptable range and outputs the inspection results and abnormal locations.

[0034] Furthermore, the fixed limiting component restricts the translation and rotation of the umbrella frame in the measurement plane through the limiting surface, and the elastic clamping component applies a pre-tightening force through the elastic clamping arm and forms a gap that allows micro-vibration. The preset amplitude micro-vibration is obtained by calibrating the vibration displacement after the standard umbrella frame is released.

[0035] The vision acquisition and light source control module ensures that the first frame exposure start time is within a preset trigger delay after the detection start time. The industrial camera outputs an exposure trigger signal to the pulse light source at a preset trigger frequency. The acquisition frame rate is 100-500fps, the exposure time is 20-100μs, and the continuous acquisition window is 40-120ms. The acquisition frame rate, the number of continuous acquisition frames, and the acquisition window are matched with each other. The controller selects the corresponding parameters according to the umbrella frame model and the fixture vibration calibration results, so that 5-12 frames of images cover the effective observation range.

[0036] Through the above settings, this embodiment achieves continuous imaging and effective edge recognition under controlled micro-vibration conditions, thereby improving the stability of umbrella frame size defect detection;

[0037] Furthermore, both the preset delay and the preset trigger delay are obtained by continuously releasing samples from the standard umbrella frame sample under a preset amplitude micro-vibration. The preset delay is taken as the average time when the vibration amplitude first enters the stable range, and the preset trigger delay is taken as the minimum time difference after the first frame exposure falls into the starting point of the stable range. Through the above settings, the detection start time corresponds one-to-one with the effective observation range.

[0038] Furthermore, the temporal feature extraction network of the multi-branch neural network model adopts a combined structure of an input layer, two one-dimensional convolutional layers, a pooling layer, and a fully connected layer. Quadratic difference sequences, The quadratic difference sequence and the displacement direction sequence together constitute a three-channel temporal feature vector after being aligned by frame, and the output is a 64-dimensional temporal feature vector;

[0039] The spatial feature extraction network adopts a combination structure of input layer, two two-dimensional convolutional layers, pooling layer and fully connected layer. Its input is the local grayscale image sequence on both sides of the edge normal, and the output is a 64-dimensional spatial feature vector.

[0040] The attention fusion layer generates weights for the two 64-dimensional features and concatenates them into a 128-dimensional fusion vector. The binary classification output layer then outputs a 2-dimensional classification result, corresponding to two categories: effective edges and reflective interference.

[0041] During training, each sample consists of continuous... The image consists of a frame image, the corresponding temporal feature sequence, the corresponding local grayscale image sequence, and effective edge or reflective interference labels.

[0042] Furthermore, the image processing module maintains a sliding buffer that increments by frame number within the continuous acquisition window. When the current frame enters, it is first written to the end of the buffer and marked as pending judgment.

[0043] The average value of the pixel gradient magnitude in the neighborhood of the candidate edge point set in the current frame is calculated as the candidate edge confidence score. When the candidate edge confidence scores of two consecutive frames are both lower than or equal to the preset confidence threshold, the corresponding edge is marked as reflective and to be removed and is not included in the static position calculation.

[0044] During inference, the multi-branch neural network model outputs three levels of states: single-frame candidate edges, continuous frame sequences, and effective edge results. The multi-branch neural network model connects the temporal branch and the spatial branch in parallel to the auxiliary normalized exponential function classifier to output preliminary probabilities. If the absolute value of the difference between the effective edge probabilities output by the two auxiliary classifiers is less than a preset difference threshold, the fused vector is input into the main classifier for judgment. If the absolute value of the probability difference is greater than or equal to the preset difference threshold, the probability judgment result of the spatial branch is forcibly adopted and a check label is returned for the current frame.

[0045] Furthermore, during the training phase, the three-channel temporal features and local grayscale stripes after frame alignment are used as paired sample inputs. Within each batch, the effective edges and reflective interference samples are kept balanced to avoid the excessive proportion of single-class samples leading to weight bias.

[0046] In this embodiment, the image processing module further constructs a candidate edge point set from the edge points of the same candidate edge, so as to... The centroid is determined by the average coordinates of the candidate edge point set in the frame. The frame number is the number of the continuously acquired frame, and the coordinates of the centroid are... and the centroid displacement vector of adjacent frames Track;

[0047] To each and Calculate the second difference to form a second difference sequence of centroid coordinates; determine the principal vibration direction based on the principal component direction of the displacement vector, and calculate the angle between each displacement vector and the principal vibration direction to form a displacement direction sequence;

[0048] Extending outwards along the edge normal from the candidate edge point as the center, the width is the preset number of pixels retrieved from the controller. The sampling regions constitute a local grayscale image sequence;

[0049] The multi-branch neural network model is pre-trained by the model training module. During training, the calibrated effective edge samples and reflective interference samples, along with their corresponding temporal sequences and image sequences, are obtained to form a training set. The training set is then input into the initial multi-branch neural network model. The classification error is calculated using the cross-entropy loss function, and the node weights of the temporal feature extraction network, spatial feature extraction network, and attention fusion layer are updated through backpropagation until the classification error meets the convergence condition.

[0050] The camera calibration parameters are used to establish the transformation relationship between the industrial camera imaging coordinate system and the umbrella frame measurement coordinate system, and the coordinate axis directions of the umbrella frame measurement coordinate system correspond to the standard coordinate system of the umbrella frame model.

[0051] The static position and size analysis module uses a preset neighborhood centered on the effective edge as the local image and calculates the Laplacian operator variance for each frame of the local image. And based on the normalized variance of the Laplace operator And the absolute value of the displacement difference between adjacent frames obtained based on the centroid displacement vectors of adjacent frames. ,in displacement vector The modulus length, i.e. Determine image weights , and then Indicates the effective edge position of the k-th frame, according to Determine the static position; where, The summation symbol indicates that the summation is performed on consecutive frames involved in the calculation;

[0052] Furthermore, preset pixel count Used to control the width of the sampling area. Used to reflect the sharpness of a local image Used to reflect the magnitude of edge position changes between adjacent frames;

[0053] With the above settings, this embodiment combines continuous frame displacement information with grayscale distribution on both sides of the edge for effective edge screening, and improves the ability to distinguish between real edges and reflective interference edges through the trained multi-branch neural network model, thereby providing a more stable positional basis for subsequent size conversion;

[0054] Furthermore, grayscale normalization is adopted. This method maps pixel grayscale values ​​to the range of 0 to 1, where... This is the original grayscale value of the current pixel. and These are the minimum and maximum gray values ​​of the candidate region in the current frame, respectively. To prevent the use of pre-defined positive real numbers with a denominator of 0;

[0055] The attention fusion layer first concatenates the two 64-dimensional features, calculates the adaptive weight allocation matrix of the two types of features through a multilayer perceptron and a normalized exponential function, performs Hadamard product operation on the weight allocation matrix with the corresponding 64-dimensional feature vectors and concatenates them into a 128-dimensional fusion vector, and outputs a 2-dimensional effective edge classification result through a fully connected layer.

[0056] Furthermore, the temporal branch input of the multi-branch neural network model is a three-channel temporal feature with a length aligned to the number of frames, and the spatial branch input is a feature with a width of... The local grayscale strip image outputs a 2D probability vector during inference, corresponding to two categories: effective edges and reflective interference.

[0057] Furthermore, when the same candidate edge becomes discontinuous in adjacent frames and the centroid cannot be continuously tracked, short-term compensation is first performed on the current frame using the displacement direction of the previous stable frame; if a continuous sequence still cannot be formed after compensation, the edge is marked as a trajectory interruption and removed. The weighted set retains only frames that meet the continuity requirement for static position determination; for The weighted result is calculated by using the median frame position as the static position when all frames are removed or the weighted sum is 0, in order to avoid distortion of the result caused by reflections or occlusions exceeding the preset intensity threshold.

[0058] Please see Figure 2 A machine vision-based method for detecting dimensional defects in umbrella frames; the method includes: acquiring the release completion signal after the external robot arm releases the umbrella frame, adding the corresponding time of the signal to the model preset delay called by the controller to obtain the detection start time; and using fixed limiting parts and elastic clamping parts to make the umbrella frame in the semi-open or fully open state generate micro-vibrations along the main vibration direction determined by the fixture structure, which have been calibrated and determined.

[0059] When the first frame exposure start time is within the preset trigger delay after the detection start time, the industrial camera and pulse light source are synchronously triggered according to the preset trigger frequency, and 5 to 12 frames of images are continuously acquired, with an acquisition window of 40 to 120 ms.

[0060] Each frame of the image is normalized in grayscale, local contrast is enhanced and candidate edges are extracted to form a set of edge points for the same candidate edge and the centroid of each frame is calculated. The centroid is tracked to obtain the second difference sequence of centroid coordinates and the displacement direction sequence, and the local grayscale image sequence on both sides of the edge normal is extracted.

[0061] The centroid coordinate quadratic difference sequence and displacement direction sequence are input into the temporal feature extraction network of the multi-branch neural network model to extract temporal dynamic features. The local grayscale image sequence is input into the spatial feature extraction network of the multi-branch neural network model to extract spatial distribution features. After the two types of features are weighted and concatenated by the attention fusion layer inside the multi-branch neural network model, the effective edges are determined and output.

[0062] In the umbrella frame measurement coordinate system, the effective edge static position is determined, and the umbrella rib width, joint gap, frame straightness and node offset are calculated after calibration transformation. The standard geometric model and dimensional acceptance range corresponding to the model to be tested are called, and the defect conclusion and abnormal location are output.

[0063] In this embodiment, the horizontal and vertical directions of the industrial camera's imaging plane are further used as the horizontal and vertical coordinate axes, respectively, with the first... The centroid is determined by the average coordinates of the candidate edge point set in the frame. The sequence number of the continuously acquired frames, i.e. And determine the displacement vector based on the centroid of adjacent frames. ;

[0064] To each and The second difference is calculated along the frame sequence to form the second difference sequence of the centroid coordinates; the displacement direction sequence is formed by the angle between the displacement vector and the principal vibration direction.

[0065] Calculate the distance between the static positions of the two relatively effective edges of the same umbrella rib, and obtain the umbrella rib width through calibration transformation; along the direction of the line connecting adjacent riveting nodes in the model standard coordinate system, calculate the projection distance of the relative contour edges of adjacent riveting nodes in the direction of the line connecting the two, and obtain the joint gap;

[0066] Using the corresponding center points of the two relatively effective edges of the same umbrella rib as fitting points, the least squares method is used to fit the reference line, and the straightness of the skeleton is determined by the maximum distance from each fitting point to the reference line.

[0067] The theoretical reference position of the riveting node is determined from the node coordinates in the standard geometric model of the model. The center of the riveting node is determined by the node profile. The difference in the lateral coordinates and the difference in the longitudinal coordinates between the center and the theoretical reference position are calculated to obtain the node offset.

[0068] Obtain the model of the umbrella frame to be tested, and retrieve the standard geometric model, standard coordinate system, theoretical reference position of riveting node, and lower and upper limits of each dimension item from the umbrella frame model database;

[0069] The umbrella rib width, joint gap, frame straightness, and node misalignment are compared with the corresponding lower and upper limits of the dimensions, respectively. When the measured value is between or equal to the lower and upper limits of the dimensions, it is considered qualified; when it exceeds the corresponding lower or upper limit of the dimensions, it is considered unqualified.

[0070] If all dimensions are qualified, output the conclusion that the whole part is qualified; otherwise, output the conclusion that the whole part is unqualified. Also output the umbrella rib number or riveting node number, measurement coordinates, measurement value, standard upper and lower limits, difference between measurement value and standard value, and judgment result for the abnormal dimension item.

[0071] Furthermore, the centroid coordinate quadratic difference sequence is used to reflect whether the edge position change is continuous, the displacement direction sequence is used to reflect whether the motion direction of the same edge is stable in consecutive frames, and the standard geometric model is used to provide the structural reference corresponding to the model.

[0072] Using the above method, this embodiment uses edge changes in continuous frames to filter out interfering edges, and completes effective edge screening, size conversion and result output under controlled micro-vibration and short exposure continuous shooting conditions, thereby giving a qualified and unqualified judgment and the corresponding abnormal position;

[0073] Furthermore, each size item is judged independently. If any item exceeds the lower or upper limit of the corresponding size, it is judged as unqualified and is not offset by the qualified results of other items. When multiple abnormal items exist in the same umbrella frame at the same time, the corresponding umbrella rib number or riveting node number and the direction of exceeding the limit are output separately to avoid the conclusion of a single whole piece from covering local extreme defects.

[0074] Furthermore, the multi-branch neural network model takes three-channel temporal features aligned with the frame number as input to the temporal branch during method execution, and the spatial branch input is a local grayscale image of the edge normal. The output is a 2-dimensional probability vector, corresponding to two categories: effective edges and reflective interference. This allows the effective edge selection results to be directly mapped to the subsequent size measurement process.

[0075] Furthermore, the method switches states in the following order during execution: release completion confirmation, delay timing, trigger acquisition, edge filtering, size conversion, and conclusion output. Each state is entered with the completion flag of the previous state as the entry condition.

[0076] When frame loss or exposure timeout occurs during the acquisition process, the controller will forcibly discard the image sequence of the current window because the physical micro-vibration state has decayed over time, and send a secondary release command to the fixture support module to re-excite the umbrella frame and synchronously restart the corresponding detection acquisition life cycle to ensure the temporal consistency of the physical measurement environment.

[0077] The standard geometric model, standard coordinate system, and upper and lower limits of dimensions in the umbrella frame model database are first checked for consistency with the current model after being called. If the model number does not match the standard coordinate system, the status of pending review is directly output without entering the dimension judgment.

[0078] Furthermore, when outputting abnormal locations, results are generated by matching the umbrella rib number or riveting node number with the measurement coordinates one-to-one, thus avoiding location confusion when multiple abnormal items overlap in the same frame.

[0079] For any part not mentioned in this invention, existing technologies can be used or referenced. Of course, the above description is not a limitation of this invention, and this invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of this invention should also be within the protection scope of this invention.

Claims

1. A machine vision-based umbrella frame dimensional defect detection system, characterized in that, include: The fixture carrying module includes a fixed limiting component and an elastic clamping component, which limit and clamp the umbrella frame in a semi-open or fully open state. The structural orientation of the two components determines the main vibration direction of the umbrella frame. The elastic clamping component causes the umbrella frame to generate a preset amplitude micro-vibration along the main vibration direction. The feeding and triggering module acquires the release completion signal and adds the time corresponding to the release completion signal to the preset delay of the called model to obtain the detection start time; The vision acquisition and light source control module synchronously controls the industrial camera and pulse light source to perform exposure at the start of the first exposure frame within a preset trigger delay after the detection start time, and continuously acquires 5 to 12 frames of images. The image processing module performs grayscale normalization, local contrast enhancement, and candidate edge extraction on the image. It tracks the change in the centroid position of the same candidate edge and calculates the centroid coordinate quadratic difference sequence and displacement direction sequence. At the same time, it extracts the local grayscale image sequence on both sides of the edge normal. The neural network analysis module has a built-in multi-branch neural network model, including a temporal feature extraction network, a spatial feature extraction network, and an attention fusion layer; it inputs the centroid coordinate quadratic difference sequence and displacement direction sequence into the temporal feature extraction network to obtain temporal dynamic features; The spatial distribution features are obtained by inputting the local grayscale image sequences on both sides of the edge normal into the spatial feature extraction network. The two features are weighted and concatenated through an attention fusion layer, and an effective edge classification result is output. The calibration and model module provides camera calibration parameters, standard geometric models, and dimensional tolerances. The static position and size analysis module determines the effective edge static position and calculates the umbrella rib width, joint gap, frame straightness and node offset from the effective edge static position; The defect judgment and result output module compares the measured values ​​of each dimension with the acceptable range and outputs the inspection results and abnormal locations.

2. The machine vision-based umbrella frame size defect detection system according to claim 1, characterized in that, The fixed limiting component includes limiting surfaces that abut against the reference parts of the umbrella frame, and the limiting surfaces restrict the translation and rotation of the umbrella frame in the measurement plane; the elastic clamping component includes elastic clamping arms that are arranged opposite to each other along the main vibration direction, the elastic clamping arms apply a pre-tightening force to the umbrella frame, and form a gap with the limiting surfaces that allows the umbrella frame to vibrate; the preset amplitude micro-vibration is obtained by calibrating the vibration displacement of a standard umbrella frame after release and is pre-stored.

3. The machine vision-based umbrella frame size defect detection system according to claim 1, characterized in that, The vision acquisition and light source control module ensures that the first frame exposure start time is within a preset trigger delay after the detection start time. The industrial camera in the vision acquisition and light source control module outputs an exposure trigger signal to the pulse light source according to a preset trigger frequency. The acquisition frame rate of the industrial camera is 100-500fps, the exposure time is 20-100μs, and the frame interval is determined by the acquisition frame rate. The continuous acquisition window is the time interval between the start of the first frame exposure and the end of the last frame exposure, and it is 40 to 120 ms.

4. The machine vision-based umbrella frame dimensional defect detection system according to claim 1, characterized in that, The image processing module constructs a candidate edge point set from the edge points of the same candidate edge, using the first edge point as the first edge point. The centroid is determined by the average coordinates of the candidate edge point set in the frame. The frame number is the number of the continuously acquired frames, and the coordinates of the centroid are... and the centroid displacement vector of adjacent frames Track; To each and The second difference is calculated to form the second difference sequence of the centroid coordinates; The displacement direction sequence is formed by calculating the angle between each displacement vector and the main vibration direction; the width of the distance extended along the edge normal from the candidate edge point to both sides is the preset number of pixels called from the controller. The sampling regions constitute the local grayscale image sequence.

5. The machine vision-based umbrella frame dimensional defect detection system according to claim 1, characterized in that, The multi-branch neural network model is pre-trained by the model training module; The model training module obtains calibrated effective edge samples and reflective interference samples, as well as their corresponding centroid coordinate quadratic difference sequences, displacement direction sequences, and local grayscale image sequences to form a training set. The training set is input into the initial multi-branch neural network model. The classification error between the classification result output by the forward propagation of the network and the true label of the sample is calculated using the cross-entropy loss function. The node weights of the temporal feature extraction network, spatial feature extraction network and attention fusion layer are updated through the backpropagation algorithm until the classification error meets the convergence condition, thus obtaining the trained multi-branch neural network model.

6. The machine vision-based umbrella frame dimensional defect detection system according to claim 4, characterized in that, The camera calibration parameters establish the transformation relationship between the industrial camera imaging coordinate system and the umbrella frame measurement coordinate system, and the coordinate axis directions of the umbrella frame measurement coordinate system correspond to the standard coordinate system of the umbrella frame model. The static position and size analysis module uses a preset neighborhood centered on the effective edge as a local image, and calculates the Laplacian operator variance of each frame of the local image. And based on the normalized variance of the Laplace operator. And the absolute value of the displacement difference between adjacent frames obtained based on the centroid displacement vectors of adjacent frames. ,in The displacement vector The modulus length, i.e. Determine image weights , and then Indicates the first Frame valid edge position, according to Determine the static position.

7. A machine vision-based method for detecting dimensional defects in umbrella frames, applied to the machine vision-based umbrella frame dimensional defect detection system according to any one of claims 1-6, characterized in that, include: The release completion signal after the external robotic arm releases the umbrella frame is obtained, and the corresponding time of the signal is added to the model preset delay called by the controller to obtain the detection start time; By using fixed limiting components and elastic clamping components, the umbrella frame in a semi-open or fully open state generates micro-vibrations along the main vibration direction determined by the fixture structure, which are calibrated and determined. When the first exposure start time is within the preset trigger delay after the detection start time, the industrial camera and pulse light source are synchronously triggered according to the preset trigger frequency, and 5 to 12 frames of images are continuously acquired, with an acquisition window of 40 to 120 ms. Each frame of the image is normalized in grayscale, local contrast is enhanced and candidate edges are extracted to form a set of edge points for the same candidate edge and the centroid of each frame is calculated. The centroid is tracked to obtain the second difference sequence of centroid coordinates and the displacement direction sequence, and the local grayscale image sequence on both sides of the edge normal is extracted. The centroid coordinate quadratic difference sequence and displacement direction sequence are input into the temporal feature extraction network of the multi-branch neural network model to extract temporal dynamic features. The local grayscale image sequence is input into the spatial feature extraction network of the multi-branch neural network model to extract spatial distribution features. After the two types of features are weighted and concatenated by the attention fusion layer inside the multi-branch neural network model, the effective edges are determined and output. In the umbrella frame measurement coordinate system, the effective edge static position is determined, and the umbrella rib width, joint gap, frame straightness and node offset are calculated after calibration transformation. The standard geometric model and dimensional acceptance range corresponding to the model to be tested are called, and the defect conclusion and abnormal location are output.

8. The machine vision-based method for detecting dimensional defects in umbrella frames according to claim 7, characterized in that, Using the horizontal and vertical directions of the industrial camera's imaging plane as the horizontal and vertical coordinate axes respectively, and taking the first... The centroid is determined by the average coordinates of the candidate edge point set in the frame. The sequence number of the continuously acquired frames, i.e. And determine the displacement vector based on the centroid of adjacent frames. ; To each and The second difference is calculated along the frame sequence to form the second difference sequence of the centroid coordinates; The displacement direction sequence is formed by the angle between the displacement vector and the main vibration direction.

9. The machine vision-based method for detecting dimensional defects in umbrella frames according to claim 7, characterized in that, Calculate the distance between the static positions of two relatively effective edges of the same umbrella rib, and obtain the umbrella rib width through calibration conversion; Along the direction of the line connecting adjacent riveting nodes in the model standard coordinate system, calculate the projection distance of the relative contour edges of adjacent riveting nodes in the direction of the line connecting them to obtain the joint clearance; Using the corresponding center points of the two relatively effective edges of the same umbrella rib as fitting points, the least squares method is used to fit the reference straight line, and the straightness of the skeleton is determined by the maximum distance from each fitting point to the reference straight line. The theoretical reference position of the riveting node is determined from the node coordinates in the standard geometric model of the model. The center of the riveting node is determined by the node profile. The difference in the lateral coordinates and the difference in the longitudinal coordinates between the center and the theoretical reference position are calculated to obtain the node offset.

10. The machine vision-based method for detecting dimensional defects in umbrella frames according to claim 9, characterized in that, Obtain the model of the umbrella frame to be tested, and retrieve the standard geometric model, standard coordinate system, theoretical reference position of the riveting node, and lower and upper limits of each dimension item associated with the model from the umbrella frame model database; The umbrella rib width, joint gap, frame straightness, and node misalignment are compared with the corresponding lower and upper limits of the dimensions, respectively. When the measured value is between or equal to the lower and upper limits of the dimensions, it is considered qualified; when it exceeds the corresponding lower or upper limit of the dimensions, it is considered unqualified. If all dimensions are within acceptable limits, output the conclusion that the whole part is acceptable; otherwise, output the conclusion that the whole part is unacceptable. Also output the umbrella rib number or riveting node number, measurement coordinates, measurement value, standard upper and lower limits, difference between measurement value and standard value, and judgment result for the abnormal dimension item.

Images