SPR (Surface Plasmon Resonance) online detection control method before riveting

Abstract

The invention discloses an SPR (Surface Plasmon Resonance) online pre-riveting detection control method, which relates to the technical field of riveting and comprises the following steps of: conveying rivets from a stock bin or an automatic feeding mechanism to a riveting area according to a preset initial feeding speed; and when the rivet enters the feeding channel and reaches the visual range of the cameras, the appearance information of the rivet is synchronously collected through the cameras installed at different positions and at different angles. According to the method, multi-angle visual inspection, deep learning deformation analysis and dynamic feeding adjustment are combined, three-dimensional reconstruction and standard comparison of rivets can be completed in a short time, and deformation is accurately quantified. Compared with fixed high-speed feeding, the method has the advantages that abnormal rivets can be intervened in real time, secondary impact is reduced by reducing the feeding speed, the detection time is prolonged, and the deformation recognition precision is improved, so that the rivet clamping risk and batch defective products are reduced, potential safety hazards caused by insufficient riveting strength are prevented, the reliability of a production line is improved, and the reworking cost is reduced; and stable riveting quality is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of riveting, and in particular to an SPR online pre-riveting detection control method. Background Art

[0002] Online pre-riveting detection and control for self-piercing riveting (SPR) is an intelligent control method based on real-time data monitoring and AI optimization, designed to improve riveting quality and stability. The process involves: first, torque is monitored throughout the riveting process, and initial torque is obtained through a pressure sensor to confirm sheet thickness. When the pressure sensor detects significant fluctuations—i.e., force changes exceeding a set threshold—the system verifies the rivet length. The servo system reads the shank position, and the rivet length is calculated using the difference in readings measured by the magnetic scale. Subsequently, the AI ​​model calculates and adjusts the optimal trigger and stop positions during the riveting process based on the detected sheet thickness changes to ensure riveting quality. Finally, the system feeds the parameters collected during the process back into the AI ​​model, analyzes their impact on the riveting effect, and determines whether the result is positive (optimization direction) or negative (requires adjustment). This enables adaptive optimization and improves riveting quality and stability.

[0003] The existing technology has the following deficiencies: To improve riveting efficiency, existing technologies typically set the rivet feed speed high and fixed. However, when a rivet undergoes slight deformation during the automatic feeding process, the high feed speed makes it difficult for the system to detect this anomaly in a timely manner, causing the rivet to become stuck (a "stuck" phenomenon). If a deformed rivet enters the riveted area, effective riveting strength may not be achieved, posing a serious structural safety hazard. If the equipment continues to operate according to normal procedures, subsequent batches of workpieces may have quality defects that are difficult for operators to detect in a short period of time, ultimately resulting in large-scale product rework or scrap, increasing production costs and affecting product reliability.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a SPR online pre-riveting detection control method, which combines multi-angle visual inspection, deep learning deformation analysis and dynamic feeding adjustment, so that the system can complete the three-dimensional reconstruction of the rivet and compare it with the standard model in a short time, and accurately quantify the degree of deformation. Compared with the traditional fixed high-speed feeding method, this method can achieve real-time intervention on abnormal rivets, reduce the secondary impact on the deformed rivets by reducing the feeding speed, and at the same time increase the detection time and improve the image acquisition accuracy, thereby improving the accuracy and stability of deformation detection. This not only reduces the risk of "stuck nails" and the production of batch defective products, but also effectively prevents structural safety hazards caused by insufficient riveting strength, ultimately improving the reliability of the production line, reducing rework costs, and ensuring the quality stability of riveted products, in order to solve the problems in the above-mentioned background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: an SPR online pre-riveting detection control method, comprising the following steps: The rivets are transported from the hopper or automatic feeding mechanism to the riveting area according to the pre-set initial feeding speed; When the rivet enters the feeding channel and reaches the visual range of the camera, the cameras installed at different positions and angles synchronously collect the rivet's shape information to generate all-round, multi-angle image data; After completing multi-angle image acquisition, the acquired images are fused to obtain a preliminary 3D model of the rivet. Subsequently, the preliminary 3D model of the rivet is aligned with the pre-stored standard 3D rivet model to place the actual rivet shape and the standard theoretical shape in the same reference frame. After successful registration, the difference features between the preliminary 3D model of the rivet and the standard 3D model are extracted and further analyzed in depth. The analyzed features are input into a trained and operational deep learning model, which then outputs an assessment result on the rivet deformation risk and quantifies the rivet deformation. When it is determined that there is a risk of rivet deformation, the actual feeding speed is automatically adjusted according to the degree of deformation of the rivet. By reducing the feeding speed, the deformed rivet is prevented from further impact or extrusion, which aggravates the deformation. At the same time, it gives the camera and image processing algorithm more time to acquire and analyze more rivet image data.

[0007] Preferably, performing data fusion on the collected multi-angle images includes the following steps: Preprocess images from different cameras and angles, including denoising, illumination normalization, and distortion correction, to ensure that all images are fused under the same visual conditions; Use feature point matching to identify and associate key feature points from different perspectives to ensure that multiple images can be correctly aligned; Using multi-view stereo reconstruction technology, images from different perspectives are converted into 3D point clouds or depth maps, and the camera projection matrix is ​​adjusted through iterative optimization; Use the ICP algorithm to fuse multi-angle point cloud data into a unified 3D model, eliminating redundancy and error points; Surface reconstruction and optimization, as well as smoothing, are performed to obtain a complete and accurate rivet three-dimensional model.

[0008] Preferably, registering the obtained preliminary three-dimensional model of the rivet with a pre-stored standard three-dimensional model of the rivet comprises the following steps: The two are preliminarily aligned based on the coarse registration algorithm of feature points; A rigid registration algorithm is used to globally align the key geometric features of the rivets; The ICP algorithm is used to gradually optimize the matching of point clouds or meshes so that the preliminary 3D model of the rivet and the standard 3D rivet model are aligned with the minimum error in 3D space. Local adjustments are then made to further optimize the alignment results based on fine-grained features.

[0009] Preferably, difference features between the preliminary three-dimensional model of the rivet and the standard three-dimensional model are extracted, wherein the extracted features include the offset of multiple cross-sectional centers of the rivet relative to the cross-sectional center of the standard rivet and the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model. The extracted features are further deeply analyzed to generate cross-sectional eccentricity reference values ​​and normal angle offset reference values, respectively. The analyzed cross-sectional eccentricity reference values ​​and normal angle offset reference values ​​are input into a deep learning model that has been trained and put into use to output a deformation coefficient. The deformation coefficient is used to evaluate the risk of rivet deformation and quantify the deformation of the rivet.

[0010] Preferably, the deformation coefficient generated when evaluating the rivet deformation risk by the deep learning model that has been trained and put into use is compared and analyzed with a preset deformation coefficient reference threshold to evaluate the deformation risk of the rivet. The specific steps are as follows: If the deformation coefficient is greater than the deformation coefficient reference threshold, the rivet is judged as a deformation risk rivet, and subsequent operations are performed on the deformation risk rivet; if the deformation coefficient is less than or equal to the deformation coefficient reference threshold, the rivet is judged as a normal rivet, and the normal rivet is transferred to the riveting area for riveting operation.

[0011] Preferably, when it is determined that there is a risk of deformation of the rivet, the actual feeding speed is automatically adjusted according to the degree of deformation of the rivet. The specific steps are as follows: After the rivet deformation detection is completed, the deformation degree of the rivet is evaluated according to the output deformation coefficient of the deep learning model. The severity of the rivet deformation is judged by the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated by the deformation coefficient and the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated using a nonlinear adjustment function. The calculation expression is: ,in: is the deformation coefficient of the rivet, which indicates the degree of deformation of the rivet. The reference threshold value of the deformation coefficient defines the maximum allowable deformation range. Rivets exceeding the reference threshold value of the deformation coefficient need to adjust the feeding speed. is the deformation sensitivity coefficient, which is used to control the influence of deformation on the adjustment of feeding speed. is the feeding speed adjustment factor, the value range is .

[0012] Preferably, the feed speed adjustment factor is calculated Afterwards, the actual feeding speed is calculated based on the current initial feeding speed. The calculation expression is: ,in: is the initial feeding speed, which is the default feeding speed under normal circumstances. is the minimum feeding speed, is the actual feeding speed, which is the final feeding speed after dynamic adjustment based on the deformation situation.

[0013] Preferably, the specific steps of further analyzing the offsets of the centers of multiple cross sections of the rivet relative to the center of the standard rivet cross section to generate a cross section eccentricity reference value are as follows: For multiple key sections of the rivet, obtain their center coordinates and calculate the offset of multiple key sections relative to the center of the corresponding section of the standard rivet. Suppose the center point of the rivet section is , the corresponding cross-section center of the standard rivet is , then the calculation expression of the center offset of each section is: ,in: For the i The center offset of each section, For the actual rivet i The coordinates of the center point of the cross section, Standard rivet i The coordinates of the center point of each cross section; The offsets of all sections are normalized to eliminate the influence of rivets of different sizes. The normalized offset is defined as the expression for normalization: ,in: is the normalized offset, L is the total length of the rivet to ensure comparability of rivet offset data for different specifications; After calculating the normalized offset Then, the importance weight of the cross section in the overall structure of the rivet is further combined to generate the cross section eccentricity reference value. The generated expression is: ,in: It is the reference value of cross-section eccentricity, which is an indicator to comprehensively measure the degree of cross-section eccentricity of rivets. is the cross-sectional weight coefficient, For the i The axial distance from the cross section to the bottom of the rivet, is the nonlinear attenuation factor.

[0014] Preferably, the specific steps of further analyzing the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model to generate a normal angle offset reference value are as follows: After the rivet 3D model is aligned with the standard model, for each vertex , respectively obtain the normal vector of the actual rivet vertex and the normal vector of the corresponding point of the standard rivet, and calculate the angle between the two as the offset angle. The calculation expression is: ,in: is the normal vector of the actual rivet vertex, is the normal vector of the corresponding point of the standard rivet, It is the angle between the normal vector of the actual rivet vertex and the normal vector of the corresponding point of the standard rivet, reflecting the degree of local deformation of the point. Represents the vector dot product, also known as the scalar product; Based on the Gaussian curvature of the surface, weighting is performed to make the weight of the deformation-significant area higher. The weighted expression is: ,in: The actual rivet vertex The Gaussian curvature at , which measures the degree of local curvature, is a regulating factor used to control the influence of curvature on weight; Split the rivet surface into k regions, each region Contains multiple vertices , calculate the reference value of the normal angle offset of the rivet surface, the calculation expression is: ,in: Indicates the reference value of the normal angle offset of the rivet surface, For the region The local normal angle deviation metric is used to measure the overall deviation degree of the area.

[0015] In the above technical solution, the technical effects and advantages provided by the present invention are: The present invention combines multi-angle visual inspection, deep learning deformation analysis, and dynamic feeding adjustment, enabling the system to complete the three-dimensional reconstruction of the rivet and compare it with the standard model in a short period of time, accurately quantifying the degree of deformation. Compared with the traditional fixed high-speed feeding method, this method can achieve real-time intervention in abnormal rivets, reduce the secondary impact on the deformed rivets by reducing the feeding speed, and at the same time increase the detection time and improve the image acquisition accuracy, thereby improving the accuracy and stability of deformation detection. This not only reduces the risk of "stuck nails" and the production of batch defective products, but also effectively prevents structural safety hazards caused by insufficient riveting strength, ultimately improving the reliability of the production line, reducing rework costs, and ensuring the quality stability of riveted products. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0017] Figure 1 This is a method flow chart of an SPR online pre-riveting detection control method of the present invention. DETAILED DESCRIPTION

[0018] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0019] The present invention provides Figure 1 The SPR online pre-riveting detection control method shown includes the following steps: The rivets are transported from the hopper or automatic feeding mechanism to the riveting area according to the pre-set initial feeding speed; At this point, no deformation analysis has been performed on the rivets so that feeding can be completed quickly under normal conditions.

[0020] When the rivet enters the feeding channel and reaches the visual range of the camera, the cameras installed at different positions and angles synchronously collect the rivet's shape information to generate all-round, multi-angle image data; Because rivets rotate and tilt during feeding, multi-directional cameras capture subtle changes in all directions, including surface deformation and curvature. This step provides detailed data for subsequent 3D reconstruction and comparison, avoiding information loss or errors caused by a single angle.

[0021] After completing multi-angle image acquisition, the collected images are fused to obtain a preliminary 3D model of the rivet. Subsequently, the preliminary 3D model of the rivet is aligned with the pre-stored standard 3D model of the rivet, so that the actual rivet shape and the standard theoretical shape are placed in the same reference frame, providing an accurate comparison basis for subsequent difference detection. Data fusion of multi-angle images involves several key steps. First, preprocessing is performed on images from different cameras and angles, including denoising, illumination normalization, and distortion correction, to ensure that all images are fused under the same visual conditions. Next, key feature points from different viewpoints are identified and associated using algorithms such as SIFT and ORB or deep learning methods, ensuring proper alignment of the multiple images. Multi-view stereo (MVS) or point cloud stitching techniques are then used to convert the images from different viewpoints into 3D point clouds or depth maps. The camera projection matrix is ​​then adjusted through iterative optimization to ensure the accuracy of the point cloud data. Subsequently, the multi-angle point cloud data is fused into a unified 3D model using the Iterative Closest Point (ICP) algorithm or other registration algorithms, eliminating redundant and inaccurate points. Finally, surface reconstruction and optimization are performed, such as using Poisson surface reconstruction or the Marching Cubes algorithm to generate a high-precision mesh model. This mesh is then smoothed to produce a complete and accurate 3D model of the rivet. This process ensures that information from different angles can be effectively integrated, providing high-precision morphological reconstruction results and laying the foundation for subsequent deformation detection.

[0022] The resulting preliminary 3D rivet model is registered with a pre-stored standard 3D rivet model. This process involves the following steps: First, a preliminary alignment is performed, typically using a coarse registration algorithm based on feature points (such as PCA or matching based on SIFT / ORB) to ensure consistent overall orientation. Next, a rigid registration algorithm (such as the Kabsch algorithm or Umeyama transform) is used to globally align the rivet's key geometric features (such as the end, cylinder, and rivet cap) to minimize initial errors. Subsequently, an ICP (Iterative Closest Point) algorithm or deep learning registration method is used to gradually optimize the matching of the point cloud or mesh, achieving minimal alignment of the preliminary rivet model with the standard 3D rivet model in 3D space. Local fine-tuning is then performed to further refine the registration results based on fine-grained features (such as surface curvature and key cross-sectional contours) to ensure a high degree of detail alignment between the rivet and the standard model. This registration process ensures that the actual rivet shape can be accurately compared with the standard shape in the same coordinate system, facilitating the identification of subtle deformations.

[0023] After successful registration, the difference features between the preliminary 3D model of the rivet and the standard 3D model are extracted and further analyzed in depth. The analyzed features are input into a trained and operational deep learning model, which then outputs an assessment result on the rivet deformation risk and quantifies the rivet deformation. The difference features between the preliminary 3D model of the rivet and the standard 3D model are extracted, where the extracted features include the offset of the centers of multiple rivet sections (such as the rivet head section, transition section section, and rivet tail section) relative to the center of the standard rivet section and the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point in the standard model. The extracted features are further analyzed in depth to generate cross-section eccentricity reference values ​​and normal angle offset reference values, respectively. The analyzed cross-section eccentricity reference values ​​and normal angle offset reference values ​​are input into a deep learning model that has been trained and put into use. The deformation coefficient is output by the model, and the risk of rivet deformation is evaluated based on the deformation coefficient, so as to quantify the deformation of the rivet.

[0024] If the centers of multiple rivet sections (such as the rivet head, transition section, and tail section) deviate significantly from the center of the standard rivet section after alignment, this typically indicates a risk of rivet deformation. Specifically, under normal conditions, each rivet section should maintain high alignment with the standard model, and the center offset should be within the allowable tolerance. However, if the centers of certain key sections exhibit significant offset, this may indicate bending, deflection, or localized uneven loading during production, storage, or feeding. For example, a large offset in the rivet head section may indicate uneven compression or impact during feeding; an abnormal offset in the transition section section may indicate overall bending of the rivet axis; and a large offset in the tail section may prevent the rivet from properly aligning during riveting, impacting the final joint quality. Because the structural integrity of the rivet directly determines the strength of the joint, abnormal cross-sectional offsets often lead to problems such as inaccurate rivet positioning, uneven loading, and reduced joint quality.

[0025] The specific steps for further in-depth analysis of the offset of multiple cross-section centers of rivets relative to the standard rivet cross-section center to generate cross-section eccentricity reference values ​​are as follows: First, for multiple key sections of the rivet (such as the rivet head section, transition section section, and rivet tail section), obtain their center point coordinates and calculate their offsets relative to the center of the corresponding section of the standard rivet. Suppose the center point of the rivet section is , the corresponding cross-section center of the standard rivet is , then the calculation expression of the center offset of each section is: ,in: For the iThe center offset of each section, For the actual rivet i The coordinates of the center point of the cross section, Standard rivet i The coordinates of the center point of each cross section; Next, the offsets of all sections are normalized to eliminate the influence of rivets of different sizes. The normalized offset is defined as the expression for normalization: ,in: is the normalized offset, L is the total length of the rivet to ensure comparability of rivet offset data for different specifications; The core goal of this step is to accurately obtain the center offset of each section and normalize it so that different rivets can be evaluated consistently during the inspection process without being affected by absolute size.

[0026] After calculating the normalized offset Then, the importance weight of the cross section in the overall structure of the rivet is further combined to generate the cross section eccentricity reference value. The generated expression is: ,in: It is the reference value of cross-section eccentricity, which is an indicator to comprehensively measure the degree of cross-section eccentricity of the rivet. The larger the value, the more serious the deformation. is the cross-section weight coefficient. Different cross-sections have different effects on the overall structure of the rivet. For example, the rivet head and transition section have a greater impact on the riveting quality. Should be higher, For the i The axial distance from the cross section to the bottom (tail) of the rivet, It is a nonlinear attenuation factor, which makes the contribution of the section farther away from the tail to the reference value of the section eccentricity gradually decrease, so as to avoid the abnormality of the head and transition section being masked by the slight deviation of the tail section; This step uses an exponential decay function to assign different degrees of influence to different cross-sections, so that the offset of key parts of the rivet (such as the head and transition section) has a greater impact on the final cross-section eccentricity reference value, while weakening the offset effect away from the riveted area, thereby improving the accuracy and stability of the detection.

[0027] The cross-sectional eccentricity reference value, generated through further in-depth analysis of the offsets of the rivet's multiple cross-sectional centers relative to the standard rivet's cross-sectional center, indicates that larger performance values ​​indicate larger offsets of the rivet's multiple cross-sectional centers relative to the standard rivet's cross-sectional center, reflecting a higher risk of rivet deformation. Conversely, smaller cross-sectional eccentricity reference values ​​indicate a closer match between the rivet's cross-sectional centers and the standard rivet, resulting in a lower risk of deformation. Specifically, the cross-sectional eccentricity reference value calculates the geometric eccentricity of each key rivet section by weighting the offsets and introduces an exponential decay factor, so that offsets near the rivet head and transition section have a greater impact on the final cross-sectional eccentricity reference value, while the impact of offsets away from the riveted area is weakened.

[0028] In the registered rivet three-dimensional model, the angle between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model is greatly offset, which usually indicates that the rivet is at risk of deformation. This is because the normal vector reflects the local geometric direction of the object surface. If the normal direction of each vertex on the surface of a normal rivet in three-dimensional space is consistent with or close to that of the standard model, it means that the geometric shape of the rivet has not deviated significantly. However, when the normal angle increases abnormally, it indicates that the rivet surface has abnormal bulges, depressions, twists or bends in local areas, causing the surface orientation of the area to change significantly. This situation may be caused by plastic deformation caused by external force extrusion, impact or stress concentration during feeding, storage or processing.

[0029] The specific steps for further in-depth analysis of the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point on the standard model to generate a reference value for the normal angle offset are as follows: After the rivet 3D model is aligned with the standard model, for each vertex , respectively obtain their normal vectors (normal of actual rivet and normal of standard rivet). Calculate the angle between the two as the offset angle. The calculation expression is: ,in: is the normal vector of the actual rivet vertex, is the normal vector of the corresponding point of the standard rivet, It is the angle between the normal vector of the actual rivet vertex and the normal vector of the corresponding point of the standard rivet, reflecting the degree of local deformation of the point. Represents the vector dot product (inner product), also known as the scalar product; Subsequently, in order to make the deformation of different areas of the actual rivet contribute differently to the overall deformation, weighted processing is performed based on the surface Gaussian curvature, so that the weight of the area with significant deformation is higher. The weighted expression is: ,in: The actual rivet vertex The Gaussian curvature at , which measures the degree of local curvature, is a regulating factor used to control the influence of curvature on weight; The purpose of this step is to calculate the normal offset at different positions on the rivet surface and use the local curvature as a weighting factor so that the area with prominent deformation has a greater impact on subsequent calculations, thereby avoiding interference from small errors.

[0030] In order to improve the local sensitivity of deformation detection, the rivet surface is divided into k regions, each region Contains multiple vertices , calculate the reference value of the normal angle offset of the rivet surface, the calculation expression is: ,in: Indicates the reference value of the normal angle offset of the rivet surface, For the region The local normal angle deviation metric measures the overall deviation degree of the area. Use sine square instead of direct angle value to make it more sensitive to small angles and enhance the effect of micro-deformation; This step normalizes the local area, making it easier to identify areas with concentrated deformation, without masking local deviations due to the overall mean. Ultimately, the reference value of the normal angle deviation of the rivet surface can be used for further deformation detection.

[0031] From the normal angle offset reference value, it can be seen that the normal angle offset reference value generated by further in-depth analysis of the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model, the larger the performance value is, the larger the offset angle of the rivet surface normal vector is, and this offset occurs in an area with high local curvature (i.e., deformation-sensitive area), indicating that the risk of rivet deformation is greater. A higher normal angle offset reference value usually means that some parts of the rivet have abnormal bending, twisting or local collapse, which may affect the riveting quality and structural strength. On the contrary, if the normal angle offset reference value is small, it means that the overall shape of the rivet is highly consistent with the standard model, the surface curvature changes smoothly, and no obvious structural deformation occurs, so the risk of rivet deformation is low.

[0032] A trained and operational deep learning model means it has completed the entire process, from data collection and preprocessing to model building, training, and validation, and is running stably in a real-world production environment. First, to accurately assess the degree of rivet deformation, researchers or engineering teams collect a large amount of 3D data on rivets in various states (including data on normal rivets and rivets with various deformations), and perform necessary data cleaning, annotation, and feature extraction. After completing this initial data organization, the research team selects an appropriate deep learning architecture, such as a convolutional neural network (CNN), a 3D convolutional network (3D-CNN), or a point cloud-based deep model (PointNet, PointNet++, etc.), and tailors and adjusts the network based on the characteristic dimensions of the rivet data. Next, through feature learning and regression or classification tasks, the model learns and captures the mapping between different deformation features and target labels (such as deformation level and pass / fail). During model training, researchers may employ various augmentation strategies (such as rotating and translating the 3D data, adding noise, etc.) to improve the model's adaptability to various disturbances in real-world production environments. In addition, a validation set or cross-validation strategy will be introduced to evaluate the accuracy and robustness of the model on rivets with different degrees of deformation. After training is completed, the model will only be deployed to the actual production line if it demonstrates sufficiently stable and high-precision prediction capabilities on the test set and passes the safety and consistency assessment. The deployment process usually includes steps to optimize inference speed, such as using TensorRT or other hardware acceleration methods to enable the model to respond quickly in a high-speed production environment. Therefore, "training has been completed and put into use" not only means that the model has completed academic discussions, but also means that the model has withstood the test of the engineering environment and has the high reliability and maintainability to run in real time on the industrial production line.

[0033] In practical applications, such deep learning models are usually integrated into the entire automated riveting production line. Specifically, after the features such as the cross-sectional eccentricity reference value and the normal angle offset reference value of the rivet are calculated, the system will package these features and pass them to the deep learning model that has been put into use. After receiving the input, the model will quantitatively evaluate the degree of deformation of the rivet based on its internally learned weights and structure, and output a deformation coefficient. This deformation coefficient can be understood as a comprehensive measure of the current degree of abnormality of the rivet. The model will comprehensively consider the local geometric mutations on the rivet surface, the overall offset trend, and various interference factors (such as noise data, acquisition angle differences, curvature sensitivity, etc.), and finally give a judgment on the deformation risk. For example, when the deformation coefficient exceeds a certain critical value, the system will initiate a deceleration or elimination strategy; if the deformation coefficient is within an acceptable range, the rivet is allowed to continue to the next process. It's worth noting that this type of model often requires periodic updates and maintenance after it's put into actual use. More measured data accumulated during the production process will be retrained or fine-tuned in the model, gradually improving the accuracy and applicability of the predictions. Once the process or equipment changes, such as replacing a new rivet batch, modifying the feed rate, or upgrading the camera, the model also needs to be retrained or its parameters adjusted accordingly to adapt to the new data distribution and process conditions. Precisely because of this ability to continuously iterate, learn online, and self-optimize, deep learning models are able to ensure high detection accuracy in automated riveting scenarios while flexibly responding to various production changes, achieving truly stable and intelligent deformation monitoring and quality control.

[0034] The machine learning model is not specifically limited here, and can achieve the cross-section eccentricity reference value Reference value of the angle offset from the normal Perform comprehensive analysis to generate deformation coefficients In order to realize the technical solution of the present invention, the present invention provides a specific implementation method; the deformation coefficient The generated expression is: , where 、 are the reference values ​​of cross-section eccentricity Reference value of the angle offset from the normal The preset scaling factor of 、 Both are greater than 0.

[0035] It can be seen from the deformation coefficient that the greater the performance value of the cross-sectional eccentricity reference value generated by further in-depth analysis of the offset of multiple cross-sectional centers of the rivet relative to the cross-sectional center of the standard rivet, the greater the performance value of the normal angle offset reference value generated by further in-depth analysis of the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model. That is, the greater the performance value of the deformation coefficient generated when the rivet deformation risk is evaluated by the deep learning model that has been trained and put into use, the greater the risk of deformation of the rivet, and vice versa.

[0036] The deformation coefficient generated by the trained and operational deep learning model when evaluating rivet deformation risk is compared with the pre-set deformation coefficient reference threshold to assess the rivet deformation risk. The specific steps are as follows: If the deformation coefficient is greater than the deformation coefficient reference threshold, the rivet is judged as a deformation risk rivet, and subsequent operations are performed on the deformation risk rivet; if the deformation coefficient is less than or equal to the deformation coefficient reference threshold, the rivet is judged as a normal rivet, and the normal rivet is transferred to the riveting area for riveting operation.

[0037] When a rivet is judged to be at risk of deformation, the actual feed speed is automatically adjusted according to the degree of deformation. By reducing the feed speed, the deformed rivet is prevented from further impact or extrusion, which further aggravates the deformation. At the same time, the camera and image processing algorithm have more time to acquire and analyze more rivet image data, thereby improving the precision and accuracy of deformation detection. When it is determined that there is a risk of rivet deformation, the actual feeding speed is automatically adjusted according to the degree of rivet deformation. The specific steps are as follows: After the rivet deformation detection is completed, the deformation degree of the rivet is evaluated according to the output deformation coefficient of the deep learning model. The severity of the rivet deformation is judged by the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated by the deformation coefficient and the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated using a nonlinear adjustment function. The calculation expression is: ,in: The deformation coefficient of the rivet indicates the degree of deformation of the rivet. The larger the value, the more serious the deformation. The reference threshold of the deformation coefficient defines the maximum allowable deformation range. Rivets exceeding this threshold need to adjust the feeding speed. It is the deformation sensitivity coefficient, which is used to control the influence of deformation on the adjustment of feeding speed. The larger the value, the faster the feeding speed decreases. is the feeding speed adjustment factor, the value range is , the greater the deformation, the more obvious the drop in feeding speed; The purpose of this step is to gradually reduce the feed rate as the deformation coefficient increases through a nonlinear exponential decay function, rather than simply decreasing it linearly. This can avoid over-adjustment caused by small deformations, while ensuring that the feed rate of severely deformed rivets can be quickly reduced to reduce the possibility of further damage.

[0038] The feed speed adjustment factor is calculated After that, the actual feeding speed is calculated based on the current initial feeding speed (usually the set high-speed feeding value). The calculation expression is: ,in: is the initial feeding speed, which is the default feeding speed under normal circumstances. The minimum feeding speed is the speed at which the rivet is severely deformed. The feeding speed should not approach zero infinitely, but should be limited to a minimum value. , ensure the continuity of the feeding process, is the actual feeding speed, which is the final feeding speed after dynamic adjustment based on the deformation situation; This step ensures that adjusting the feed speed not only reduces the risk of rivet deformation but also optimizes image acquisition time, allowing the camera more time to capture rivet data from more angles, thereby improving deformation detection accuracy. This way, the system achieves an optimal balance between efficient production and precise detection, ensuring rivet quality without affecting the overall production rhythm.

[0039] After the actual feed speed is dynamically reduced, the multi-directional camera and processing unit have more processing time to carefully inspect rivets, especially for slight or localized deformations. Higher image resolution and more adequate analysis time significantly improve recognition accuracy. In this way, the system improves riveting efficiency while enabling early removal or correction of abnormal rivets, avoiding the risk of batch failures in subsequent processes. This step deeply integrates real-time detection and adaptive control, balancing feed efficiency and product quality, and fundamentally enhancing the reliability and flexibility of the automated production line.

[0040] The SPR online pre-riveting detection and control method can accurately identify tiny deformations of rivets during high-speed automatic feeding, and effectively reduce the impact of deformed rivets on riveting quality through intelligent control strategies. The core advantage of this solution lies in the combination of multi-angle visual inspection, deep learning deformation analysis, and dynamic feeding adjustment, which enables the system to complete the three-dimensional reconstruction of rivets and compare them with the standard model in a short period of time, accurately quantifying the degree of deformation. Compared with the traditional fixed high-speed feeding method, this method can achieve real-time intervention on abnormal rivets, reduce the secondary impact on deformed rivets by reducing the feeding speed, and at the same time increase the detection time and improve the image acquisition accuracy, thereby improving the accuracy and stability of deformation detection. This not only reduces the risk of "stuck nails" and the production of batch defective products, but also effectively prevents structural safety hazards caused by insufficient riveting strength, ultimately improving the reliability of the production line, reducing rework costs, and ensuring the quality stability of riveted products.

[0041] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0042] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0043] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. A SPR online pre-riveting detection control method, characterized in that: The following steps are involved: The rivets are transported from the hopper or automatic feeding mechanism to the riveting area according to the pre-set initial feeding speed; When the rivet enters the feeding channel and reaches the visual range of the camera, the cameras installed at different positions and angles synchronously collect the rivet's shape information to generate all-round, multi-angle image data; After completing multi-angle image acquisition, the acquired images are fused to obtain a preliminary 3D model of the rivet. Subsequently, the preliminary 3D model of the rivet is aligned with the pre-stored standard 3D rivet model to place the actual rivet shape and the standard theoretical shape in the same reference frame. After successful registration, the difference features between the preliminary 3D model of the rivet and the standard 3D model are extracted and analyzed. The analyzed features are input into a trained and operational deep learning model, which then outputs an assessment result on the rivet deformation risk and quantifies the rivet deformation. When it is determined that there is a risk of rivet deformation, the actual feeding speed is automatically adjusted according to the degree of deformation of the rivet.

2. The SPR online pre-riveting detection control method according to claim 1, characterized in that: Data fusion of the collected multi-angle images includes the following steps: Preprocess images from different cameras and angles, including denoising, illumination normalization, and distortion correction, to ensure that all images are fused under the same visual conditions; Use feature point matching to identify and associate key feature points from different perspectives to ensure that multiple images can be correctly aligned; Using multi-view stereo reconstruction technology, images from different perspectives are converted into 3D point clouds or depth maps, and the camera projection matrix is ​​adjusted through iterative optimization; Use the ICP algorithm to fuse multi-angle point cloud data into a unified 3D model, eliminating redundancy and error points; Surface reconstruction and optimization are performed, and smoothing is performed to obtain a three-dimensional model of the rivet.

3. The SPR online pre-riveting detection control method according to claim 1, characterized in that: The preliminary 3D model of the rivet is registered with the pre-stored standard 3D model of the rivet, including the following steps: The two are preliminarily aligned based on the coarse registration algorithm of feature points; A rigid registration algorithm is used to globally align the key geometric features of the rivets; The ICP algorithm is used to gradually optimize the matching of point clouds or meshes so that the preliminary 3D model of the rivet and the standard 3D rivet model are aligned with the minimum error in 3D space. Local adjustments are then made to further optimize the alignment results based on fine-grained features.

4. The SPR online pre-riveting detection control method according to claim 1, characterized in that: The difference features between the preliminary three-dimensional model of the rivet and the standard three-dimensional model are extracted, where the extracted features include the offset of multiple cross-section centers of the rivet relative to the cross-section center of the standard rivet and the angular offset between the normal vector of the rivet vertex and the normal vector of the corresponding point of the standard model. The extracted features are analyzed to generate cross-section eccentricity reference values ​​and normal angle offset reference values, respectively. The analyzed cross-section eccentricity reference values ​​and normal angle offset reference values ​​are input into a deep learning model that has been trained and put into use to output the deformation coefficient. The deformation coefficient is used to evaluate the risk of rivet deformation and quantify the deformation of the rivet.

5. The SPR online pre-riveting detection control method according to claim 4, characterized in that: The deformation coefficient generated by the trained and operational deep learning model when evaluating rivet deformation risk is compared with the pre-set deformation coefficient reference threshold to assess the rivet deformation risk. The specific steps are as follows: If the deformation coefficient is greater than the deformation coefficient reference threshold, the rivet is judged as a deformation risk rivet, and subsequent operations are performed on the deformation risk rivet; if the deformation coefficient is less than or equal to the deformation coefficient reference threshold, the rivet is judged as a normal rivet, and the normal rivet is transferred to the riveting area for riveting operation.

6. The SPR online pre-riveting detection control method according to claim 5, characterized in that: When it is determined that there is a risk of rivet deformation, the actual feeding speed is automatically adjusted according to the degree of rivet deformation. The specific steps are as follows: After the rivet deformation detection is completed, the deformation degree of the rivet is evaluated according to the output deformation coefficient of the deep learning model. The severity of the rivet deformation is judged by the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated by the deformation coefficient and the deformation coefficient reference threshold. The feeding speed adjustment factor is calculated using a nonlinear adjustment function. The calculation expression is: ,in: is the deformation coefficient of the rivet, which indicates the degree of deformation of the rivet. The reference threshold value of the deformation coefficient defines the maximum allowable deformation range. Rivets exceeding the reference threshold value of the deformation coefficient need to adjust the feeding speed. is the deformation sensitivity coefficient, which is used to control the influence of deformation on the adjustment of feeding speed. is the feeding speed adjustment factor, the value range is .

7. The SPR online pre-riveting detection control method according to claim 6, characterized in that: The feed speed adjustment factor is calculated Afterwards, the actual feeding speed is calculated based on the current initial feeding speed. The calculation expression is: ,in: is the initial feeding speed, which is the default feeding speed under normal circumstances. is the minimum feeding speed, is the actual feeding speed, which is the final feeding speed after dynamic adjustment based on the deformation situation.

8. The SPR online pre-riveting detection control method according to claim 4, characterized in that: The specific steps for further in-depth analysis of the offset of multiple cross-section centers of rivets relative to the standard rivet cross-section center to generate cross-section eccentricity reference values ​​are as follows: For multiple key sections of the rivet, obtain their center coordinates and calculate the offset of multiple key sections relative to the center of the corresponding section of the standard rivet. Suppose the center point of the rivet section is , the corresponding cross-section center of the standard rivet is , then the calculation expression of the center offset of each section is: ,in: For the i The center offset of each section, For the actual rivet i The coordinates of the center point of the cross section, Standard rivet i The coordinates of the center point of each cross section; The offsets of all sections are normalized to eliminate the influence of rivets of different sizes. The normalized offset is defined as the expression for normalization: ,in: is the normalized offset, L is the total length of the rivet; After calculating the normalized offset Then, the importance weight of the cross section in the overall structure of the rivet is further combined to generate the cross section eccentricity reference value. The generated expression is: ,in: It is the reference value of cross-section eccentricity, which is an indicator to comprehensively measure the degree of cross-section eccentricity of rivets. is the cross-sectional weight coefficient, For the i The axial distance from the cross section to the bottom of the rivet, is the nonlinear attenuation factor.

9. The SPR online pre-riveting detection control method according to claim 4, characterized in that: The specific steps for further in-depth analysis of the angle offset between the normal vector of the rivet vertex and the normal vector of the corresponding point on the standard model to generate a reference value for the normal angle offset are as follows: After the rivet 3D model is aligned with the standard model, for each vertex , respectively obtain the normal vector of the actual rivet vertex and the normal vector of the corresponding point of the standard rivet, and calculate the angle between the two as the offset angle. The calculation expression is: ,in: is the normal vector of the actual rivet vertex, is the normal vector of the corresponding point of the standard rivet, It is the angle between the normal vector of the actual rivet vertex and the normal vector of the corresponding point of the standard rivet, reflecting the degree of local deformation of the point. Represents the vector dot product, also known as the scalar product; Based on the Gaussian curvature of the surface, weighting is performed to make the weight of the deformation-significant area higher. The weighted expression is: ,in: The actual rivet vertex The Gaussian curvature at , which measures the degree of local curvature, is a regulating factor used to control the influence of curvature on weight; Split the rivet surface into k regions, each region Contains multiple vertices , calculate the reference value of the normal angle offset of the rivet surface, the calculation expression is: ,in: Indicates the reference value of the normal angle offset of the rivet surface, For the region The local normal angle deviation metric is used to measure the overall deviation degree of the area.

Images