Full-automatic optical detection and sorting method and system for elastic sheet connectors

Abstract

The invention relates to a full-automatic optical detection and sorting method and system for elastic sheet connectors in the production field of elastic sheet connectors, and the method comprises the steps: carrying out the weighted integration of deformation contour parameters and temperature difference abnormal intensity in an initial feature set through a data fusion processing technology, compressing redundant information through a principal component analysis method, and obtaining a fusion feature vector; aiming at the fusion feature vector, adopting a support vector machine algorithm to carry out defect classification training, and determining a classification boundary through a pre-established defect sample library to obtain a preliminary judgment result of a defect type; for a sorted finished product and defective product set, a statistical analysis method is adopted to calculate the detection efficiency and the sorting accuracy rate, multi-modal imaging parameters are adjusted through real-time feedback, and an optimized system operation state is obtained; and according to the optimized system operation state, high-throughput data output after the detection efficiency is improved is obtained, and continuously-improved detection sorting performance is obtained by updating the feature extraction and classification model through loop iteration.

Description

Technical Field

[0001] The present invention relates to the field of information technology, specifically to the field of production of spring connectors, and in particular to a method and system for fully automatic optical detection and sorting of spring connectors. Background Art

[0002] As a core component in the field of electronic manufacturing, the quality of shrapnel connectors is directly related to the reliability and service life of the products. They play an indispensable role in high-speed communications, automotive electronics and smart devices. With the growing demand for high precision and miniaturization in the industry, ensuring the defect detection and sorting efficiency of shrapnel connectors has become a key link in improving production quality and reducing costs. However, traditional manual visual inspection methods have exposed significant shortcomings in the face of this demand. Manual inspection can only process 20 products per minute, with a defect miss rate of more than 15%, and is almost powerless against micron-level deformation defects. This inefficient and high-error status quo can no longer meet the requirements of modern manufacturing for high throughput and high precision.

[0003] The limitations of existing detection methods are mainly reflected in low efficiency and insufficient recognition capabilities. Manual detection is limited by the resolution and subjective judgment of the human eye and cannot meet the needs of identifying small deformations or internal defects. Even if some automated equipment is introduced, single-mode optical detection is difficult to take into account the comprehensive analysis of surface deformation and internal thermal anomalies at the same time, resulting in incomplete defect classification and difficulty in breaking through the bottleneck of sorting efficiency. These problems are particularly prominent in the production of high-precision spring connectors, becoming a technical bottleneck restricting the development of the industry.

[0004] In this field, the core challenge is how to achieve efficient detection and accurate sorting of multi-dimensional defects. The detection system needs to capture surface deformation at micron-level resolution and sense internal temperature anomalies, which requires the synchronization of imaging technology in spatial and temporal dimensions. In addition, how to effectively fuse multi-source data and extract related features to improve the accuracy of defect classification is another key technical problem. If these challenges are not resolved, it will directly lead to low detection efficiency and insufficient interception of defective products, which will in turn affect the stability of product quality.

[0005] Therefore, how to achieve comprehensive detection of surface and internal defects through the collaborative work of multimodal imaging technology, and combine efficient classification algorithms with fast sorting mechanisms, has become a key issue in improving the performance of the fully automatic optical inspection and sorting system for spring connectors. Solving this problem will provide the manufacturing industry with a high-precision and high-efficiency inspection method, and promote the industry to move towards intelligence and automation. Summary of the invention

[0006] The present invention provides a fully automatic optical detection and sorting method for spring-type connectors, comprising the following steps:

[0007] Step S101: Obtain the high-resolution image of the surface of the shrapnel connector and the internal thermal imaging data. Capture the surface deformation characteristics at the micron-level resolution through a multi-modal imaging device, and simultaneously record the abnormal distribution of internal temperature differences to obtain a multi-source original data set.

[0008] Step S102: For the multi-source original data set, use the spatial synchronization calibration method to perform pixel-level alignment on the surface image and the thermal imaging data, and ensure the consistency of the acquisition time through time synchronization constraints to obtain the aligned multi-modal data combination.

[0009] Step S103: Extract the spatial features of the surface deformation from the aligned multi-modal data combination, calculate the deformation contour parameters through the edge detection algorithm, and simultaneously use the thermal imaging analysis method to quantify the intensity distribution of the abnormal internal temperature difference to obtain the initial feature set.

[0010] Step S104: Through the data fusion processing technology, perform weighted integration on the deformation contour parameters and the abnormal temperature difference intensity in the initial feature set, and use the principal component analysis method to compress redundant information to obtain the fusion feature vector.

[0011] Step S105: For the fusion feature vector, use the support vector machine algorithm for defect classification training, and determine the classification boundary through the pre-established defect sample library to obtain the preliminary judgment result of the defect type.

[0012] Step S106: According to the preliminary judgment result, if the classification confidence is lower than the preset threshold, then perform secondary analysis on the fusion feature vector through the convolutional neural network, and combine multi-layer convolution to extract deep associated features to obtain the optimized defect classification label.

[0013] Step S107: Obtain the defect category and location information of each shrapnel connector from the optimized defect classification label, and drive the robotic arm to perform the sorting operation according to the classification label through the fast sorting mechanism to obtain the sorted finished products and defective products set.

[0014] Step S108: For the sorted finished products and defective products set, use the statistical analysis method to calculate the detection efficiency and sorting accuracy rate, and adjust the multi-modal imaging parameters through real-time feedback to obtain the optimized system operation state.

[0015] Step S109: According to the optimized system operation state, obtain the high-throughput data output with improved detection efficiency, and update the feature extraction and classification model through cyclic iteration to obtain the continuously improved detection and sorting performance.

[0016] The present invention provides a fully automatic optical detection and sorting system for shrapnel connectors, mainly including:

[0017] A multimodal imaging module, which is used to acquire high-resolution images of the surface of the shrapnel connector and internal thermal imaging data, capture surface deformation features at the micron-level resolution through a multimodal imaging device, and record the abnormal distribution of internal temperature differences simultaneously to obtain a multi-source original dataset;

[0018] A data alignment module, which is used for the multi-source original dataset, and adopts a spatial synchronization calibration method to perform pixel-level alignment on the surface image and thermal imaging data, and ensures the consistency of the acquisition time through time synchronization constraints to obtain an aligned multimodal data combination;

[0019] A feature extraction module, which is used to extract the spatial features of surface deformation from the aligned multimodal data combination, calculate the deformation contour parameters through an edge detection algorithm, and quantify the intensity distribution of internal temperature differences abnormally by using a thermal imaging analysis method to obtain an initial feature set;

[0020] A data fusion module, which is used to weight and integrate the deformation contour parameters and the intensity of abnormal temperature differences in the initial feature set through data fusion processing technology, and adopts a principal component analysis method to compress redundant information to obtain a fused feature vector;

[0021] A defect classification module, which is used for the fused feature vector, and adopts a support vector machine algorithm to perform defect classification training, and determines the classification boundary through a pre-established defect sample library to obtain a preliminary judgment result of the defect type;

[0022] An optimized classification module, which is used according to the preliminary judgment result. If the classification confidence is lower than a preset threshold, the fused feature vector is analyzed twice through a convolutional neural network, and deep correlation features are extracted by combining multi-layer convolution to obtain an optimized defect classification label;

[0023] A sorting execution module, which is used to obtain the defect category and location information of each shrapnel connector from the optimized defect classification label, and drive the robotic arm to perform sorting operations according to the classification label through a fast sorting mechanism to obtain a set of sorted finished products and defective products;

[0024] A statistical analysis module, which is used for the set of sorted finished products and defective products, and adopts a statistical analysis method to calculate the detection efficiency and sorting accuracy rate, and adjusts the multimodal imaging parameters in real time through feedback to obtain an optimized system operation state;

[0025] A performance optimization module, which is used according to the optimized system operation state, obtains high-throughput data output after the detection efficiency is improved, and updates the feature extraction and classification models through iterative cycles to obtain continuously improved detection and sorting performance.

[0026] The technical solution provided by the embodiment of the present invention may include the following beneficial effects:

[0027] The present invention discloses a fully automatic optical detection and sorting method for a shrapnel connector. This method acquires high-resolution images of the connector surface and internal thermal imaging data through a multimodal imaging device, extracts surface deformation and internal temperature difference features after spatially and temporally synchronizing and aligning the data. It analyzes and classifies the features using data fusion and machine learning algorithms to determine the defect type and location. According to the classification results, it drives the robotic arm to perform automatic sorting and continuously optimizes the system performance through statistical analysis. The present invention realizes the efficient and automatic defect detection and sorting of the shrapnel connector, improving production efficiency and product quality. This method integrates technologies such as multimodal imaging, data analysis, and machine learning, and has the advantages of high detection accuracy, high sorting efficiency, and sustainable optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of the fully automatic optical detection and sorting method for the shrapnel connector of the present invention.

[0029] Figure 2 It is a schematic diagram of the fully automatic optical detection and sorting method and system for the shrapnel connector of the present invention.

[0030] Figure 3 It is another schematic diagram of the fully automatic optical detection and sorting method and system for the shrapnel connector of the present invention.

[0031] Figure 4 It is a schematic framework diagram of the fully automatic optical detection and sorting system for the shrapnel connector of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] Such as Figures 1-4 , the fully automatic optical detection and sorting method for the shrapnel connector in this embodiment may specifically include:

[0034] Step S101, acquire high-resolution images of the shrapnel connector surface and internal thermal imaging data, capture surface deformation features at the micron-level resolution through a multimodal imaging device, and record the abnormal distribution of internal temperature differences simultaneously to obtain a multi-source original data set.

[0035] Obtain a surface image and thermal imaging data, where the surface image and thermal imaging data are multi-modal data of the same target object; perform high-pass filtering on the surface image, extract the deformation features in the surface image, and obtain a deformation distribution map; according to a preset threshold, determine whether there is an abnormal area in the temperature difference distribution in the thermal imaging data. If there is an abnormal area, mark the abnormal area to obtain an abnormal distribution map; fuse the deformation distribution map and the abnormal distribution map, and use the principal component analysis algorithm to determine the significant features in the deformation distribution map and the abnormal distribution map to obtain a feature set; according to the feature set, calculate the correlation coefficient between the deformation features and the temperature difference distribution to obtain a correlation coefficient matrix; input the correlation coefficient matrix into a pre-trained support vector machine classification model to determine the coupling mode between the deformation features and the temperature difference distribution to obtain a classification result; according to the classification result, generate a multi-modal coupling map of the deformation features and the temperature difference distribution.

[0036] Exemplarily, a surface image and thermal imaging data are obtained by a multi-modal imaging device, which involves simultaneously capturing the deformation and temperature information of a target object using an optical camera and an infrared thermal imager.

[0037] For example, when detecting an industrial pipeline, the optical camera captures the minute deformations on the pipeline surface, and the thermal imager records the temperature difference distribution. The original data set may contain pixel points with a surface image resolution of 1920×1080 and corresponding thermal imaging temperature values ranging from 20°C to 80°C. These data lay the foundation for subsequent analysis and ensure the integrity of multi-source information. The high-pass filtering algorithm is used to process the surface image, aiming to highlight the high-frequency components of the deformation features and filter out low-frequency noise.

[0038] Exemplarily, in pipeline detection, high-pass filtering can extract the surface elevation or depression features caused by pressure and generate a deformation distribution map. The deformation amplitude may be manifested as a displacement of 0.5 mm to 2 mm. This method effectively enhances the visualization of detailed features and provides a clear deformation basis for subsequent analysis. When extracting the temperature difference distribution from the thermal imaging data, if the preset threshold is 10°C, the areas where the temperature difference exceeds this value are marked as abnormal.

[0039] Specifically, in the high-temperature area of the pipeline, the local temperature may suddenly rise from 30°C to 45°C, indicating possible leakage or overheating, and thus an abnormal distribution map is generated. This marking method intuitively reflects the potential problem areas and helps to quickly locate the abnormalities.

[0040] In a possible implementation, when fusing the deformation distribution map and the anomaly distribution map, the principal component analysis algorithm is used to extract significant features. The deformation and temperature difference data are converted into feature vectors, and the feature set may include the deformation peak value and the temperature difference gradient. This dimensionality reduction processing reduces data redundancy, highlights key variables, and improves the analysis efficiency. The correlation is calculated based on the feature set, and the correlation coefficient matrix reflects the coupling degree between deformation and temperature difference.

[0041] For example, when the area with a large deformation amplitude exactly corresponds to the area with a significant temperature difference, the correlation coefficient may be close to 0.8, indicating a high degree of correlation between the two. This quantitative analysis reveals the internal relationship between physical phenomena and provides data support for subsequent classification.

[0042] Preferably, the support vector machine algorithm is used to classify the correlation coefficient matrix to determine the coupling mode.

[0043] In one embodiment, pipeline detection may be divided into three modes: "deformation-dominated", "temperature-difference-dominated", and "strong coupling". If both deformation and temperature difference are significant, it is classified as "strong coupling", and the result accuracy can reach 90%. This classification method effectively identifies the characteristic relationships in different states. A multi-modal coupling map is generated through the classification results, and the final multi-source analysis data is presented in a visual form.

[0044] For example, the pipeline surface may show the overlap of red abnormal areas and deformation concentration areas, intuitively displaying the problem distribution. Such a chart is not only easy to understand but also can guide maintenance decisions, enhancing the practical value of detection.

[0045] It should be noted that the advantage of multi-modal analysis lies in integrating the complementarity of the two types of data. Deformation reflects structural changes, and temperature difference reveals thermal anomalies. The combination of the two improves the detection sensitivity.

[0046] For example, in the case where a single thermal imaging may ignore minor deformations, the fusion analysis can more comprehensively evaluate the pipeline state and reduce the risk of missed detection.

[0047] It can be understood that this method has strong scalability. In another embodiment, if a time dimension is added to analyze the change trends of deformation and temperature difference over time, the development process of faults can be predicted. This dynamic analysis further enhances the prediction ability of the technology and provides support for preventive maintenance.

[0048] Step S102, for the multi-source original data set, the spatial synchronization calibration method is used to perform pixel-level alignment on the surface image and the thermal imaging data, and the acquisition time is ensured to be consistent through time synchronization constraints to obtain the aligned multi-modal data combination.

[0049] Obtain multi-source data to get the original dataset, where the original dataset includes surface images and thermal imaging data; for the surface images and thermal imaging data, use the synchronization calibration method for processing to obtain the pixel-level alignment result; obtain the acquisition time information corresponding to the pixel-level alignment result, and adjust the pixel-level alignment result through time synchronization constraints to determine the time-consistent data after alignment; use the spatial synchronization method to process the time-consistent data, judge the spatial correspondence between the surface image and the thermal imaging data to obtain the preliminary multi-modal data; use the data combination technology to fuse the preliminary multi-modal data, determine the integrity of the multi-modal data to obtain the fused multi-modal dataset; if there are deviations in the fused multi-modal dataset, use the alignment method to calibrate the spatial synchronization and time synchronization to obtain the calibrated multi-modal dataset; according to the calibrated multi-modal dataset, obtain the feature distribution of the multi-source data to get the final multi-modal data combination; for the final multi-modal data combination, use the support vector machine algorithm to extract key features, determine the classification basis of the multi-modal data to obtain the feature classification result.

[0050] Exemplarily, when obtaining the original dataset through multi-source data.

[0051] It can be understood that the acquisition of surface images and thermal imaging data is often affected by equipment and environment.

[0052] Exemplarily.

[0053] In a possible implementation, assume that a high-resolution camera and an infrared thermal imager are used to synchronously collect the surface and internal data of a shrapnel connector. The surface image resolution is 2000×1500 pixels, and the thermal imaging data resolution is 640×480 pixels. Due to the resolution difference, the synchronization calibration method needs to be used for processing.

[0054] In one embodiment, the thermal imaging data can be upsampled to the same resolution as the surface image through an interpolation algorithm, and then the two sets of data can be aligned using the feature point matching technique to obtain the pixel-level alignment result. This method ensures the accuracy of subsequent analysis. When obtaining the acquisition time information for the pixel-level alignment result.

[0055] It should be noted that time synchronization is the key to multi-modal data processing.

[0056] For example, during the acquisition process, the surface image may record 10 frames per second, while the thermal imaging data is 5 frames.

[0057] Preferably, each frame of data can be marked with a timestamp, and time synchronization constraints can be set, such as adjusting the acquisition frequency in milliseconds, and finally forming time-consistent data. This adjustment ensures the unity of the data on the time axis. When using the spatial synchronization method to process the time-consistent data.

[0058] Specifically, the correspondence between the surface image and the thermal imaging data can be determined through spatial coordinate mapping.

[0059] In one embodiment, assume that the coordinates of a certain point on the surface of the shrapnel connector are (x1, y1), which are mapped to (x2, y2) in the thermal imaging data through the device calibration parameters, with the error controlled within 5 micrometers, to obtain preliminary multimodal data. The establishment of this spatial correspondence provides a basis for subsequent fusion. When fusing the preliminary multimodal data through data combination techniques.

[0060] It can be understood that integrity is an important indicator for measuring the fusion effect.

[0061] For example, the gray value of the surface image and the temperature value of the thermal imaging data can be combined according to certain weights to generate a fused multimodal data set. If a deviation is found after fusion, such as the temperature distribution not matching the deformation area, alignment methods are used for calibration. In one possible implementation, the spatial and temporal parameters can be adjusted through an iterative optimization algorithm to reduce the deviation to less than 2%, obtaining a calibrated multimodal data set. This calibration improves the reliability of the data. When obtaining the feature distribution based on the calibrated multimodal data set.

[0062] Exemplarily, the effectiveness of data combination can be judged through statistical analysis.

[0063] In one embodiment, the maximum value of the surface deformation height difference, such as 10 micrometers, and the temperature difference in the temperature anomaly area, such as 5 degrees Celsius, are extracted, and combined with the distribution laws of both to generate a final multimodal data combination. This combination method facilitates subsequent feature extraction. When using the support vector machine algorithm to extract key features.

[0064] Specifically, the deformation height and temperature difference values can be used as input vectors to train a classification model.

[0065] For example, it is set that the deformation greater than 8 micrometers and the temperature difference exceeding 4 degrees Celsius are abnormal classes, and below this threshold are normal classes, to obtain the feature classification result. This classification basis helps to quickly identify potential problem areas of the shrapnel connector and improve the analysis efficiency.

[0066] Step S103, extract the spatial features of the surface deformation from the aligned multimodal data combination, calculate the deformation contour parameters through an edge detection algorithm, and at the same time use the thermal imaging analysis method to quantify the intensity distribution of the internal temperature anomaly, to obtain an initial feature set.

[0067] Obtain the result after alignment processing from multimodal data. For the result after alignment processing, use an edge detection algorithm to calculate the contour parameters of surface deformation to obtain deformation contour data; for the deformation contour data, use a spatial feature extraction method to determine the spatial distribution characteristics of surface deformation; analyze the multimodal data by thermal imaging method to obtain the intensity distribution data of abnormal internal temperature difference; if the intensity distribution data exceeds a preset threshold, combine the deformation contour data to judge the boundary range of the abnormal area; according to the spatial distribution characteristics and the intensity distribution data, fuse the feature extraction results to obtain a comprehensive feature set; use a support vector machine algorithm to classify the comprehensive feature set to judge the correlation between surface deformation and temperature difference abnormality; update the initial feature set according to the classification result to obtain the feature description of deformation and temperature difference abnormality.

[0068] Exemplarily, for the result after alignment processing in multimodal data, an edge detection algorithm can be used to extract the contour parameters of surface deformation. For example.

[0069] In a possible implementation manner, through the classical Canny edge detection method, first smooth the surface image to reduce noise interference, and then calculate the gradient intensity to identify the edge lines of the deformation area.

[0070] Exemplarily, assume that the detected contour shows that the length of a certain area on the surface changes by 2 mm and the width changes by 1.5 mm. These parameters constitute the deformation contour data. The advantage of this method is that it can clearly outline the specific boundary of the deformation, which is convenient for subsequent analysis.

[0071] In one embodiment, a spatial feature extraction method is used for the deformation contour data to determine the spatial distribution characteristics of surface deformation.

[0072] Specifically, a spatial analysis method based on grid division can be used to divide the surface into multiple small areas of 10 mm × 10 mm, and count the degree of deformation in each area.

[0073] Preferably, if the deformation of a certain area is concentrated on the edge and deviates from the central value by 30%, it can be judged that this area has significant spatial distribution characteristics. This way helps to reveal the non-uniformity of deformation in space. For the temperature information in multimodal data analyzed by thermal imaging method.

[0074] It can be understood that the intensity distribution data of abnormal internal temperature difference is the key.

[0075] For example, for the data collected by a thermal imager, assume that the temperature of a certain area is 5 degrees Celsius higher than the surrounding area, and the distribution range is a circular area with a diameter of 20 mm. This forms the intensity distribution data.

[0076] It should be noted that this kind of analysis can quickly locate potential abnormal points and provide a basis for subsequent judgment. If the intensity distribution data exceeds the preset threshold, for example, the temperature difference exceeds 4 degrees Celsius, then the boundary range of the abnormal area is judged in combination with the deformation contour data.

[0077] In a possible implementation manner, if the deformation contour shows that the edge of this area bulges by 1 mm, in combination with the temperature difference data, it can be speculated that there may be internal stress concentration here. The advantage of this joint judgment is that it can confirm the scope and nature of the abnormality from multiple dimensions.

[0078] Specifically, when integrating the spatial distribution characteristics and the intensity distribution data to form a comprehensive feature set, the deformation height, the range area, and the temperature difference intensity can be used as input features.

[0079] For example, the deformation height of a certain area is 1.2 mm, the area is 50 square millimeters, and the temperature difference is 5 degrees Celsius. After these data are integrated, a multi-dimensional feature vector is formed. This kind of integration can more comprehensively describe the relationship between deformation and temperature difference. When using the support vector machine algorithm to classify the comprehensive feature set.

[0080] Exemplarily, the data can be divided into two categories: "high correlation" and "low correlation".

[0081] In one embodiment, if the deformation and temperature difference feature vectors of a certain area fall within the classification boundary and the classification result shows a high correlation between the two, it indicates that the deformation may be induced by temperature difference abnormality. The advantage of this kind of classification is that it can quantify the causal relationship between the two. When updating the initial feature set through the classification result to determine the final feature description.

[0082] Preferably, irrelevant features in the classification can be removed. For example, if the temperature difference in a certain area is only 1 degree Celsius and the deformation is not significant, it will be removed from the feature set.

[0083] For example, the final feature description may focus on the area where the deformation height is greater than 1 mm and the temperature difference exceeds 4 degrees Celsius. This kind of update can improve the accuracy of the feature set and provide more reliable data support for subsequent analysis.

[0084] Step S104, through the data fusion processing technology, the deformation contour parameters in the initial feature set and the temperature difference abnormality intensity are weighted and integrated, and the principal component analysis method is used to compress redundant information to obtain a fusion feature vector.

[0085] Obtain the deformed contour and temperature difference anomaly data, and use a fusion technique to perform weighted processing on the deformed contour and temperature difference anomaly to obtain a preliminary integrated feature; for the preliminary integrated feature, use principal component analysis for redundant information compression to obtain a dimensionality-reduced feature set; extract the anomaly intensity data from the dimensionality-reduced feature set to determine the anomaly distribution range; if the anomaly distribution range exceeds a preset threshold, perform secondary weighted processing on the dimensionality-reduced feature set to obtain an adjusted feature set; perform fusion and comparative analysis on the adjusted feature set and the initial feature to obtain a difference vector; calculate the main components of the fusion vector based on the difference vector to obtain the final feature representation; use a clustering algorithm to divide the feature categories for the final feature representation and judge the association strength between the categories.

[0086] Exemplarily, when performing weighted processing on the deformed contour and temperature difference anomaly through a fusion technique, it can be understood as integrating the two types of data in a certain proportion. For example.

[0087] In a possible implementation, the deformed contour data may be given a weight of 0.6, while the temperature difference anomaly data is given a weight of 0.4. This weighting method can highlight the main contribution of the deformation while retaining the auxiliary information of the temperature difference.

[0088] Exemplarily, assume that the width change of the deformed contour on the surface of a certain component is 5 mm, and the intensity of the temperature difference anomaly region is 3 °C. The preliminary integrated feature value obtained through weighted calculation may be 4.2. The advantage of this method is that it can balance the characteristics of the two types of data and provide a more comprehensive basis for subsequent analysis. When using principal component analysis to compress the preliminary integrated feature, the purpose is to reduce redundant information.

[0089] In an embodiment, if the preliminary integrated feature includes three dimensions: deformation width, deformation depth, and temperature difference intensity, principal component analysis may find that the deformation width and depth are highly correlated, thus compressing the three-dimensional data into a two-dimensional dimensionality-reduced feature set.

[0090] For example, the original data has a deformation width of 5 mm, a depth of 2 mm, and a temperature difference of 3 °C. After analysis, the comprehensive deformation value 4 and the temperature difference 3 may be retained as new features. This dimensionality reduction can effectively reduce the data complexity while retaining the main information. When extracting the anomaly intensity data from the dimensionality-reduced feature set.

[0091] Specifically, the anomalies can be screened by setting a threshold.

[0092] For example, assume that the normal temperature difference range is 1 - 2 degrees Celsius and the comprehensive deformation value range is 2 - 3. If the current data shows a temperature difference of 3 degrees Celsius and a comprehensive deformation value of 4, both exceed the threshold, and the abnormal distribution range may be defined to cover the entire detection area. This screening method helps to quickly locate the problem area. If the abnormal distribution range exceeds the preset threshold, a secondary weighted processing is performed on the dimensionality-reduced feature set.

[0093] Preferably, the weights can be adjusted according to the severity of the abnormality. For example, the temperature difference weight is increased to 0.6 and the deformation weight is decreased to 0.4, and the adjusted feature set is recalculated.

[0094] For example, the original eigenvalue of the temperature difference is 3 and the deformation is 4. After secondary weighting, a new eigenvalue of 3.6 may be obtained. This adjustment can more sensitively reflect the impact of the abnormality. When comparing and analyzing the adjusted feature set with the initial features through the fusion technology, a difference vector can be generated.

[0095] In a possible implementation, the initial eigenvalue is 4.2 and the adjusted eigenvalue is 3.6, and the difference vector is -0.6. This difference reflects the change trend in data processing and helps to discover potential problems. When calculating the main components of the fusion vector based on the difference vector.

[0096] For example, the difference vector -0.6 can be combined with other dimensional data to obtain the final feature representation through vector decomposition.

[0097] It should be noted that this representation focuses more on the core characteristics of the abnormality and can provide a clearer basis for subsequent classification. When using the clustering algorithm to divide the feature categories for the final feature representation, it can be understood as grouping the data to analyze the association strength.

[0098] For example, assume that the final feature representation contains multiple groups of data. After clustering, it is divided into a normal group and an abnormal group. In the abnormal group, both the temperature difference and the deformation value are relatively high, and the association strength may be manifested as the synchronous change of the two. This classification method helps to reveal the internal relationship between the deformation and the temperature difference and provides more reliable support for abnormal detection.

[0099] Step S105, for the fusion feature vector, use the support vector machine algorithm for defect classification training, determine the classification boundary through the pre-established defect sample library, and obtain the preliminary judgment result of the defect type.

[0100] Obtain a pre-established defect sample library, which includes multiple defect samples; perform feature fusion on the defect samples to obtain fused features; use the support vector machine algorithm to process the fused features to obtain feature vectors; perform classification training based on the feature vectors to obtain an initial classification model, and determine the classification boundary during the classification training process; if the accuracy of the classification boundary is lower than a preset threshold, optimize the classification training process by adjusting the dimension of the feature vector to obtain an updated classification model; according to the updated classification model, judge the defect type to which the defect sample belongs to obtain a preliminary classification result; use a statistical tool to verify the distribution of the preliminary classification result to determine the stability of the preliminary classification result to obtain verified classification data; compare the verified classification data with the defect sample library to obtain the final judgment result of the defect type; for the final judgment result, use a clustering algorithm to perform grouping processing to obtain defect distribution characteristics.

[0101] Exemplarily, extracting feature vectors through fused features is to integrate feature data from multiple sources to form a unified vector representation.

[0102] Exemplarily, in the field of defect detection, surface deformation parameters and temperature distribution data can be fused to obtain a multi-dimensional feature vector, and the dimension may be 10 or 15, specifically depending on the complexity of the input data. The core of this method lies in retaining key information while reducing redundancy.

[0103] In a possible implementation, different features can be assigned importance through a weighting method. For example, the deformation parameter accounts for 60% and the temperature data accounts for 40% to generate an initial vector. Using the support vector machine algorithm to process the feature vector aims to construct a classification model.

[0104] It should be noted that the support vector machine distinguishes different categories by finding the optimal hyperplane.

[0105] For example, when detecting defects in metal parts, the feature vector may include crack length and temperature difference value, and the support vector machine can classify this data into two categories: "defective" and "non-defective".

[0106] Specifically, during training, sample data such as 100 positive samples and 100 negative samples will be used to calculate the classification boundary. If the accuracy of the initial boundary does not meet the standard, for example, it only reaches 85% while the preset threshold is 90%, then optimization is required. Adjusting the dimension of the feature vector to optimize the training process is a key step in improving the model performance.

[0107] In one embodiment, through feature selection technology, dimensions with low correlation can be removed, such as reducing from 15 dimensions to 8 dimensions, while retaining the main information. This adjustment can make the classification boundary clearer.

[0108] Preferably, the accuracy of the optimized model on the test set may be increased to 92%, making it applicable to actual detection. Analyzing the fused features according to the updated classification model to determine the defect type is the core in the following steps.

[0109] For example, the analysis result may show that the crack length of a certain sample is 5 mm and the temperature difference is 3 °C, and the model determines it as "surface crack".

[0110] It can be understood that such preliminary classification results need to be further verified to ensure reliability. Using statistical tools to verify the distribution of the results, we can draw a distribution diagram and observe the proportion of various defects. For example, surface cracks account for 70% and internal cavities account for 20%, to judge whether the results are stable. Comparing the verified classification data with the sample database is to improve the accuracy of judgment.

[0111] In one embodiment, the sample database may contain 500 labeled defect samples, and the matching degree between the verification data and the data in the database reaches 95% to finally confirm the defect type. This comparison can reduce the losses caused by misjudgment. For the final judgment result, using the clustering algorithm for grouping can reveal the defect distribution characteristics.

[0112] For example, after clustering, it is found that most surface cracks are concentrated on the edge of the part, which provides a basis for subsequent process improvement.

[0113] Specifically, clustering algorithms such as K-means can divide the defects into 3 categories, and the mean values of the feature vectors of each category are different. For example, the central vector of one category shows that the average crack length is 4 mm and the temperature difference is 2 °C.

[0114] Exemplarily, this grouping can not only clearly display the defect distribution, but also help optimize the detection strategy, such as increasing the detection frequency for edge defects. This method gradually improves the accuracy and practicality of defect recognition through multi-level analysis.

[0115] Step S106, according to the preliminary judgment result, if the classification confidence is lower than the preset threshold, then perform a secondary analysis on the fused feature vector through a convolutional neural network, extract deep associated features by combining multi-layer convolutions, and obtain an optimized defect classification label.

[0116] Obtain the feature data of the object to be classified, perform preliminary classification through a preset classification model to obtain preliminary classification labels and confidence values; determine whether the confidence value is lower than a preset threshold, if so, extract fusion features from the feature data; use a convolutional neural network to process the fusion features and obtain deep features through multiple layers of convolution; analyze associated features based on the deep features to determine defect labels; optimize the preliminary classification labels according to the defect labels to obtain optimized classification labels; determine whether the optimized classification labels are consistent with the preliminary classification labels, if they are consistent, output the preliminary classification labels as the final classification labels, otherwise, output the optimized classification labels as the final classification labels.

[0117] Exemplarily, obtaining the confidence value through preliminary classification is a key link in defect classification.

[0118] Exemplarily, when detecting metal surface defects, the preliminary classification may be based on the fusion of color and texture features to obtain the confidence value of each sample, such as 0.85 or 0.6. Judging whether the confidence is lower than a preset threshold (such as 0.8) can be understood as a screening mechanism, and samples with low confidence need further analysis.

[0119] For example, a sample with a confidence of 0.6 indicates that the classifier is not very sure about its attribution, which may be due to insufficient feature extraction or noise interference. If the confidence is lower than the threshold, data is extracted from the fusion features and secondary analysis is performed through a convolutional neural network.

[0120] In a possible implementation, the fusion features may be a combination of image pixels and edge information. Before inputting into the convolutional neural network, it can be normalized to 64x64 pixels. The multiple-layer convolution operation of the convolutional neural network can gradually extract more abstract features.

[0121] Specifically, the first layer of convolution may focus on local texture, and the second layer focuses on shape contours, finally forming deep features. Compared with the original fusion features, these deep features can better reflect the internal patterns of defects, which helps to improve the classification accuracy. The process of analyzing associated features based on the deep features to determine defect labels can be regarded as a mapping from the feature space to the category space.

[0122] For example, the deep features may show that a certain sample has a continuous linear pattern and local highlighted areas. After comparing with the sample library, it is inferred that it is a "scratch" defect.

[0123] It should be noted that the determination of defect labels depends on the labeled data during network training, and the reliability of the labels directly affects the results.

[0124] Preferably, a small amount of manual verification can be introduced to ensure the accuracy of the label. After the optimization results are adjusted according to the defect labels and the preliminary classification labels are obtained, the consistency is compared with the initial classification.

[0125] In one embodiment, if the initial classification label is "crack" and the secondary analysis label is "scratch", the consistency judgment will be marked as "inconsistent" to trigger further verification. This comparison can effectively reduce the misjudgment rate.

[0126] For example, a crack may be misjudged as a scratch due to feature confusion, and such errors can be corrected through deep analysis of the convolutional network. After obtaining the consistency judgment results, the final classification label is output.

[0127] Specifically, samples with high consistency directly output labels, such as "scratch"; inconsistent samples can be re-evaluated in combination with confidence and finally determined.

[0128] For example, when the confidence level is increased from 0.6 to 0.9, the classification label is more reliable. The advantage of this method is that it improves the robustness of classification through multi-level verification, which is especially suitable for complex defect scenarios.

[0129] In one embodiment, for tiny defects, secondary analysis can also uncover detail features that were overlooked by the initial classification, significantly improving detection capabilities.

[0130] For example, in metal sheet inspection, the initial classification may have low confidence due to uneven lighting, and the secondary analysis of the convolutional neural network can identify hidden microcracks through multi-layer feature extraction. This multi-step collaborative approach not only ensures the comprehensiveness of the classification, but also enhances the interpretability of the results through deep features.

[0131] It can be understood that the output of the final classification label provides a reliable basis for subsequent defect processing and effectively supports the quality control process.

[0132] Step S107, obtaining the defect category and location information of each spring connector from the optimized defect classification label, and driving the robotic arm to perform sorting operations according to the classification label through a fast sorting mechanism to obtain a collection of sorted finished products and defective products.

[0133] Obtain the image information of the shrapnel connector, extract the defect classification and location information from the image information using a preset defect detection model, and obtain the labeled shrapnel connector dataset; use a clustering algorithm to group the labeled shrapnel connector dataset to obtain multiple clustering centers representing defect classifications; determine classification labels corresponding one-to-one to the defect classifications according to the multiple clustering centers; generate a driving instruction for the robotic arm according to the classification labels, and obtain the sorting path corresponding to the robotic arm; use a path planning algorithm to optimize the sorting path to obtain an adjusted robotic arm execution sequence; if the adjusted robotic arm execution sequence meets a preset efficiency threshold, drive the robotic arm to perform the sorting operation according to the adjusted robotic arm execution sequence to obtain a preliminary sorting result; obtain the classification verification information of the preliminary sorting result, and determine whether the classification verification information is consistent with the classification label; if the classification verification information is consistent with the classification label, separate the shrapnel connectors in the preliminary sorting result into a finished product set and a defective product set; according to the statistical information of the finished product set and the defective product set, determine whether the accuracy of the sorting completion meets a preset accuracy threshold.

[0134] Exemplarily, extract the defect classification and location information from the shrapnel connector through a preset defect detection model.

[0135] It can be understood that this method relies on the accurate recognition of the surface features of the shrapnel connector by the model. For example.

[0136] In a possible implementation manner, the detection model may be based on image processing technology. First, scan the features such as the edges and textures of the shrapnel connector to generate a labeled dataset containing defect types such as "crack" and "deformation" and specific coordinates such as "X: 12.5, Y: 8.3". This labeling not only facilitates subsequent processing but also provides clear guidance for mechanical operations. Use a clustering algorithm to group the labeled dataset to determine the classification labels corresponding to the defect classifications.

[0137] Specifically, the K-means clustering method can be used to group the crack-type defects into one group and the deformation-type defects into another group.

[0138] Exemplarily, if the dataset contains 100 defect points, 3 main classification labels may be obtained through clustering: crack, deformation, and foreign object, and the number of defect points in each group is 40, 35, and 25 respectively. This grouping method helps to quickly lock the defect patterns and improve the classification efficiency. Generate a driving instruction according to the classification label and obtain the sorting path of the robotic arm.

[0139] In one embodiment, assuming that the crack label corresponds to "Path A" and the deformation label corresponds to "Path B", the system generates instructions based on the labels, such as "Move along Path A to pick up the item at X: 12.5, Y: 8.3". This instruction design intuitively reflects the correspondence between the defect location and the operation requirements, providing a clear operation basis for the robotic arm. Optimize the sorting path of the robotic arm through a path planning algorithm to obtain an adjusted execution sequence.

[0140] Preferably, the algorithm may be based on the shortest path principle, adjusting the originally scattered picking order into a continuous curve trajectory.

[0141] For example, the original path may require the robotic arm to move repeatedly between the coordinates "X: 12.5, Y: 8.3" and "X: 15.0, Y: 9.0". After optimization, it is adjusted to a clockwise circular path, reducing the moving distance by about 20%. This not only improves the sorting speed but also reduces energy consumption. If the adjusted execution sequence meets the preset threshold, drive the robotic arm to perform the sorting operation to obtain a preliminary sorting result.

[0142] It should be noted that the threshold may be a time limit such as "less than 5 seconds for a single operation" or a precision requirement such as "positioning deviation less than 0.1 mm".

[0143] For example, in a certain operation, the robotic arm grabs 10 defective parts along the optimized path, with a total time consumption of 45 seconds, meeting the threshold requirements. The result shows that 8 cracked parts and 2 deformed parts are correctly sorted. Verify the preliminary sorting result. If the verification result is consistent with the classification label, separate the finished product set and the defective product set.

[0144] For example, by rechecking the sorting result through an optical sensor, it is found that all 8 cracked parts are correctly placed in the defective product set, which is consistent with the label. This verification ensures the reliability of the sorting and avoids missed inspections or misjudgments. Judge the accuracy of the sorting completion by statistically analyzing the finished product set and the defective product set.

[0145] In a possible implementation, statistics show that the defective product set contains 40 cracked parts and 35 deformed parts, and the finished product set is 25 defect-free parts, with a total accuracy rate of 98%.

[0146] Specifically, if it is found that 2 deformed parts are misjudged as finished products in a certain sorting, it can be further optimized by adjusting the clustering parameters or the path planning logic. This analysis provides data support for subsequent improvements and ensures the efficiency and stability of the sorting process.

[0147] Step S108, for the finished product and defective product sets after sorting, use statistical analysis methods to calculate the detection efficiency and sorting accuracy rate, and adjust the multi-modal imaging parameters through real-time feedback to obtain an optimized system operating state.

[0148] Calculate the detection efficiency and sorting accuracy from the completed finished product set and defective product set through statistical analysis to obtain preliminary quantification results. Obtain real-time feedback data based on the preliminary quantification results to determine the change trends of the detection efficiency and sorting accuracy. Judge whether there are deviations in the imaging parameters of multi-modal imaging through the change trends to obtain deviation evaluation data. Adjust the imaging parameters using the deviation evaluation data to obtain the adjusted parameter configuration. Update the operating state of multi-modal imaging through the adjusted parameter configuration to obtain optimized imaging output. Recalculate the detection efficiency and sorting accuracy for the optimized imaging output to obtain updated quantification results. If the updated quantification results do not reach the preset threshold, repeat the above adjustment process to obtain the final system operating state.

[0149] Exemplarily, calculating the detection efficiency and sorting accuracy from the completed finished product set and defective product set through statistical analysis to obtain preliminary quantification results is the key first step.

[0150] For example, in the scene of sorting shrapnel connectors, assume that 1000 products are processed in a day, there are 950 products in the finished product set, and 50 products in the defective product set. Through statistics, it is found that the detection efficiency is 20 pieces per minute, and the sorting accuracy is 98%. Such quantification results provide a data basis for subsequent optimization.

[0151] It can be understood that the detection efficiency reflects the system processing speed, while the sorting accuracy measures the reliability of classification. When obtaining real-time feedback data based on the preliminary quantification results to determine the change trends, the dynamics of the data need to be concerned.

[0152] Exemplarily, if the detection efficiency drops from 20 pieces / minute to 18 pieces / minute continuously for three days, and the sorting accuracy slightly drops from 98% to 97%, it indicates that there may be potential problems in the system.

[0153] Specifically, the feedback data can be collected in real time through sensors, such as the number of image frames output by the imaging device per second. This trend analysis helps to detect the signs of anomalies. Judging whether there are deviations in the multi-modal imaging parameters through the change trends is a possible implementation method.

[0154] Preferably, if the detection efficiency drops, it may be related to the exposure time or resolution setting in the imaging parameters.

[0155] For example, after the exposure time is adjusted from 0.01 second to 0.02 second, the image is blurred, resulting in a decrease in the defect recognition rate. The deviation evaluation data can be obtained by comparing the standard parameters with the current parameters. For example, the resolution deviates from the standard value by 5%, thus providing a basis for adjustment. When adjusting the imaging parameters using the deviation evaluation data, it is necessary to ensure the pertinence of the adjustment.

[0156] In one embodiment, if the resolution deviation is 5%, it can be increased from 800x600 to 840x630 while keeping the frame rate stable at 30 frames per second. After the adjusted parameter configuration is updated, the multi-modal imaging device can capture the details of the shrapnel connector more clearly. This adjustment process reflects the practicality of parameter optimization. After updating the operating state with the adjusted parameter configuration and obtaining the optimized imaging output, the effect needs to be re-verified.

[0157] For example, the adjusted imaging output shows that the defect edges are sharper, the detection efficiency rises back to 21 pieces per minute, and the sorting accuracy increases to 99%. The acquisition of this optimized output significantly improves the stability of the system.

[0158] It should be noted that the improvement of the imaging output directly affects the accuracy of subsequent sorting. Recalculating the quantization result for the optimized imaging output is the key to verifying the effectiveness of the adjustment.

[0159] In one embodiment, if the updated detection efficiency is 21 pieces per minute and the accuracy is 99%, but the preset thresholds are 22 pieces per minute and 99.5%, then further adjustment is required.

[0160] It can be understood that this iterative process ensures that the system gradually approaches the target performance. If the updated quantization result does not reach the threshold, the adjustment process is repeated to obtain the final operating state.

[0161] For example, the light source intensity can be further fine-tuned from 500 lumens to 550 lumens to enhance the imaging contrast, and finally the detection efficiency reaches 22 pieces per minute and the accuracy stabilizes at 99.5%.

[0162] In one possible implementation, this repeated adjustment can also reduce the misjudgment rate and improve the quality of the finished product set.

[0163] Preferably, a stable operating state can reduce manual intervention and extend the service life of the device.

[0164] Step S109, according to the optimized system operating state, obtain the high-throughput data output with improved detection efficiency, and update the feature extraction and classification model through cyclic iteration to obtain continuously improved detection and sorting performance.

[0165] Obtain the original data stream collected by the sensor, and use preprocessing techniques to perform noise removal processing on the original data stream to obtain a stable data sequence; extract time window features from the stable data sequence, and use a pre-constructed convolutional neural network to process the time window features to generate an initial feature vector; perform clustering analysis on the initial feature vector to obtain a classification label set; obtain a preset expected classification result, and determine whether the difference between the classification label set and the expected classification result is less than a first preset threshold. If so, use the classification label set to update the network parameters of the convolutional neural network to obtain an optimized feature vector; if the sorting accuracy of the optimized feature vector is lower than a second preset threshold, adjust the length of the time window, obtain new time window features, and re-perform the convolutional neural network processing until the sorting accuracy is higher than the second preset threshold; perform detection and sorting operations according to the optimized feature vector to obtain a sorting result set; obtain the pre-stored historical sorting data, compare the sorting result set with the historical sorting data, judge the performance improvement range, and output the final result of detection and sorting.

[0166] Exemplarily, collecting the original data stream through the sensor is the basis of the entire process, and usually relies on high-precision sensor devices to capture real-time signals.

[0167] For example, in an industrial sorting scenario, the sensor can be an optical sensor or a vibration sensor, which is used to record the physical characteristics of finished products and defective products. The original data stream often contains a lot of noise, such as fluctuations caused by ambient light interference or mechanical vibration. When using preprocessing techniques to remove noise, it can be understood as being achieved through filtering methods.

[0168] Exemplarily, a low-pass filter can effectively remove high-frequency noise and retain key signals. Assuming the original data sampling rate is 1000Hz, a smooth stable data sequence can be obtained after filtering, which improves the reliability of subsequent analysis. Extracting time window features from the stable data sequence is a key step, and the length of the time window directly affects the quality of the features.

[0169] In a possible implementation, set the window length to 0.5 seconds, and collect features such as peaks, means, and variances within each window segment.

[0170] For example, for optical sensor data, the change rate of the reflected light intensity may be extracted within the window, generating data points containing 10 dimensions. Then, use a convolutional neural network to generate an initial feature vector.

[0171] Preferably, the network structure may include multiple convolutional layers and pooling layers to gradually refine the spatial correlation in the data. For example, after inputting 10-dimensional features, the network outputs an initial feature vector of 64 dimensions for subsequent classification. When performing clustering analysis on the initial feature vector, common methods such as K-means clustering can be used.

[0172] Specifically, assume that the sorting task needs to distinguish between two categories of qualified and defective products. After clustering, a classification label set is generated, and the labels may be "0" and "1". By iteratively comparing the classification label set with the expected results, the network parameters are updated.

[0173] For example, the expected results are provided by manual annotation. If it is found that the classification error rate reaches 20%, the weights of the convolutional kernels are adjusted through backpropagation to obtain an optimized feature vector. This iterative method can gradually approach a more accurate classification boundary. If the sorting accuracy of the optimized feature vector is lower than a preset threshold, such as 90%, the time window length is adjusted.

[0174] It should be noted that a too short window may lose global information, while a too long window may introduce redundancy.

[0175] For example, when the window is adjusted from 0.5 seconds to 1 second and the features are re-extracted, the accuracy may be increased to 92%. After performing the detection and sorting operation based on the optimized feature vector, a sorting result set is obtained.

[0176] In one embodiment, the result set shows that 995 out of 1000 products are correctly classified. By comparing the sorting result set with historical data, the degree of performance improvement is judged.

[0177] It can be understood that the historical data may show an accuracy of 85%, while the new method has increased to 92%, indicating a significant performance gain. The output final result not only reflects the sorting efficiency but also provides a basis for subsequent parameter adjustment.

[0178] For example, this method can adapt to product specification changes faster, reduce manual intervention, and optimize the stability of the overall process.

[0179] The present invention provides a fully automatic optical detection and sorting system for elastic sheet connectors, mainly including:

[0180] A multi-modal imaging module for obtaining high-resolution images of the surface of the elastic sheet connector and internal thermal imaging data, capturing surface deformation characteristics at the micron-level resolution through a multi-modal imaging device, and simultaneously recording abnormal internal temperature difference distributions to obtain a multi-source original data set;

[0181] A data alignment module for pixel-level alignment of the surface image and thermal imaging data for the multi-source original data set using a spatial synchronization calibration method, and ensuring the same acquisition time through time synchronization constraints to obtain an aligned multi-modal data combination;

[0182] A feature extraction module, which is used to extract the spatial features of surface deformation from the aligned multi-modal data combination, calculate the deformation contour parameters through an edge detection algorithm, and at the same time use a thermal imaging analysis method to quantify the intensity distribution of abnormal internal temperature differences, so as to obtain an initial feature set;

[0183] A data fusion module, which is used to weight and integrate the deformation contour parameters and the temperature difference abnormal intensity in the initial feature set through data fusion processing technology, and use the principal component analysis method to compress redundant information to obtain a fused feature vector;

[0184] A defect classification module, which is used to perform defect classification training on the fused feature vector by using a support vector machine algorithm, determine the classification boundary through a pre-established defect sample library, and obtain a preliminary judgment result of the defect type;

[0185] An optimized classification module, which is used to, according to the preliminary judgment result, if the classification confidence is lower than a preset threshold, perform a secondary analysis on the fused feature vector through a convolutional neural network, and combine multi-layer convolution to extract deep associated features to obtain an optimized defect classification label;

[0186] A sorting execution module, which is used to obtain the defect category and position information of each chip connector from the optimized defect classification label, and drive the robotic arm to perform sorting operations according to the classification label through a fast sorting mechanism, so as to obtain a set of sorted finished products and defective products;

[0187] A statistical analysis module, which is used to calculate the detection efficiency and sorting accuracy rate for the set of sorted finished products and defective products by using statistical analysis methods, and adjust the multi-modal imaging parameters through real-time feedback to obtain an optimized system operation state;

[0188] A performance optimization module, which is used to obtain high-throughput data output after the detection efficiency is improved according to the optimized system operation state, and update the feature extraction and classification models through iterative loops to obtain continuously improved detection and sorting performance.

[0189] In addition, it should be noted that, for the various specific technical features described in the above specific embodiments, they can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention does not separately describe various possible combination methods. In addition, any combination can be made between various different embodiments of the present invention, as long as it does not violate the idea of the present invention, it should also be regarded as the content disclosed by the present invention.

Claims

1. A fully automatic optical inspection and sorting method for a shrapnel connector, characterized in that, The method includes the following steps: Step S101: Obtain the high-resolution image of the surface of the shrapnel connector and the internal thermal imaging data. Capture the surface deformation characteristics at the micron-level resolution through a multi-modal imaging device, and simultaneously record the abnormal distribution of the internal temperature difference to obtain a multi-source original data set. Step S102: For the multi-source original data set, use the spatial synchronization calibration method to perform pixel-level alignment on the surface image and the thermal imaging data, and ensure the consistency of the acquisition time through time synchronization constraints to obtain an aligned multi-modal data combination. Step S103: Extract the spatial features of the surface deformation from the aligned multi-modal data combination, calculate the deformation contour parameters through an edge detection algorithm, and simultaneously use the thermal imaging analysis method to quantify the intensity distribution of the abnormal internal temperature difference to obtain an initial feature set. Step S104: Use the data fusion processing technology to perform weighted integration of the deformation contour parameters and the abnormal temperature difference intensity in the initial feature set, and use the principal component analysis method to compress redundant information to obtain a fusion feature vector. Step S105: For the fusion feature vector, use the support vector machine algorithm to perform defect classification training, and determine the classification boundary through a pre-established defect sample library to obtain a preliminary judgment result of the defect type. Step S106: According to the preliminary judgment result, if the classification confidence level is lower than the preset threshold, then perform secondary analysis on the fusion feature vector through a convolutional neural network, and combine multi-layer convolution to extract deep associated features to obtain an optimized defect classification label. Step S107: Obtain the defect category and location information of each shrapnel connector from the optimized defect classification label, and drive the robotic arm to perform sorting operations according to the classification label through a fast sorting mechanism to obtain a set of sorted finished products and defective products. Step S108: For the set of sorted finished products and defective products, use the statistical analysis method to calculate the detection efficiency and sorting accuracy, and adjust the multi-modal imaging parameters through real-time feedback to obtain an optimized system operating state. Step S109: According to the optimized system operating state, obtain the high-throughput data output after the detection efficiency is improved, and update the feature extraction and classification model through cyclic iteration to obtain continuously improved detection and sorting performance.

2. The fully automatic optical inspection and sorting method for the elastic sheet connector according to claim 1, wherein The said step S101 includes: Obtain the surface image and the thermal imaging data, and the surface image and the thermal imaging data are multi-modal data of the same target object. Perform high-pass filtering processing on the surface image, extract the deformation characteristics in the surface image to obtain a deformation distribution map. According to a preset threshold, judge whether there is an abnormal area in the temperature difference distribution of the thermal imaging data. If there is an abnormal area, mark the abnormal area to obtain an abnormal distribution map. Fuse the deformation distribution map and the abnormal distribution map, and use the principal component analysis algorithm to determine the significant features in the deformation distribution map and the abnormal distribution map to obtain a feature set. According to the feature set, calculate the correlation coefficient between the deformation characteristics and the temperature difference distribution to obtain a correlation coefficient matrix. Input the correlation coefficient matrix into a pre-trained support vector machine classification model to judge the coupling mode between the deformation characteristics and the temperature difference distribution to obtain a classification result. Generate a multimodal coupling graph of the deformation characteristics and the temperature difference distribution according to the classification result.

3. The full-automatic optical inspection and sorting method for the elastic sheet connector according to claim 1, wherein, The step S102 includes: Obtain multi-source data to get an original data set, where the original data set includes surface images and thermal imaging data; Process the surface images and thermal imaging data using a synchronization calibration method to obtain a pixel-level alignment result; Obtain the acquisition time information corresponding to the pixel-level alignment result, and adjust the pixel-level alignment result through time synchronization constraints to determine the time-consistent data after alignment; Process the time-consistent data using a spatial synchronization method, judge the spatial correspondence between the surface image and the thermal imaging data, and obtain preliminary multimodal data; Fuse the preliminary multimodal data using a data combination technique, determine the integrity of the multimodal data, and obtain a fused multimodal data set; If there are deviations in the fused multimodal data set, use an alignment method to calibrate the spatial synchronization and time synchronization to obtain a calibrated multimodal data set; According to the calibrated multimodal data set, obtain the feature distribution of the multi-source data to get a final multimodal data combination; For the final multimodal data combination, use a support vector machine algorithm to extract key features, determine the classification basis of the multimodal data, and obtain a feature classification result.

4. The full-automatic optical inspection and sorting method for the shrapnel connector according to claim 1, characterized in that, The step S104 includes: Obtain deformation contours and temperature difference anomaly data, and use a fusion technique to perform weighted processing on the deformation contours and temperature difference anomalies to obtain preliminary integrated features; Perform redundant information compression on the preliminary integrated features using principal component analysis to obtain a dimensionality-reduced feature set; Extract anomaly intensity data from the dimensionality-reduced feature set and determine the anomaly distribution range; If the anomaly distribution range exceeds a preset threshold, perform secondary weighted processing on the dimensionality-reduced feature set to obtain an adjusted feature set; Fuse and compare the adjusted feature set with the initial features to obtain a difference vector; Calculate the main components of the fusion vector according to the difference vector to obtain a final feature representation; Use a clustering algorithm to divide the feature categories for the final feature representation and judge the association strength between the categories.

5. The full-automatic optical inspection and sorting method for the elastic sheet connector according to any one of claims 1-4, characterized in that The step S105 includes: Obtain a pre-established defect sample library, where the defect sample library includes multiple defect samples; Perform feature fusion on the defect samples to obtain fused features; Process the fused features using a support vector machine algorithm to obtain feature vectors; Perform classification training according to the feature vectors to obtain an initial classification model, and determine the classification boundary during the classification training process; If the accuracy of the classification boundary is lower than a preset threshold, optimize the classification training process by adjusting the dimension of the feature vectors to obtain an updated classification model; According to the updated classification model, judge the defect type to which the defect sample belongs to obtain a preliminary classification result; Use a statistical tool to perform distribution verification on the preliminary classification result to determine the stability of the preliminary classification result and obtain verified classification data; Compare the verified classification data with the defect sample library to obtain a final judgment result of the defect type. For the final judgment result, a clustering algorithm is used for grouping processing to obtain defect distribution characteristics.

6. The full-automatic optical inspection and sorting method for the shrapnel connector according to any one of claims 1-4, characterized in that, The step S106 includes: Obtain the characteristic data of the object to be classified, and perform preliminary classification through a preset classification model to obtain a preliminary classification label and a confidence value; Judge whether the confidence value is lower than a preset threshold. If so, extract the fusion features from the characteristic data; Process the fusion features by using a convolutional neural network to obtain deep features through multiple layers of convolution; Analyze the associated features according to the deep features to determine the defect label; Optimize the preliminary classification label according to the defect label to obtain an optimized classification label; Judge whether the optimized classification label is consistent with the preliminary classification label. If they are consistent, output the preliminary classification label as the final classification label; otherwise, output the optimized classification label as the final classification label.

7. The fully automatic optical inspection and sorting method for the shrapnel connector according to any one of claims 1-4, characterized in that The step S107 includes: Obtain the image information of the shrapnel connector, and use a preset defect detection model to extract defect classification and position information from the image information to obtain a labeled shrapnel connector data set; Use a clustering algorithm to group the labeled shrapnel connector data set to obtain multiple clustering centers representing defect classification; Determine the classification labels corresponding one-to-one to the defect classification according to the multiple clustering centers; Generate a drive instruction for the robotic arm according to the classification label, and obtain the sorting path corresponding to the robotic arm; Use a path planning algorithm to optimize the sorting path to obtain an adjusted robotic arm execution sequence; If the adjusted robotic arm execution sequence meets the preset efficiency threshold, drive the robotic arm to perform the sorting operation according to the adjusted robotic arm execution sequence to obtain a preliminary sorting result; Obtain the classification verification information of the preliminary sorting result, and judge whether the classification verification information is consistent with the classification label; If the classification verification information is consistent with the classification label, separate the shrapnel connectors in the preliminary sorting result into a finished product set and a defective product set; Judge whether the accuracy of the sorting completion meets the preset accuracy threshold according to the statistical information of the finished product set and the defective product set.

8. The full-automatic optical inspection and sorting method for the shrapnel connector according to any one of claims 1-4, characterized in that, The step S108 includes: Calculate the detection efficiency and sorting accuracy from the finished product set and the defective product set after the sorting is completed through statistical analysis to obtain a preliminary quantification result; Obtain real-time feedback data according to the preliminary quantification result, and determine the change trend of the detection efficiency and sorting accuracy; Judge whether there is a deviation in the imaging parameters of the multimodal imaging through the change trend to obtain deviation evaluation data; Adjust the imaging parameters by using the deviation evaluation data to obtain an adjusted parameter configuration; Update the operating state of the multimodal imaging through the adjusted parameter configuration to obtain an optimized imaging output; Recalculate the detection efficiency and sorting accuracy for the optimized imaging output to obtain an updated quantification result; If the updated quantification result does not reach the preset threshold, repeat the above adjustment process to obtain the final system operating state.

9. The full-automatic optical detection and sorting system for the shrapnel connector is characterized in that, This system is used to implement the full-automatic optical detection and sorting method for the shrapnel connector according to any one of claims 1-8. The system includes: A multimodal imaging module, which is used to acquire high-resolution images of the surface of the shrapnel connector and internal thermal imaging data, capture surface deformation features at the micron-level resolution through a multimodal imaging device, and record the abnormal distribution of internal temperature differences simultaneously to obtain a multi-source original dataset; A data alignment module, which is used to perform pixel-level alignment on the surface image and thermal imaging data for the multi-source original dataset by using a spatial synchronization calibration method, and ensure the consistency of the acquisition time through time synchronization constraints to obtain an aligned multimodal data combination; A feature extraction module, which is used to extract the spatial features of surface deformation from the aligned multimodal data combination, calculate the deformation contour parameters through an edge detection algorithm, and quantify the intensity distribution of abnormal internal temperature differences by using a thermal imaging analysis method to obtain an initial feature set; A data fusion module, which is used to perform weighted integration of the deformation contour parameters and abnormal temperature difference intensities in the initial feature set through data fusion processing technology, and compress redundant information by using the principal component analysis method to obtain a fused feature vector; A defect classification module, which is used to perform defect classification training on the fused feature vector by using a support vector machine algorithm, and determine the classification boundary through a pre-established defect sample library to obtain a preliminary judgment result of the defect type; An optimized classification module, which is used to, according to the preliminary judgment result, if the classification confidence is lower than a preset threshold, perform secondary analysis on the fused feature vector through a convolutional neural network, and extract deep correlation features by combining multiple layers of convolution to obtain an optimized defect classification label; A sorting execution module, which is used to obtain the defect category and position information of each shrapnel connector from the optimized defect classification label, and drive the robotic arm to perform sorting operations according to the classification label through a fast sorting mechanism to obtain a set of sorted finished products and defective products; A statistical analysis module, which is used to calculate the detection efficiency and sorting accuracy rate for the set of sorted finished products and defective products by using a statistical analysis method, and adjust the multimodal imaging parameters in real-time feedback to obtain an optimized system operation state; A performance optimization module, which is used to obtain high-throughput data output after the detection efficiency is improved according to the optimized system operation state, and update the feature extraction and classification models through iterative cycles to obtain continuously improved detection and sorting performance.

Images