Machine vision based blemished electrical component sorting system

Abstract

The application discloses a machine vision-based blemish electrical accessory sorting system, and relates to the technical field of sorting, which comprises the following steps: collecting electrical accessory image data sets and transmitting the electrical accessory image data sets to an electrical accessory sorting channel for classification processing to obtain electrical accessory image data; performing blemish identification on the electrical accessory image data to obtain blemish electrical accessory image data; performing blemish classification on the blemish electrical accessory image data to generate blemish category electrical accessory image data; generating sorting instructions according to the blemish category electrical accessory image data to perform simulation sorting and generate simulation sorting results; and performing backtracking optimization on the electrical accessory sorting channel according to the simulation sorting results, and intelligently sorting the electrical accessories through the electrical accessory sorting optimization channel. The application solves the technical problem that the blemish identification in the prior art is not accurate, and the blemish sorting category is not accurate, and achieves the technical effect that the blemish electrical accessories can be sorted flexibly and adaptively.

Description

Technical Field

[0001] This invention relates to the field of sorting technology, and more specifically to a machine vision-based sorting system for defective electrical parts. Background Technology

[0002] With the rapid development of the electrical industry, the detection and sorting of defective electrical components has become a crucial link in ensuring product quality and safety. However, current technologies still face many challenges in defect identification and sorting. Existing technologies are inaccurate in defect identification, leading to unreasonable defect sorting and making it difficult to accurately sort defective electrical components of different types or specifications. Summary of the Invention

[0003] This application provides a machine vision-based defective electrical parts sorting system to address the technical problem of inaccurate defect identification in existing technologies, which leads to inaccurate defect sorting categories.

[0004] In view of the above problems, this application provides a machine vision-based sorting system for defective electrical parts.

[0005] This application provides a machine vision-based defective electrical parts sorting system, the system comprising:

[0006] The system comprises the following modules: a data acquisition module, which continuously acquires images of multiple electrical components on the target production line using an industrial camera, generating an image dataset of electrical components; a defect sorting module, which transmits the image dataset of electrical components to an electrical component sorting channel, wherein the sorting channel includes an electrical component classification branch, a defect identification branch, and a defect classification branch; a classification processing module, which classifies the image dataset of electrical components based on a set of electrical component classes using the electrical component classification branch, obtaining multiple electrical component class image data; and a defect identification module, which classifies the multiple electrical component class image data based on a standard image set of electrical components using the defect identification branch. The system performs defect identification to obtain multiple defective electrical component image data; a defect classification module classifies the multiple defective electrical component image data based on defect feature vector sets through the defect classification branch, generating multiple defective categories of electrical component image data; a simulated sorting module generates sorting instructions based on the multiple defective categories of electrical component image data, performs simulated sorting of the multiple electrical components according to the sorting instructions, and generates simulated sorting results; an intelligent defect sorting module optimizes the electrical component sorting channel based on the simulated sorting results, and performs intelligent defect sorting of the multiple electrical components through the optimized electrical component sorting channel.

[0007] The technical solution provided in this application has at least the following technical effects or advantages:

[0008] This application uses an industrial camera to continuously acquire images of multiple electrical components on a target production line, generating an image dataset of these components. This dataset is then transmitted to an electrical component sorting channel, which includes a classification branch, a defect recognition branch, and a defect classification branch. The classification branch classifies the image dataset based on a set of electrical component classes, obtaining multiple image datasets for each class. The defect recognition branch identifies defects in these image datasets based on a standard image set of electrical components, generating multiple image datasets of defective electrical components. The defect classification branch classifies these defective image datasets based on a set of defect feature vectors, generating image datasets of electrical components with multiple defect categories. Sorting instructions are generated based on these image datasets, and the components are simulated for sorting according to these instructions, generating simulated sorting results. The sorting channel is then optimized based on these simulated sorting results, and an optimized sorting channel is used to perform intelligent defect sorting of the components. This invention solves the technical problem of inaccurate defect identification in existing technologies, which leads to inaccurate defect sorting categories, and achieves the technical effect of flexible and adaptive sorting of defective electrical parts. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic diagram of a machine vision-based defective electrical parts sorting system provided in an embodiment of this application;

[0011] Figure 2 This is a schematic diagram of a machine vision-based method for sorting defective electrical parts, provided in an embodiment of this application.

[0012] Figure labeling: Data acquisition module 11, defect sorting module 12, classification processing module 13, defect identification module 14, defect classification module 15, simulated sorting module 16, intelligent defect sorting module 17. Detailed Implementation

[0013] This application provides a machine vision-based defective electrical parts sorting system to address the technical problem of inaccurate defect identification in existing technologies, which leads to inaccurate defect sorting categories, thereby achieving the technical effect of flexible and adaptive sorting of defective electrical parts.

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0015] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0016] Example

[0017] like Figure 1 As shown, this application provides a machine vision-based defective electrical parts sorting system for performing tasks such as... Figure 2 The machine vision-based method for sorting defective electrical parts shown includes a system comprising:

[0018] The data acquisition module 11 continuously acquires images of multiple electrical components of the target production line using an industrial camera, generating an image dataset of the electrical components.

[0019] In this embodiment, the industrial camera is installed at a key location on the production line to ensure comprehensive and seamless coverage of the electrical components on the production line.

[0020] When the production line starts operating, the data acquisition module uses an industrial camera to capture images of the electrical components on the target production line. The industrial camera takes pictures of the electrical components according to preset parameters, such as exposure, time, and focal length, obtaining an image dataset of the electrical components. This image dataset contains consecutive images of multiple electrical components on the production line.

[0021] The defect sorting module 12 transmits the electrical component image dataset to the electrical component sorting channel, wherein the electrical component sorting channel includes an electrical component classification branch, a defect recognition branch, and a defect classification branch.

[0022] In this embodiment, the defect sorting module transmits the electrical component image dataset to the electrical component sorting channel for defect sorting.

[0023] The electrical parts sorting channel includes three branches: electrical parts classification, defect detection, and defect classification. These branches work together to generate corresponding image data.

[0024] The classification processing module 13 performs classification processing on the electrical component image dataset based on the electrical component class set through the electrical component classification branch to obtain multiple electrical component class image data.

[0025] In this embodiment, the classification processing module receives an image dataset of electrical components and performs classification processing. Before performing classification processing, a set of electrical component classes is predefined, which contains all possible categories of electrical components, and each category has a set of preset features and parameters to describe it.

[0026] For each image in the electrical component image dataset, the classification module identifies key features such as shape, size, color, and texture of the electrical component using image processing techniques. Next, a pre-trained deep learning network, such as a convolutional neural network, is used to analyze the extracted features. Based on the output of the deep learning model, the classification module assigns a category label to each electrical component image and organizes images belonging to the same category into multiple electrical component category image datasets.

[0027] The deep learning network in the classification module learns from a large amount of training data and can identify features that match the categories in the electrical accessories set through training.

[0028] The defect identification module 14 performs defect identification on the multiple electrical component image data based on the electrical component standard image set through the defect identification branch, and obtains multiple defective electrical component image data.

[0029] In this embodiment of the application, the defect identification module pre-stores a standard image set of electrical components, which consists of samples of defect-free electrical components.

[0030] Before defect identification, the defect identification module preprocesses the image data of electrical components, improving image quality and recognition accuracy through operations such as image denoising, enhancement, and normalization. Next, image processing techniques are used to extract key features from the images of the electrical components to be detected, such as shape, texture, and color. Then, a pre-trained defect identification model, such as a CNN-based classifier, is used for defect identification. During training, this model learns how to extract features from electrical component images and distinguish between defective and flawless images. During recognition, the model compares the extracted features with features from a standard image set to determine whether the electrical component has defects. Multiple images of defective electrical components are then obtained through this process.

[0031] The defect classification module 15 classifies the multiple defective electrical component image data based on the defect feature vector set through the defect classification branch, and generates multiple defective categories of electrical component image data.

[0032] In this embodiment, the defect feature vector set contains feature descriptions of various known defect types. These feature descriptions include multi-dimensional information such as shape, texture, color, and location.

[0033] For each defective electrical component image, feature extraction is performed using a deep learning model, such as a convolutional neural network. In this application, the convolutional neural network is trained on a large amount of data to automatically learn key features related to the defect.

[0034] Next, a pre-trained defect classification algorithm is used to classify the extracted features. This algorithm is based on a machine learning or deep learning model, trained on a large dataset of labeled defective electrical component images, learning to map features to different defect categories. Based on the algorithm's output, the defect classification module assigns a defect category label to each defective electrical component image and generates multiple defect category images.

[0035] The simulation sorting module 16 generates sorting instructions based on the image data of the electrical components of the multiple defect categories, performs simulated sorting of the multiple electrical components according to the sorting instructions, and generates simulated sorting results.

[0036] In this embodiment, the simulated sorting module first receives image data of electrical components from multiple defect categories in the defect classification branch. These image data of electrical components, after undergoing the preceding classification and recognition processing, are labeled with the type of electrical component and the specific defect category.

[0037] The simulated sorting module contains predefined sorting rules and strategies. These rules include sorting priorities, target sorting areas, and sorting paths corresponding to different defect categories. Strategies include how to select appropriate sorting equipment or methods based on information such as the size and shape of electrical components.

[0038] Based on the parsed electrical component defect categories and other relevant information, as well as predefined sorting rules and strategies, the simulation sorting module generates specific sorting instructions. These instructions include the electrical component's unique identifier, defect category, sorting priority, target sorting area, sorting path, and sorting equipment or method.

[0039] Next, simulation software, such as FlexSim or Arena, is used to simulate the entire sorting process based on the generated sorting instructions. Following the paths and equipment specified in the instructions, electrical components are simulatedly transferred from the input to the designated sorting area. After the simulated sorting is complete, simulated sorting results are generated. These results include the sorting status of all electrical components, including both correct and incorrect sorting instances.

[0040] The intelligent defect sorting module 17 performs backtracking optimization of the electrical component sorting channel based on the simulated sorting results, and performs intelligent defect sorting of the multiple electrical components through the optimized electrical component sorting channel.

[0041] In this embodiment, the intelligent defect sorting module analyzes the simulated sorting results, identifies problems in the sorting process, and adjusts and optimizes the sorting process based on the analysis results. After backtracking optimization, an optimized sorting channel for electrical components is generated. The optimized sorting channel for electrical components includes reconfiguration of sorting equipment and replanning of sorting paths. Finally, multiple electrical components are intelligently sorted for defects using the optimized sorting channel for electrical components.

[0042] Furthermore, the data acquisition module 11 in the system provided in the application embodiment is also used for:

[0043] The movement trajectories of the multiple electrical components within the target production line are obtained, and the deployment location information of the industrial camera is determined based on the movement trajectories.

[0044] Extract the appearance features of the multiple electrical components, set the light source parameters, and determine the imaging lighting conditions of the industrial camera;

[0045] Based on the deployment location information, the imaging lighting conditions, and by adjusting the deployment angle of the industrial camera, the camera deployment information is determined;

[0046] Based on the camera deployment information, the multiple electrical components of the target production line are continuously acquired according to preset camera parameters to generate an image dataset of the electrical components.

[0047] In this embodiment, the movement trajectories of multiple electrical components are first extracted from the design documents, CAD models, or previous actual operation data of the target production line. Next, based on the movement trajectories of the electrical components, key positions of the components on the production line are determined, such as entering the inspection area or passing through inspection points. The required field of view is determined according to the size, shape, and movement trajectory of the electrical components. Industrial cameras are then installed at suitable locations on the production line to ensure that the electrical components remain within the camera's field of view throughout their movement. The deployment location information of the industrial cameras is determined through the above process.

[0048] By analyzing samples or using historical data, the appearance characteristics of electrical components, such as material, color, and texture, are determined. Based on these characteristics, a suitable light source type is selected. For components requiring highlighted details or textures, high-brightness LED light sources are chosen; for components requiring uniform illumination, ring light sources are selected, etc. The color temperature of the light source is adjusted according to the color and material of the electrical components. Testing is then conducted in a real-world environment, and the light source parameters are fine-tuned based on the test results until satisfactory image quality is achieved. This process determines the imaging lighting conditions for the industrial camera.

[0049] When adjusting the placement angle of industrial cameras, ensure the camera lens is perpendicular to the surface of electrical components by adjusting the camera's pitch angle. Adjust the camera's yaw angle to align the camera directly with the production line. Adjust the camera's roll angle to correct any horizontal tilt.

[0050] The camera deployment information is determined by integrating the deployment location information, imaging lighting conditions, and adjusting the deployment angle of the industrial camera.

[0051] Based on the determined camera deployment information, continuous image acquisition is performed on multiple electrical components on the target production line according to preset camera parameters. These preset camera parameters, including camera resolution, frame rate, and exposure time, are determined based on the characteristics of the electrical components and the inspection requirements. Through continuous acquisition, an image dataset of the electrical components is obtained.

[0052] Furthermore, the classification processing module 13 is also used for:

[0053] Based on industrial big data retrieval, the electrical component categories are determined to define the set of electrical component categories.

[0054] Based on the electrical component category set, a convolutional neural network is used to divide the electrical component image dataset into grids to obtain a grid image dataset.

[0055] The grid image dataset is traversed and matched with the electrical component class set to generate matching results;

[0056] Based on the matching results, labels are assigned to the electrical component image dataset to generate multiple electrical component category labels;

[0057] A training dataset and a validation dataset are constructed based on the electrical component image dataset and the multiple electrical component category labels;

[0058] The electrical component classification branch is trained based on the training dataset, and the training results are tested using the validation dataset. When the test is passed, the image data of the multiple electrical component categories are obtained.

[0059] In this embodiment, web crawling technology or API interfaces are used to acquire industrial big data related to electrical components. This data includes product descriptions, models, usage scenarios, and technical parameters. Next, natural language processing techniques, such as word segmentation, part-of-speech tagging, and named entity recognition, are used to preprocess the text data. Effective features are extracted from the text data using techniques such as word embedding, TF-IDF, and text topic models. These features reflect the different attributes and categories of electrical components. Based on the extracted features, unsupervised learning algorithms, such as K-means clustering, are used to perform preliminary classification of the electrical components. The preliminary classification results are then verified and corrected. After verification and correction, a set containing multiple categories of electrical components is finally generated, i.e., the electrical component category set. The electrical component category set includes switches, sockets, circuit breakers, etc.

[0060] Next, a convolutional neural network (CNN) is used to divide the electrical component image dataset into grids. Before dividing, a suitable CNN model is selected for image processing, such as VGG, ResNet, or Inception. The electrical component image dataset is loaded before gridding. Preprocessing operations, such as size normalization and color correction, are performed to ensure image quality and consistency. The preprocessed images are then input into the selected CNN model, where convolutional layers divide the images into multiple grids. Each grid contains a portion of the original image's features. Pooling and other operations are applied to each grid to further extract and refine features, generating a gridded image dataset.

[0061] The features extracted from each grid cell are converted into feature vectors. For each grid cell feature vector in the grid image dataset, its similarity to the feature vectors of each category in the electrical parts category set is calculated using cosine similarity, Euclidean distance, etc. A similarity threshold, such as 0.8, is set; a grid cell is considered a match only if its feature vector has a similarity exceeding this threshold with a certain electrical parts category. Based on the similarity calculation results, the electrical parts category to which each grid cell belongs is determined. Through the above process, matching results are generated.

[0062] Next, based on the matching results, labels are assigned to the electrical component image dataset, generating multiple electrical component category labels. For example, if the electrical component image dataset contains a switch button, a "switch" label is generated; if the image contains a socket, a "socket" category label is generated.

[0063] Training and validation datasets were constructed based on an image dataset of electrical components and multiple category labels for electrical components. 70% of the data was allocated to the training dataset, and the remaining 30% to the validation dataset. A random algorithm was used to randomly divide the dataset into training and validation sets.

[0064] Next, a convolutional neural network was selected for training, and the chosen model was trained using a training dataset. During this process, the model learned how to extract features from images and classify electrical components based on these features. Optimization algorithms, such as gradient descent, were used during training to minimize classification error, thereby improving the model's accuracy. After training, a validation dataset was used to test the model's performance. Once the test was passed, the electrical component image dataset was classified, generating multiple categories of electrical component images.

[0065] Furthermore, the defect detection module 14 is also used for:

[0066] Based on the electrical component class set, the standard image set of electrical components is extracted by traversing the electrical component image repository.

[0067] Based on YOLO, target detection is performed on the image data of the multiple electrical component categories according to the standard image set of electrical components to generate target detection sample data;

[0068] Based on the target detection sample data, defects are identified, and based on the defect identification results, defects are located to generate defect location information.

[0069] The defect location information is added to the image data of the multiple defective electrical components.

[0070] In this embodiment, an electrical component image repository is traversed using a programming language and relevant file manipulation libraries, such as Python's `os` or `glob` libraries, based on the electrical component category set. The electrical component image repository is pre-prepared and contains a large number of electrical component images already categorized and stored. Each image is accompanied by relevant metadata, such as capture time, equipment information, and component type. During traversal, images of corresponding categories are filtered based on the category information in the electrical component category set. Filtering is achieved through filenames, file paths, or category tags in the file metadata. For images with category information in their filenames or paths, string matching or regular expressions are used for filtering; for images containing category tags, the tags are read and compared. For the initially filtered images, further quality assessment and filtering are performed, using image processing techniques such as edge detection and contrast analysis to evaluate image sharpness and detail retention. Simultaneously, manual review is conducted to ensure that the selected standard images accurately represent the normal state of the electrical component category. Through the above process, a standard image set for electrical components is extracted.

[0071] Next, a standard image set of electrical components is used to train the YOLO model, enabling it to identify electrical components in the images. Multiple image datasets of electrical component categories are input into the trained YOLO model. The YOLO model performs forward propagation calculations on the input images, extracts image features through a convolutional neural network, and predicts the location and category of the electrical components. To remove highly overlapping predicted boxes and improve detection accuracy, the YOLO algorithm employs non-maximum suppression (NMS) to filter the predicted boxes. After processing by the YOLO model, the location and category information of each detected electrical component are extracted from the model's output, yielding the target detection sample data. The target detection sample data includes image filenames, electrical component categories, bounding box coordinates, etc.

[0072] The target detection sample data is input into the defect recognition model for identification. The defect recognition model extracts features from each image patch, including color variations, texture anomalies, and shape distortions. These extracted features are then classified, and by comparing them with normal samples, abnormal features, i.e., defects, are identified. Based on the classification results, the model determines whether defects exist in the image patch and identifies the type of defect. Once a defect is identified, it is located by outputting a heatmap or segmentation map. In the heatmap, the model highlights the defect area; the darker the color, the greater the likelihood of a defect. The segmentation map more precisely identifies the boundaries and extent of the defect. Through this process, defect location information is generated.

[0073] Finally, based on the identifiers in the defect location information, such as file name and image ID, the defect location information is matched with the corresponding electrical component images, and the defect location information is added to multiple defective electrical component image data.

[0074] Furthermore, the defect classification module 15 is also used for:

[0075] Retrieve historical defect data records of the multiple electrical components, and extract multiple defect sample data from the historical defect data records;

[0076] The feature description operator is used to detect the multiple defect sample data to obtain multiple feature points;

[0077] The collection range is determined based on the multiple feature points, and the multiple feature points are calculated according to the collection range to generate a defect feature matrix;

[0078] Based on the defect feature matrix, a vector transformation is performed to obtain an initial vector set of defect features;

[0079] The initial set of defect feature vectors is reduced in dimensionality using the vector similarity probability distribution function to obtain the defect feature vector set.

[0080] In this embodiment, historical defect data records for multiple electrical components are first retrieved from a historical database. These records contain detailed information about various previously detected defects, such as defect type, location, and size. Multiple defect sample data are then selected from the retrieved historical defect data records. These multiple defect sample data cover different types of defects.

[0081] When using feature descriptor operators to detect multiple defect sample data, this application selects SIFT, SURF, ORB, etc. The defect sample data is then preprocessed, including grayscale conversion, noise reduction, and contrast enhancement, to improve the accuracy of feature point detection.

[0082] The selected feature descriptor is run on the preprocessed defective sample data. This process automatically scans the image, detects and extracts feature points. For SIFT, a scale-space pyramid is constructed, and extrema are detected at multiple scales using the difference of Gaussians function. For each extrema, a gradient orientation histogram in its neighborhood is calculated, generating a 128-dimensional descriptor. For SURF, extrema are detected using the Hessian matrix. A principal orientation is assigned to each extrema, and a 64-dimensional descriptor is calculated. For ORB, corners are detected using the FAST algorithm. For each corner, its orientation is determined using the gray-scale centroid method, and a short binary descriptor is calculated. After the feature descriptor runs, a series of feature points are output. Each feature point contains location, scale, orientation, and descriptor.

[0083] Multiple detected feature points are grouped or clustered. Since the feature points are distributed in different locations within and around the defect area, clustering algorithms such as K-means and DBSCAN are used to group similar feature points together. Next, the acquisition range is determined. For each cluster, its center position is calculated based on the average position of the feature points, and the cluster boundary is determined by calculating the maximum and minimum coordinate values ​​of the feature points within the cluster. For each feature point within the acquisition range, its relevant feature information is extracted, including location coordinates, scale, orientation, descriptors, etc. An empty matrix is ​​initialized, and the extracted feature point attributes are filled into the matrix according to the location of the feature points to generate the defect feature matrix.

[0084] The defect feature matrix is ​​flattened by converting each row or column into a feature vector, directly transforming each feature point in the matrix into a vector in a high-dimensional space, where each dimension represents a feature attribute. All the transformed feature vectors are then combined to form the initial defect feature vector set.

[0085] For each pair of vectors in the initial set of defective feature vectors, the similarity between them is calculated using methods such as cosine similarity and Euclidean distance. Next, the divergence of the similarity is obtained based on the vector similarity probability distribution function. This divergence describes the similarity relationship between the vectors. Finally, the initial set of defective feature vectors is dimensionality reduced using t-SNE to obtain the defective feature vector set.

[0086] Furthermore, the defect classification module 15 is also used for:

[0087] Construct the vector similarity probability distribution function:

[0088] ;

[0089] in, As a measure index for characterizing the proximity between the feature point distribution of the initial defect feature vector set and the dimensionality-reduced feature point distribution of the initial defect feature vector set, let tend to 0 as the constraint condition for dimensionality reduction processing, be the divergence representing the similarity between the feature point distribution of the initial defect feature vector set and the dimensionality-reduced feature point distribution of the initial defect feature vector set, be the similarity probability regarding the feature point and the feature point in the dimensionality-reduced feature point distribution of the initial defect feature vector set, be the similarity probability regarding the feature point and the feature point in the feature point distribution of the initial defect feature vector set. (i, j) is any coordinate point in the defect feature matrix, where the value range of i is 0 < i < n, the value range of j is 0 < j < n, and n is the total number of coordinate points in the defect feature matrix.

[0090] In the embodiments of the present application, is the divergence representing the similarity between the feature point distribution of the initial defect feature vector set and the dimensionality-reduced feature point distribution of the initial defect feature vector set.

[0091] When calculating through the vector similarity probability distribution function, first quantify the similarity probability between vectors in the initial defect feature vector set by methods such as cosine similarity, Pearson correlation coefficient or Euclidean distance. Substitute the calculated similarity probability into the vector similarity probability distribution function for calculation to obtain the divergence representing the similarity between the feature point distribution of the initial defect feature vector set and the dimensionality-reduced feature point distribution of the initial defect feature vector set.

[0092] Furthermore, the defect classification module 15 is further configured to:

[0093] Perform clustering analysis on the multiple defect electrical accessory image data based on the defect feature vector set to generate a defect electrical accessory clustering result;

[0094] Evaluate according to the defect electrical accessory clustering result, and perform identification according to the evaluation result according to the defect category to determine multiple defect categories;

[0095] Classify the multiple defect electrical accessory image data according to the multiple defect categories to obtain the electrical accessory image data of the multiple defect categories.

[0096] In this embodiment, clustering algorithms such as K-means, hierarchical clustering, and DBSCAN are used to cluster the defect feature vector set. After clustering, the clustering result of defective electrical parts is obtained. This result groups the defect feature vectors into several clusters, each cluster representing a type of defective electrical part. Through the above process, the clustering result of defective electrical parts is obtained.

[0097] Internal evaluation metrics, such as the silhouette coefficient and Davies-Bouldin Index, are used to measure the quality of clustering. These metrics assess clustering effectiveness based on the compactness within clusters and the separation between clusters. Next, the clustering results are interpreted to determine and label the type of defect represented by each cluster. When samples with known defect categories are available, they are compared with the clustering results, and the defect category of each cluster is determined based on similarity. Finally, based on the labels from the previous step, a specific defect category name is assigned to each cluster, such as crack, stain, or deformation.

[0098] Finally, based on the results of cluster analysis and defect category identification, multiple defective electrical component image data are assigned to corresponding defect categories, resulting in multiple defective electrical component image data categories.

[0099] Furthermore, the intelligent defect sorting module 17 is also used for:

[0100] Based on the simulated sorting results, a response analysis is performed to generate sorting response results;

[0101] Based on the sorting response results, the electrical component sorting channel is traced back to determine the abnormal data source.

[0102] The electrical parts sorting channel is verified based on the backtracking anomaly data source. The electrical parts sorting channel is iteratively optimized based on the verification results. When the electrical parts sorting channel meets the preset sorting performance indicators, the optimized electrical parts sorting channel is output.

[0103] In this embodiment, the simulated sorting results include, but are not limited to, sorting accuracy, sorting speed, time consumption of each sorting step, and the type and number of sorting errors. The collected simulated sorting results are statistically analyzed using Excel or other data processing software. The ratio of correct sorting times to the total number of sorting times is calculated to obtain the sorting accuracy rate. For cases of sorting errors, common error types are identified, such as missorting, omissions, and incorrect placement, and the frequency of each type of error is statistically analyzed. Data analysis software is used to calculate the average time, fastest time, and slowest time for each sorting operation. The results of the accuracy and efficiency analysis are compiled to generate sorting response results. The sorting response results include sorting accuracy rate, error type statistics, and sorting time analysis.

[0104] When backtracking the electrical component sorting channel, all erroneous sorting records are identified based on the sorting response results. These records serve as the starting point for backtracking analysis. For each erroneous sorting record, the time of occurrence and the status and operation of the sorting channel before and after that time are examined. Erroneous sorting records are compared with correct sorting records to identify differences. These differences point to abnormal data sources causing the sorting errors. The sorting error frequency for various types of electrical components and the frequency of errors in each sorting step are statistically analyzed. Electrical components or sorting steps with high error rates may be abnormal data sources. The above process is used to determine the abnormal data sources for backtracking.

[0105] Next, professional logistics sorting simulation software, such as FlexSim, Simulink, or other relevant simulation tools, will be selected. Based on the actual sorting conditions of the electrical components sorting channels, a corresponding sorting channel model will be constructed in the simulation software. The abnormal data sources obtained from the previous backtracking analysis will be integrated into the simulation model to reproduce these abnormal situations during the simulation process.

[0106] Based on actual needs, set parameters such as simulation duration, quantity and type of items to be sorted. Start the simulation software and observe the performance of the sorting channel with integrated backtracking anomaly data sources. Record key data such as sorting accuracy, sorting speed, and channel throughput during the simulation process.

[0107] Based on the simulation data analysis results, statistical methods are used to process the collected data, calculating the average, standard deviation, maximum, and minimum values ​​of various indicators. By analyzing the statistical data, problematic stages in the sorting process are identified. For example, the sorting time in a certain stage may be significantly longer, or the sorting error rate of a certain piece of equipment may be significantly higher than that of other equipment. Based on the problem identification results, adjustments are made to the sorting process. For example, if a certain stage is found to frequently cause congestion, the buffer zone for that stage may be increased or the sorting order adjusted. For equipment with poor performance, hardware upgrades or equipment parameters may be performed. For example, the conveyor belt speed may be increased, or the mechanical structure of the sorting device may be optimized.

[0108] In the simulation software, the sorting channel model is adjusted according to the established optimization plan. The simulation software is then rerun to verify whether the performance of the optimized sorting channel has improved. This process is iterative until the electrical components sorting channel meets the preset sorting performance indicators. The preset sorting performance indicators are set by the user according to their needs. Specifically, the required sorting accuracy rate is over 99%, meaning that at least 99 out of every 100 sorting operations should be accurate. The required sorting speed is at least 1000 items per minute.

[0109] When the electrical parts sorting channel meets the preset sorting performance indicators, the optimized electrical parts sorting channel scheme shall be used as the optimized electrical parts sorting channel.

[0110] In summary, the embodiments of this application have at least the following technical effects:

[0111] This application uses an industrial camera to continuously acquire images of multiple electrical components on a target production line, generating an image dataset of these components. This dataset is then transmitted to an electrical component sorting channel, which includes a classification branch, a defect recognition branch, and a defect classification branch. The classification branch classifies the image dataset based on a set of electrical component classes, obtaining multiple image datasets for each class. The defect recognition branch identifies defects in these image datasets based on a standard image set of electrical components, generating multiple image datasets of defective electrical components. The defect classification branch classifies these defective image datasets based on a set of defect feature vectors, generating image datasets of electrical components with multiple defect categories. Sorting instructions are generated based on these image datasets, and the components are simulated for sorting according to these instructions, generating simulated sorting results. The sorting channel is then optimized based on these simulated sorting results, and an optimized sorting channel is used to perform intelligent defect sorting of the components. This invention solves the technical problem of inaccurate defect identification in existing technologies, which leads to inaccurate defect sorting categories, and achieves the technical effect of flexible and adaptive sorting of defective electrical parts.

[0112] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0113] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0114] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A machine vision based blemished electrical component sorting system, characterized in that, The system comprises: a data acquisition module, which continuously acquires a plurality of electrical fittings of a target flow line through an industrial camera, to generate electrical fitting image data sets; a defect sorting module, which transmits the electrical fitting image data sets to an electrical fitting sorting channel, wherein the electrical fitting sorting channel comprises an electrical fitting classification branch, a defect recognition branch, and a defect classification branch; a classification processing module, which classifies the electrical fitting image data sets based on an electrical fitting class set through the electrical fitting classification branch, to obtain a plurality of electrical fitting class image data; a defect recognition module, which recognizes defects in the plurality of electrical fitting class image data based on an electrical fitting standard image set through the defect recognition branch, to obtain a plurality of defective electrical fitting image data; a defect classification module, which classifies the plurality of defective electrical fitting image data based on a defect feature vector set through the defect classification branch, to generate electrical fitting image data of a plurality of defect categories; a simulation sorting module, which generates sorting instructions according to the electrical fitting image data of the plurality of defect categories, and simulates sorting of the plurality of electrical fittings according to the sorting instructions, to generate a simulation sorting result; an intelligent defect sorting module, which optimizes the electrical fitting sorting channel according to the simulation sorting result, and sorts the plurality of electrical fittings through an electrical fitting sorting optimization channel; wherein the defect recognition module comprises: extracting the electrical fitting standard image set from an electrical fitting image storage based on the electrical fitting class set; performing target detection on the plurality of electrical fitting class image data according to the electrical fitting standard image set based on a YOLO model, to generate target detection sample data; performing defect recognition based on the target detection sample data, and performing defect positioning according to a defect recognition result, to generate defect point position information; adding the defect point position information to the plurality of defective electrical fitting image data; wherein the defect classification module comprises: calling a historical defect data record archive of the plurality of electrical fittings, and extracting a plurality of defect sample data from the historical defect data record archive; detecting the plurality of defect sample data using a feature description operator, to obtain a plurality of feature points; determining an acquisition range based on the plurality of feature points, calculating the plurality of feature points according to the acquisition range, and generating a defect feature matrix; performing vector conversion according to the defect feature matrix, to obtain a defect feature initial vector set; performing dimension reduction processing on the defect feature initial vector set using a vector similarity probability distribution function, to obtain a defect feature vector set; wherein the vector similarity probability distribution function is constructed as follows: ； wherein, a measure index representing the closeness between the feature point distribution of the initial feature vector set of the defect feature and the reduced dimension feature point distribution of the initial feature vector set of the defect feature, let tends to 0 as a constraint condition for dimension reduction processing, a divergence representing the similarity between the feature point distribution of the initial feature vector set of the defect feature and the reduced dimension feature point distribution of the initial feature vector set of the defect feature, a similarity probability based on the feature point and the feature point in the reduced dimension feature point distribution of the initial feature vector set of the defect feature, a similarity probability based on the feature point and the feature point in the feature point distribution of the initial feature vector set of the defect feature, (i, j) is any coordinate point in the defect feature matrix, the value range of i is 0 2. The machine vision-based blemished electrical component sorting system of claim 1, wherein, the data acquisition module comprises: obtaining movement trajectories of the plurality of electrical fittings in the target flow line, and determining layout position information of the industrial camera based on the movement trajectories; extracting appearance features of the plurality of electrical components to set light source parameters, and determining imaging lighting conditions of the industrial camera; adjusting a layout angle of the industrial camera based on the layout position information and the imaging lighting conditions, and determining camera layout information; collecting the plurality of electrical components of the target pipeline according to preset camera parameters based on the camera layout information, and generating the electrical component image dataset.

3. The machine vision-based blemished electrical component sorting system of claim 1, wherein, The classification processing module includes: retrieving electrical component categories based on industrial big data, and determining the electrical component category set; dividing the electrical component image dataset into a grid image dataset using a convolutional neural network based on the electrical component category set; matching the grid image dataset with the electrical component category set, and generating a matching result; labeling the electrical component image dataset based on the matching result, and generating a plurality of electrical component category labels; constructing a training dataset and a verification dataset based on the electrical component image dataset and the plurality of electrical component category labels; training the electrical component classification branch based on the training dataset, testing the training result using the verification dataset, and obtaining the plurality of electrical component category image data when the test passes.

4. The machine vision-based blemished electrical component sorting system of claim 1, wherein, The defect classification module includes: performing cluster analysis on the plurality of defect electrical component image data based on the defect feature vector set, and generating defect electrical component cluster results; evaluating the defect electrical component cluster results, identifying defect categories based on the evaluation results, and determining a plurality of defect categories; classifying the plurality of defect electrical component image data according to the plurality of defect categories, and obtaining electrical component image data of the plurality of defect categories.

5. The machine vision-based blemished electrical component sorting system of claim 1, wherein, The intelligent defect sorting module includes: performing response analysis based on the simulation sorting result, and generating a sorting response result; backtracking the electrical component sorting channel based on the sorting response result, and determining a backtracking abnormal data source; verifying the electrical component sorting channel based on the backtracking abnormal data source, iteratively optimizing the electrical component sorting channel based on the verification result, and outputting the electrical component sorting optimization channel when the electrical component sorting channel meets preset sorting performance indicators.

Images