A robot-assisted automotive injection molded part defect sorting method and system

Abstract

This invention discloses a robot-assisted method and system for sorting defects in automotive injection molded parts, relating to the field of image recognition technology. The method includes: performing initialization actions on a pre-configured sorting system; acquiring raw image data and triggering preprocessing actions when the target workpiece is transported to a designated area; constructing a deep learning-based defect identification and classification model and introducing a post-processing enhancement unit. Its key technical points are: based on secondary defect proportion statistics and review deviation analysis, a periodic feedback mechanism is constructed to achieve dual-path optimization of image preprocessing and model training; and a hierarchical response mechanism sends suspicious defects into the review channel, forming a continuous optimization feedback learning process. This solves the technical problem that traditional fixed-parameter systems cannot adapt to dynamic changes in the production line such as mold wear and material changes, realizing the adaptive evolution capability of the solution itself and ensuring that the defect identification action can be continuously optimized and improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a robot-assisted method and system for sorting defects in automotive injection molded parts. Background Technology

[0002] Image recognition is a technology that uses computer technology to automatically analyze and understand the content of images and identify objects or features within them. It belongs to the core field of computer vision. Current technologies are mainly based on deep learning, especially convolutional neural networks, such as YOLO and Faster R-CNN models, which can achieve high-precision target detection and classification. Compared with traditional methods that rely on manually designed features, deep learning can automatically extract features and has stronger generalization and adaptability. It has been widely used in industrial defect detection, face recognition, autonomous driving and other fields. When combined with multi-view imaging and lightweight models, it is more suitable for automated systems with high real-time requirements, such as defect sorting of workpieces.

[0003] Currently, defect detection in traditional automotive injection molded parts largely relies on manual visual inspection or fixed camera photo comparison, which suffers from low efficiency and high subjectivity. For example, workers are prone to fatigue when working under strong light for extended periods, and small flashes or shrinkage in dark areas are often missed. Early automated solutions used fixed-angle photography with simple threshold segmentation, which resulted in numerous false alarms when encountering reflective, shadowy, or structurally complex areas, such as the intersection of reinforcing ribs. More importantly, even with relevant adjustments, these systems lack an understanding of the spatial relationships and morphological patterns between defects. Common problems include the inability to determine whether two short scratches belong to the same weld line, and difficulty in distinguishing between genuine pits and false defects formed by mold release agent reflection. Furthermore, once traditional models are deployed, parameters become fixed or parameter feedback adjustments are ineffective. When the mold wears slightly or the surface texture changes due to material batch changes, the original detection logic quickly fails, requiring repeated shutdowns for debugging. The entire process cannot guarantee stability or achieve self-optimization, severely hindering subsequent defect sorting operations. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A robot-assisted method for sorting defects in automotive injection molded parts, the method comprising:

[0006] Perform initialization actions on the pre-configured sorting system;

[0007] When the target workpiece is transported to the designated area, the original image data is acquired and the preprocessing action is triggered;

[0008] A deep learning-based defect identification and classification model is constructed, and a post-processing enhancement unit is introduced to obtain the defect identification results. The operation of the post-processing enhancement unit is based on: constructing an initial identification function model, setting multi-dimensional constraints, and triggering an iterative overlap correction mechanism for collaborative correction.

[0009] Based on the operation results of the post-processing enhancement unit, a graded response mechanism is triggered to classify the target defects into different grades. Within a preset period, the comparison results between the target grade proportion and the preset standard threshold are extracted. When the target grade proportion exceeds the preset standard threshold, the feedback strategy is activated to complete the optimization process through dual-path feedback.

[0010] After the defect identification results are determined, the robotic arm in the execution end of the scheduling and sorting system completes the sorting action.

[0011] Furthermore, the sorting execution system includes at least a processing end and an execution end; wherein, the processing end is used to send instructions to the execution end, and the execution end also includes: a camera array, a light source subsystem, and a conveying system; the initialization action includes at least: calibrating the intrinsic and extrinsic parameters of the camera array configured in the execution end; the robotic arm performs a zeroing action and loads a preset grasping trajectory library.

[0012] Furthermore, the original image data includes at least a combined image set consisting of a top view, a front view, and a side view of the target workpiece;

[0013] The triggered preprocessing steps are as follows: denoising is performed using nonlocal mean denoising combined with wavelet transform filtering; grayscale and contrast enhancement are performed, and the pre-configured CLAHE algorithm is applied; morphological closing operations are run to fill the pseudo-fracture region; the target workpiece contour is initially extracted through Canny edge detection; and SURF-based key point matching technology is introduced simultaneously to register and fuse the original image data to generate a 3D point cloud map.

[0014] Furthermore, the operation of the defect identification and classification model is based on:

[0015] In the backbone network of the defect identification and classification model, CSPDarknet53 is used as the basic structure, and SE Blocks are embedded in its key residual blocks to implement the channel attention mechanism. A 1×1×C vector z is generated by squeezing each feature map channel through global average pooling (GAP). c ;

[0016] The stimulation process is completed on two fully connected layers, and the normalized weight vector s is output.

[0017] In the feature pyramid part of the defect identification and classification model, BiFPN is used to achieve efficient cross-scale fusion, through the weight parameter w. iThe importance of each input path is dynamically adjusted to obtain the output feature map O.

[0018] Furthermore, when constructing the initial recognition function model, let the preliminary detection results be set D={(b k c k p k )} k=1 K ;

[0019] Among them, b k =(x k y k W k H k ) represents the center coordinates, width, and height of the candidate bounding box corresponding to the k-th defect; c k ∈{1, 2, ..., 12} represents the category of the k-th defect, serving as the category label; p k ∈(0,1], which is the confidence score; k=1,2,...,K, where K represents the maximum value of k; when setting multi-dimensional constraints, the set of constraints introduced includes at least: first constraint, second constraint, third constraint and fourth constraint; among which, the first constraint adopts geometric consistency constraint, the second constraint adopts material reflection consistency constraint, the third constraint adopts size rationality constraint, and the fourth constraint adopts topological structure constraint.

[0020] Furthermore, the iterative overlap correction mechanism works as follows: after each correction, a new detection result set D' is generated, which is then intersected with the previous round of detection results to output the final detection result set D_final. Simultaneously, a pre-built confidence accumulation function is run to score the final detection results, outputting the final confidence score pf_final for the k-th defect. k .

[0021] Furthermore, the hierarchical response mechanism operates as follows: when pf_final... k If the value is ≥0.8Qt, it is classified as a Level 1 defect, representing a high-confidence real defect; if 0.5Qt≤pf_final k If the value is less than 0.8Qt, it is classified as a Level 2 defect, indicating a medium-confidence, questionable defect; when pf_final... k If the value is less than 0.5Qt, it is classified as Level 3, indicating a low-confidence false alarm tendency; where Qt represents a set value, and its range is greater than 0.

[0022] The comparison results between the target level percentage extracted within the preset period and the preset standard threshold are based on the following:

[0023] If the target level is level 2, the total number of target defect judgments within a preset period is obtained. The total number of target defect judgments within the preset period is divided by the number of times the target defect is judged as level 2 to obtain the target level proportion. If the target level proportion exceeds the preset standard threshold, the judgment is in an unstable state; otherwise, the feedback strategy is not activated.

[0024] Furthermore, the process of initiating the feedback strategy includes: Path 1: Feedback to the image preprocessing stage, the corresponding optimization actions include at least enhanced de-polarization processing: enabling polarization imaging mode for the target workpiece, acquiring multi-angle polarized light images, and synthesizing depolarized images; Path 2: Feedback to the model training stage, the corresponding optimization actions include at least: constructing an incremental training set and initiating incremental fine-tuning; wherein, constructing an incremental training set: extracting all samples corresponding to level 2 targets within a preset period as a hard sample set, sending them to the labeling queue, and re-labeling the defect location and category, and adding them to the training data pool after labeling; initiating incremental fine-tuning involves: using the newly added data to the training data pool to perform lightweight training on the model.

[0025] Furthermore, if the defect is identified as Level 1 after identification, it is sorted to the first channel by a robotic arm; if it is identified as Level 2, it is sorted to the second channel by a robotic arm; if it is identified as Level 3, no response is made; among them, the first channel is the rework channel and the second channel is the review channel.

[0026] A robot-assisted defect sorting system for automotive injection molded parts, the system comprising:

[0027] Initialization module: Performs initialization actions on the pre-configured sorting system;

[0028] Preprocessing module: When the target workpiece is transported to the designated area, it acquires raw image data and triggers preprocessing actions;

[0029] Defect recognition module: Constructs a deep learning-based defect recognition and classification model, and introduces a post-processing enhancement unit to obtain defect recognition results; The operation of the post-processing enhancement unit is based on: constructing an initial recognition function model, setting multi-dimensional constraints, and triggering an iterative overlap correction mechanism for collaborative correction;

[0030] Feedback Correction Module: Based on the operation results of the post-processing enhancement unit, a graded response mechanism is triggered to divide the target defects into different grades. Within a preset period, the comparison results between the target grade proportion and the preset standard threshold are extracted. When the target grade proportion exceeds the preset standard threshold, the feedback strategy is activated to complete the optimization process through dual-path feedback.

[0031] Sorting Execution Module: After the defect identification results are determined, the robotic arm in the execution end of the sorting execution system is scheduled to complete the sorting action.

[0032] This invention provides a robot-assisted method and system for sorting defects in automotive injection molded parts, which has the following advantages:

[0033] (1) This scheme achieves high-contrast imaging of weak defects on the surface of the target workpiece through the coordinated configuration of various components in the execution end, which significantly improves the image signal-to-noise ratio. At the same time, by combining non-local mean denoising and CLAHE enhancement algorithm, it effectively suppresses ambient light interference and local shadow artifacts, preserves real texture details, and solves the problems of missed detection or misjudgment caused by occlusion and uneven lighting in traditional single-view imaging, providing an effective data foundation for subsequent analysis.

[0034] (2) This scheme utilizes the improved lightweight YOLOv8 architecture DefectNet-V3 and introduces the SE attention mechanism and BiFPN cross-scale fusion module, enabling the model to accurately capture the channel features and multi-scale spatial information of minor defects with low latency. It can also use the pre-set labeled dataset for pre-training and embedding of category prior knowledge, which enhances the model's semantic understanding of typical defects such as fly edges and weld lines. To a certain extent, it achieves a recognition mechanism that balances real-time performance and accuracy, and provides reliable initial screening results for subsequent fine-tuning.

[0035] (3) This scheme utilizes the combined effect of multiple constraints to improve the accuracy and robustness of the overall scheme operation results. The first constraint ensures that the spatial distribution of adjacent defects on the same workpiece conforms to the mold structure law, avoiding false detection. The second constraint utilizes multi-view image information to ensure that the detection results are not affected by lighting, and can distinguish between real depressions and reflective artifacts to a certain extent. The third constraint defines a reasonable size range based on historical data to prevent false alarms of abnormal sizes. The fourth constraint filters out false positive results that do not conform to the expected shape based on the geometric shape of the defect. Through the above four constraints, the overall scheme can more accurately identify various defects on automotive injection molded parts and reduce false detection and missed detection.

[0036] (4) On the one hand, this scheme introduces a set of constraints and constructs an iterative overlap correction mechanism. It eliminates abnormal candidate defects through set intersection operation. On the other hand, it designs a confidence accumulation function to dynamically weight the support coefficients of each constraint according to the defect category, thereby realizing the soft fusion of confidence scores. This solves the problems of traditional pure model-driven methods being susceptible to noise interference, having a high false alarm rate, and lacking interpretability. It also improves the reliability of discrimination through multi-dimensional fusion.

[0037] (5) Based on the statistics of the proportion of secondary defects and the analysis of the deviation of the review, this solution constructs a periodic feedback mechanism to realize the dual-path optimization of image preprocessing and model training. Through the hierarchical response mechanism, suspicious defects are sent to the review channel to form a learning process of continuous optimization feedback. This solves the technical problem that the traditional fixed parameter system cannot adapt to the dynamic changes of the production line such as mold wear and material changes. It realizes the adaptive evolution capability of the solution operation itself and ensures that the defect identification action can be continuously optimized and improved. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the steps in a robot-assisted defect sorting method for automotive injection molded parts according to the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1:

[0041] Please see Figure 1 This embodiment provides a robot-assisted method for sorting defects in automotive injection molded parts. The core of this sorting method is to effectively and stably identify defects in automotive injection molded parts, thereby ensuring accurate subsequent sorting.

[0042] The specific steps of this sorting method are explained below:

[0043] S1. Perform initialization actions on the pre-configured sorting system;

[0044] The sorting execution system includes a processing end and an execution end. The processing end, as the software, is used to process and analyze image data and send instructions to the execution end. The execution end, as the hardware, is used to send the acquired image data to the processing end and synchronize the instructions sent by the receiver. The execution end includes a camera array, a light source subsystem, a robotic arm, and a conveying system. The camera array can be an industrial-grade camera array, deployed in three positions above, in front of, and diagonally to the designated detection area to achieve synchronous shooting. The light source subsystem uses a ring LED light source system to ensure clear imaging during shooting. In this embodiment, the robotic arm can be a six-axis collaborative robotic arm used to pick up, place, and sort corresponding workpieces, i.e., automotive injection molded parts. This robotic arm is an auxiliary robot. The conveying system includes a conveyor belt and photoelectric encoders installed on it for real-time tracking of workpiece position, mainly for conveying.

[0045] Therefore, the initialization actions performed by the sorting system should include at least the following:

[0046] The camera array configured in the execution end is calibrated using a calibration board to ensure distortion-free imaging;

[0047] The robotic arm performs a zeroing action and loads a preset gripping trajectory library; the zeroing action means returning to the initial set state, and the preset gripping trajectory library contains the regular gripping action trajectories for the same type of workpiece in the past.

[0048] Specifically, after the system starts up, it enters the initialization state to achieve synchronous calibration. Based on this, a unified data communication protocol, such as Modbus TCP / IP, is set to realize the response linkage between the processing end and the execution end. In addition, to improve robustness, redundancy design can be selectively introduced. That is, when the main control server in the sorting execution system fails, the edge nodes can take over the basic sorting logic to ensure continuous operation. At the same time, this solution can effectively eliminate glare by dynamically adjusting the light source angle and enabling polarization mode, so that the real texture is clearly presented in the image, laying the foundation for subsequent accurate recognition.

[0049] S2. When the target workpiece is transported to the designated area, the original image data is collected and the preprocessing action is triggered;

[0050] In this embodiment, the target workpiece refers to an automotive injection molded part, which is transported to the designated area via a conveyor system in the execution end. The acquired raw image data is a combination of top view, front view, and side view images of the target workpiece. The triggered preprocessing action is as follows:

[0051] S2.1. Perform denoising operation, using non-local mean denoising combined with wavelet transform filtering; its function is to effectively remove noise without losing detail and texture. S2.2. Perform grayscale conversion and contrast enhancement, and apply the pre-configured CLAHE algorithm; specifically, limit contrast adaptive histogram equalization to improve the visibility of dark details, suitable for detecting minor scratches or shrinkage patterns on the target workpiece. S2.3. Run morphological closing operation to fill pseudo-fracture regions, initially extract the outline of the target workpiece through Canny edge detection, and simultaneously introduce SURF-based keypoint matching technology to register and fuse the original image data, generating a 3D point cloud map to assist in subsequent spatial defect localization; the pseudo-fracture regions are caused by actual shadows; then, when registering and fusing the original image data, three images from different perspectives are used, namely the aforementioned top view, front view, and tilt view of the target workpiece; the tilt angle in this embodiment typically refers to 45°.

[0052] It should be noted that in the critical image data acquisition stage, the corresponding solution under S2 aims to comprehensively capture the three-dimensional surface information of automotive injection molded parts through multi-view and multi-spectral imaging strategies, overcoming the problems of single-view occlusion and local feature omission. Among them, each camera in the camera array is equipped with megapixel resolution and the frame rate is set to no less than 60fps to ensure clear imaging under the high-speed pipeline formed during the operation of the conveyor system. In addition, the light source subsystem can also automatically adjust the intensity and wavelength of LED light sources in each direction according to the pre-stored part type-lighting configuration mapping table. For example, blue light illumination is enhanced for light-colored translucent parts to highlight internal bubbles, and white light diffused illumination is used for dark matte parts to highlight the contour edges. This part of the configuration will not be elaborated here.

[0053] By adopting the above technical solution and through the coordinated configuration of various components in the execution end, high-contrast imaging of weak defects on the surface of the target workpiece is achieved, significantly improving the image signal-to-noise ratio. At the same time, by combining non-local mean denoising and CLAHE enhancement algorithms, ambient light interference and local shadow artifacts are effectively suppressed, and real texture details are preserved. This solves the problems of missed detection or misjudgment caused by occlusion and uneven lighting in traditional single-view imaging, providing an effective data foundation for subsequent analysis and ensuring the reliability of subsequent model recognition and decision-making.

[0054] S3. Construct a deep learning-based defect identification and classification model, and introduce a post-processing enhancement unit to complete defect correction processing and obtain defect identification results;

[0055] Specifically, the output of the deep learning-based defect identification and classification model, i.e., the preliminary detection results, is refined and corrected. The deep learning-based defect identification and classification model employs an improved lightweight convolutional neural network model, DefectNet-V3, optimized for detecting surface anomalies in automotive injection molded parts. This model is evolved from the YOLOv8 architecture, with key innovations including the introduction of the attention mechanism SE Block and the cross-scale feature fusion module BiFPN, maintaining a balance between low latency and defect capture accuracy. The training dataset used in the deep learning-based defect identification and classification model covers over 100,000 labeled images, encompassing over 100 typical injection molded parts from mainstream car models, such as dashboard frames, air conditioning ducts, and bumper liners. Each image is labeled with defect categories by professional quality inspectors and includes bounding box coordinates and confidence scores.

[0056] The operation of the deep learning-based defect identification and classification model is based on the following:

[0057] S3.1. In the backbone network of this model, CSPDarknet53 is used as the basic structure, and SE Blocks are embedded in its key residual blocks to implement the channel attention mechanism; a 1×1×C vector z is generated by squeezing each feature map channel through global average pooling (GAP). c The basis is:

[0058] ;

[0059] In the formula, H and W are the height and width of the feature map, respectively, and the feature map is the original image data after preprocessing; x c (i, j) represents the activation value of the c-th channel at position (i, j), where C is the number of channels;

[0060] S3.2. Complete the stimulation process for the two fully connected layers and output the normalized weight vector s, based on:

[0061] s=σ(W2δ(W1z));

[0062] In the formula, W1∈ C / r×C W2∈ C×C / r Let r be the weight matrix for dimensionality reduction and δ be the weight matrix for dimensionality increase, r be the compression ratio, δ be the ReLU activation function, and σ be the Sigmoid function. The output normalized weight vector s∈[0,1] C The weight vector s is multiplied with the feature map channel by channel for adaptive enhancement of the feature channels, which improves the model's sensitivity to minor defects. Minor defects include silver streaks, shallow scratches, etc. It is the set of real numbers;

[0063] S3.3 In the feature pyramid part of this model, this scheme abandons the traditional FPN or PANet structure and adopts BiFPN to achieve efficient cross-scale fusion. Its core idea is to allow information to flow bidirectionally between different resolution levels and to achieve this through learnable weight parameters w. i The importance of each input path is dynamically adjusted based on:

[0064] ;

[0065] In the formula, O represents the output feature map, which is the final result after weighted summation and is used for subsequent processing or decision-making; i1 represents the index, indicating the different sources of the input feature map; w i1 I is the learning weight for the i1th input feature map. i1This represents input feature maps from different layers. These feature maps may come from different depths of the neural network. Shallow feature maps contain detailed information, such as edges and textures, while deep feature maps contain semantic information, such as the shape and category of objects; qt is the minimum constant used to prevent the denominator from being 0; ∑ j1 w j1 The learned weights of all paths are summed, where j1 is also an index, similar to i1, but used as the denominator here, to ensure weight normalization. This step helps to balance the relative importance between different input feature maps and ensures that no single input dominates the entire output due to its excessively large original weight.

[0066] It should be noted that the formula used in S3.3 performs a weighted summation of input feature maps from different levels to produce a comprehensive output feature map; where each input feature map I... i Each has a corresponding weight w i1 The sum of all weights and the minimum constant qt are added to the denominator to adjust their contribution to the final output. In order to prevent any set of weights from being absolutely dominant and to ensure the validity of mathematical operations, the sum of all weights and the minimum constant qt are added to the denominator.

[0067] By adopting the above technical solutions, utilizing the improved lightweight YOLOv8 architecture DefectNet-V3, and introducing the SE attention mechanism and BiFPN cross-scale fusion module, the model can accurately capture the channel features and multi-scale spatial information of minor defects even with low latency. It can also use pre-set labeled datasets for pre-training and embedding of category prior knowledge, which enhances the model's semantic understanding of typical defects such as fly edges and weld lines. To a certain extent, it achieves a recognition mechanism that balances real-time performance and accuracy, and provides reliable initial screening results for subsequent refined corrections.

[0068] The introduced post-processing enhancement unit is:

[0069] The multi-stage defect correction mechanism based on function models and constraints does not rely on additional training. Instead, it dynamically constructs mathematical function relationships during the inference stage and applies prior knowledge from the physical and technological levels as constraints, thereby refining the preliminary detection results.

[0070] The operation process of the post-processing enhancement unit is as follows:

[0071] S3.4 Constructing the initial recognition function model:

[0072] Let the preliminary detection results be set D = {(b k c k p k )} k=1K Among them, b k =(x k y k W k H k ) represents the center coordinates, width, and height of the candidate bounding box corresponding to the k-th defect; c k ∈{1, 2, ..., 12} represents the category of the k-th defect, serving as the category label; in this embodiment, there are a total of 12 defect categories: flash, shrinkage, excessively long weld lines, excessively deep ejector pin marks, silver streaks, black spots, flow marks, warping, missing material, bubbles, scratches, and oil stains. In actual operation, the number of categories may exceed or fall below 12; p k ∈(0,1], which is the confidence score; define the mapping function f: 4 →C× That is, the joint output from the bounding box space to the defect category and confidence score; in the initial state, f is directly provided by DefectNet-V3; k=1, 2, ..., K, where K represents the maximum value of k;

[0073] S3.5, Set multi-dimensional constraints:

[0074] Introduce a set of constraints, which includes at least the first constraint, the second constraint, the third constraint, and the fourth constraint.

[0075] In this embodiment, the first constraint represents the geometric consistency constraint:

[0076] For multiple adjacent defects on the same target, their spatial distribution should conform to the mold structure rules. Weld lines usually appear at the confluence of two material flows, near the intersection of reinforcing ribs. Therefore, a distance function is defined based on the following:

[0077] ;

[0078] In the formula, d i2j2 b is the distance between two suspected defective frames. i2 b j2 τ_d represents the center coordinates of the i2th and j2nd defect frames, respectively; τ_d represents the set distance threshold. If the distance between two suspected defect frames is less than the distance threshold, they are considered to be the same continuous defect.

[0079] In this embodiment, the second constraint represents the material reflection consistency constraint:

[0080] Using the characteristics of the intensity change of multi-view images obtained in S2, a relationship model between the surface normal and the illumination direction is established. If the brightness change of a certain area under multiple views does not conform to the Lambertian reflection law: I(θ)=I_0cos(θ), where I(θ) is the observed brightness, θ is the incident angle, and I_0 is the initial intensity of the incident light, it is determined as a non-genuine depression, which may be a highlight artifact.

[0081] The third constraint represents the dimensional rationality constraint in this embodiment:

[0082] The target size range of each type of defect, that is, the typical size range, is statistically obtained in advance based on historical data; for example, the flash width generally does not exceed 0.3 mm, and the bubble diameter mostly ranges between 0.5 and 2 mm. The upper and lower limit functions are set: a k ∈[μ_0 - 2σ_0, μ_0 + 2σ_0]. If it exceeds, a review process is performed; where, a k represents the actual size parameter of the k-th defect, which at least includes: width, diameter, etc. μ_0 is the target size mean of each type of defect, and σ_0 is the typical size standard deviation of each type of defect; the review process usually adopts manual review in this embodiment.

[0083] The fourth constraint represents the topological structure constraint in this embodiment:

[0084] Some defects have specific morphological rules. For example, the pits formed by shrinkage are often circular or elliptical. The shape factor Fr is used: Fr = 4πAr / Pc 2 When Fr < Fr_0, it is excluded; where, Ar represents the area of the defect region, Pc represents the perimeter of the defect region, and Fr_0 represents the shape factor threshold, with a value range of 0 to 1. In this embodiment, it is usually set to 0.7. Therefore, if the shape factor of the shrinkage defect is less than 0.7, it is considered that it does not conform to the typical circular or elliptical characteristics and may be noise or other non-defect characteristics, so it is excluded.

[0085] The above-set multi-dimensional constraint conditions work together to improve the accuracy and robustness of the detection system. The first constraint ensures that the spatial distribution of adjacent defects on the same workpiece conforms to the mold structure law, avoiding false detection; the second constraint uses multi-view image information to ensure that the detection result is not affected by illumination and distinguishes genuine depressions and reflection artifacts; the third constraint defines a reasonable size range based on historical data to prevent false alarms of abnormal sizes; the fourth constraint filters out false positive results that do not conform to the expected shape based on the geometric morphology of the defects; through the above four constraint conditions, the system can more accurately identify various defects on automotive injection molded parts, reducing the situations of false detection and missed detection.

[0086] S3.6, Multi-round overlapping data correction and fusion decision-making:

[0087] Based on the aforementioned set of constraints, an iterative overlap correction mechanism is triggered for collaborative correction to obtain the final detection result; specifically, after each correction, a new detection result set D' is generated, and its intersection with the previous round of detection results is calculated.

[0088] ;

[0089] In the formula, D_final represents the final set of detection results, that is, the set of real defects retained after multiple rounds of constraint screening; t represents the iteration round index, and t=1, 2, ..., T; T represents the total number of iterations, which is set to 4 in this scheme, that is, four rounds of screening are performed by applying four constraints in sequence; D(t) represents the candidate defect set in the t-th round.

[0090] Simultaneously, a pre-built confidence accumulation function is run to score the final detection results, based on the following criteria:

[0091] ;

[0092] In the formula, pf_final k For the final confidence score of the k-th defect, pf_init k The initial confidence score for the k-th defect is given, where m is the constraint type index, m = 1, 2, ..., M. In this embodiment, M is 4, representing the total number of constraints; α m (c k ) represents the condition of the m-th constraint on the k-th defect (category: c) k The support coefficient is 0 to 1, representing the reliability of the constraint in identifying this type of defect. It is derived from historical accuracy statistics. For example, for black spot defects, the support coefficient corresponding to the second constraint is higher because it is easily confused by reflection. For material shortage defects, the support coefficient corresponding to the third constraint is higher. The higher value is at least 0.9, which is set according to actual needs.

[0093] Traditional methods employ hard filtering, eliminating items that do not meet any constraint. This approach may be too conservative, leading to a higher false negative rate. However, if left unchecked, the false positive rate will rise. Therefore, this solution uses a confidence accumulation function, which incorporates the support level of each constraint into the final confidence level in a continuous numerical manner through a weighted product. This achieves a more flexible fusion decision. Each constraint verification provides evidence of the existence of the defect, and the final confidence level is the cumulative representation of all evidence, thus effectively reducing both the false negative and false positive rates.

[0094] By adopting the above technical solution, on the one hand, the required set of constraints is introduced to construct an iterative overlap correction mechanism, and abnormal candidate defects are eliminated through set intersection operation; on the other hand, by designing a confidence accumulation function, the support coefficients of each constraint are dynamically weighted according to the defect category, realizing the soft fusion of confidence scores. This solves the problems of traditional pure model-driven methods being susceptible to noise interference, having a high false alarm rate, and lacking interpretability. It reflects a refined defect verification operation, improves the reliability of discrimination through multi-dimensional fusion, and further ensures the effectiveness of subsequent sorting and discrimination.

[0095] S4. Based on the operation results of the post-processing enhancement unit, trigger the hierarchical response mechanism to divide the target defects into different levels. Extract the comparison results between the target level proportion and the preset standard threshold within the preset period as the condition for whether to start the feedback strategy. When the target level proportion exceeds the preset standard threshold, the feedback strategy is started and the optimization process is completed through dual-path feedback.

[0096] The operation process of the tiered response mechanism is as follows:

[0097] When pf_final k If the value is ≥0.8Qt, it is judged as a Level 1 defect, which is a high-confidence real defect;

[0098] When 0.5Qt≤pf_final k If the value is less than 0.8Qt, it is classified as a Level 2 defect, which is a medium-confidence suspected defect.

[0099] When pf_final k If the value is less than 0.5Qt, it is classified as Level 3, indicating a tendency for low-confidence false alarms.

[0100] Where Qt represents a set value, the range of which is greater than 0, and in this embodiment, Qt is usually set to 1;

[0101] The comparison results between the target level percentage extracted within the preset period and the preset standard threshold are based on the following:

[0102] The target level is level two. Since level two has suspicious cases, feasible optimizations are needed, which constitute the subsequent feedback strategy. The total number of target defect judgments within the preset period is obtained, and the total number of target defect judgments within the preset period is divided by the number of times the target defect is judged as level two to obtain the target level proportion. If the target level proportion exceeds the preset standard threshold, the judgment system may be in an unstable state of identification, so the feedback strategy needs to be activated; otherwise, it reflects that the system is operating normally, so the feedback strategy does not need to be activated.

[0103] The process of initiating a feedback strategy is as follows:

[0104] Path 1: Feedback to the image preprocessing stage, corresponding to S2:

[0105] Since the proportion of secondary images is high, it may mean that the image quality is unstable, resulting in blurred features and low contrast, making it difficult for the model to recognize with high confidence. Therefore, specific optimization actions should include at least: enhanced light removal processing.

[0106] The enhanced light removal process includes:

[0107] Polarization imaging mode is activated on the target workpiece to acquire multi-angle polarized light images, which are then synthesized into a depolarized image.

[0108] ;

[0109] In the formula, I_depol represents the depolarized image, i.e., the polarization map, which suppresses specular reflection and highlights the surface morphology; I0, I 45 I 90 and I 135 The image grayscale values ​​are at polarization angles of 0°, 45°, 90°, and 135°, respectively.

[0110] Path 2: Feedback to the model training stage, corresponding to S3:

[0111] The high proportion of level 2 defects indicates that the model has not fully learned the current defect features and the discrimination boundary is unclear. It should be noted that the deep learning-based defect identification and classification model constructed in S2 above requires model training by default. Data augmentation strategies can be implemented during the model training stage, including random rotation ±15°, affine transformation, color dithering, adding simulated noise and occlusion blocks, etc., to improve generalization ability, etc., which will not be elaborated here. Therefore, the specific optimization actions should at least include: constructing an incremental training set and initiating incremental fine-tuning.

[0112] The incremental training set is constructed as follows:

[0113] Extract all samples corresponding to the target level 2 within the preset period as the hard sample set, send them to the labeling queue, and re-label the defect location and category. After labeling, add them to the training data pool.

[0114] The incremental fine-tuning action is initiated as follows:

[0115] Lightweight training of the DefectNet-V3 model using newly added training data pool includes at least: freezing the backbone network, training only the detector head, and increasing the number of training epochs by a preset increment value; where the increment value is usually set to 1 or 2.

[0116] By adopting the above technical solution, on the one hand, it constructs a periodic feedback mechanism based on the statistical analysis of the proportion of secondary defects and the analysis of verification deviations, thereby achieving dual-path optimization of image preprocessing and model training. On the other hand, it sends suspicious defects into the verification channel through a hierarchical response mechanism, forming a learning process of continuous optimization feedback. This can solve the technical problems related to the inability of traditional fixed parameter systems to adapt to dynamic changes in the production line such as mold wear and material changes, and realize the adaptive evolution capability of the solution operation itself, ensuring that the defect identification action can be continuously optimized and improved.

[0117] S5. After the defect identification result is determined, the robotic arm in the execution end of the scheduling and sorting system completes the sorting action.

[0118] In this process, if a defect is identified as Level 1, it is sorted to the first channel by a robotic arm; if it is identified as Level 2, it is sorted to the second channel by a robotic arm; if it is identified as Level 3, no response is made. The first channel is the rework / scrap channel, and the second channel is the verification channel. No response indicates a false alarm, and the target workpiece can be considered normal and continue to be transported through the conveyor system. If an abnormality is found after verification, it is sent to the first channel; otherwise, it is sent back to the conveyor system.

[0119] For the robotic arm, highly dynamic response grasping path planning and force control operations can be configured to ensure safe, efficient, and damage-free completion of the task. For example, the PLC controller on the robotic arm receives defect level and classification instructions, and calculates the spatial transformation matrix of the robotic arm base relative to the target point by combining the coordinates of the current target workpiece on the conveying system. Then, it calls the MoveIt! motion planning framework in ROS to generate the optimal joint trajectory and avoid surrounding obstacles, such as grippers and sensor supports. This will not be elaborated further. At the same time, in order to grasp target workpieces of different shapes and weights, the end effector of the robotic arm can also be equipped with an adaptive flexible gripper with built-in miniature pressure sensors and Hall elements to monitor the gripping force and opening angle in real time. The robotic arm's built-in system can also preset a variety of grasping templates corresponding to common workpiece types. Each template includes the optimal contact point distribution and clamping force threshold. For example, for thin-walled and easily deformable ventilation pipes, a low torque (≤3N·m) and multi-point support mode is used; while for heavy battery tray frames, a high-strength two-finger gripping combined with a bottom lifting auxiliary device is used.

[0120] Example 2:

[0121] Based on Example 1, this embodiment also provides a robot-assisted automotive injection molded part defect sorting system, which includes: an initialization module: performing initialization actions on a pre-configured sorting execution system;

[0122] Preprocessing module: When the target workpiece is transported to the designated area, it acquires raw image data and triggers preprocessing actions;

[0123] Defect recognition module: Constructs a deep learning-based defect recognition and classification model, and introduces a post-processing enhancement unit to obtain defect recognition results; The operation of the post-processing enhancement unit is based on: constructing an initial recognition function model, setting multi-dimensional constraints, and triggering an iterative overlap correction mechanism for collaborative correction;

[0124] Feedback Correction Module: Based on the operation results of the post-processing enhancement unit, a graded response mechanism is triggered to divide the target defects into different grades. Within a preset period, the comparison results between the target grade proportion and the preset standard threshold are extracted. When the target grade proportion exceeds the preset standard threshold, the feedback strategy is activated to complete the optimization process through dual-path feedback.

[0125] Sorting Execution Module: After the defect identification results are determined, the robotic arm in the execution end of the sorting execution system is scheduled to complete the sorting action.

[0126] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot-assisted method for sorting defects in automotive injection molded parts, characterized in that, The method includes: Perform initialization actions on the pre-configured sorting system; When the target workpiece is transported to the designated area, the original image data is acquired and the preprocessing action is triggered; A deep learning-based defect identification and classification model is constructed, and a post-processing enhancement unit is introduced to obtain defect identification results. The operation of the post-processing enhancement unit is based on: constructing an initial identification function model, setting multi-dimensional constraints, and triggering an iterative overlap correction mechanism for collaborative correction. When constructing the initial identification function model, the preliminary detection results are set as D={(bk, ck, pk)}k=1K. Where bk=(xk, yk, Wk, Hk) are the center coordinates, width, and height of the candidate box corresponding to the k-th defect; ck∈{1, 2, ..., 12} is the category of the k-th defect, serving as the category label; pk∈(0, 1] is the confidence score; k=1, 2, ..., K, where K represents the maximum value of k; the constraint set introduced when setting multi-dimensional constraints includes at least: the first constraint, the second constraint, the third constraint, and the fourth constraint; where the first constraint adopts geometric consistency constraint, the second constraint adopts material reflection consistency constraint, the third constraint adopts size rationality constraint, and the fourth constraint adopts topological structure constraint; The process of running the iterative overlap correction mechanism is as follows: after each correction, a new set of detection results D' is generated, and the intersection operation is performed with the previous round of detection results to output the final set of detection results D_final. At the same time, the pre-built confidence accumulation function is run to score the final detection results and output the final confidence score pf_finalk of the k-th defect. Based on the operation results of the post-processing enhancement unit, a hierarchical response mechanism is triggered to classify target defects into different levels. Within a preset period, the comparison results between the proportion of target level and a preset standard threshold are extracted. When the proportion of target level exceeds the preset standard threshold, a feedback strategy is initiated, and optimization processing is completed through dual-path feedback. The hierarchical response mechanism operates as follows: when pf_finalk ≥ 0.8Qt, it is classified as Level 1, a high-confidence real defect; when 0.5Qt ≤ pf_finalk < 0.8Qt, it is classified as Level 2, a medium-confidence suspicious defect; when pf_finalk < 0.5Qt, it is classified as Level 3, a low-confidence false alarm tendency. Here, Qt represents a set value, with a range greater than 0. The comparison results between the target level percentage extracted within the preset period and the preset standard threshold are based on the following: If the target level is level 2, the total number of target defect judgments within a preset period is obtained. The total number of target defect judgments within the preset period is divided by the number of times the target defect is judged as level 2 to obtain the target level percentage. If the target level percentage exceeds the preset standard threshold, the judgment is in an unstable state; otherwise, the feedback strategy is not activated. After the defect identification results are determined, the robotic arm in the execution end of the scheduling and sorting system completes the sorting action.

2. The robot-assisted defect sorting method for automotive injection molded parts according to claim 1, characterized in that: The sorting execution system includes at least a processing end and an execution end; wherein, the processing end is used to send instructions to the execution end, and the execution end also includes: a camera array, a light source subsystem, and a conveying system; the initialization action includes at least: calibrating the intrinsic and extrinsic parameters of the camera array configured in the execution end; the robotic arm performs a zeroing action and loads a preset grasping trajectory library.

3. The robot-assisted defect sorting method for automotive injection molded parts according to claim 1, characterized in that: The original image data includes at least a combined image set consisting of a top view, a front view, and a side view of the target workpiece; The triggered preprocessing steps are as follows: denoising is performed using nonlocal mean denoising combined with wavelet transform filtering; grayscale and contrast enhancement are performed, and the pre-configured CLAHE algorithm is applied; morphological closing operations are run to fill the pseudo-fracture region; the target workpiece contour is initially extracted through Canny edge detection; and SURF-based key point matching technology is introduced simultaneously to register and fuse the original image data to generate a 3D point cloud map.

4. The robot-assisted defect sorting method for automotive injection molded parts according to claim 1, characterized in that: The operation of the defect identification and classification model is based on: In the backbone network of the defect identification and classification model, CSPDarknet53 is used as the basic structure, and SE Block is embedded in its key residual blocks to realize the channel attention mechanism. Each feature map channel is squeezed using global average pooling (GAP) to generate a 1×1×C vector zc. The stimulation process is completed on two fully connected layers, and the normalized weight vector s is output. In the feature pyramid part of the defect identification and classification model, BiFPN is used to achieve efficient cross-scale fusion. The importance of each input path is dynamically adjusted by the weight parameter wi to obtain the output feature map O.

5. The robot-assisted defect sorting method for automotive injection molded parts according to claim 1, characterized in that: The process of initiating the feedback strategy includes: Path 1: Feedback to the image preprocessing stage, the corresponding optimization actions include at least enhanced de-polarization processing: enabling polarization imaging mode for the target workpiece, acquiring multi-angle polarized light images, and synthesizing depolarized images; Path 2: Feedback to the model training stage, the corresponding optimization actions include at least: constructing an incremental training set and initiating incremental fine-tuning; wherein, constructing an incremental training set: extracting all samples corresponding to level 2 targets within a preset period as a hard sample set, sending them to the labeling queue, and re-labeling the defect location and category, and adding them to the training data pool after labeling; initiating incremental fine-tuning involves using the newly added data to the training data pool to perform lightweight training on the model.

6. The robot-assisted defect sorting method for automotive injection molded parts according to claim 1, characterized in that: If the defect is identified as Level 1, it will be sorted to the first channel by a robotic arm; if it is identified as Level 2, it will be sorted to the second channel by a robotic arm; if it is identified as Level 3, no response will be made. The first channel is the rework channel and the second channel is the review channel.

7. A robot-assisted automotive injection molded part defect sorting system, applied to the robot-assisted automotive injection molded part defect sorting method according to any one of claims 1-6, characterized in that, The system includes: Initialization module: Performs initialization actions on the pre-configured sorting system; Preprocessing module: When the target workpiece is transported to the designated area, it acquires raw image data and triggers preprocessing actions; Defect recognition module: Constructs a deep learning-based defect recognition and classification model, and introduces a post-processing enhancement unit to obtain defect recognition results; The operation of the post-processing enhancement unit is based on: constructing an initial recognition function model, setting multi-dimensional constraints, and triggering an iterative overlap correction mechanism for collaborative correction; Feedback Correction Module: Based on the operation results of the post-processing enhancement unit, a graded response mechanism is triggered to classify the target defects into different grades. Within a preset period, the comparison results between the target grade proportion and the preset standard threshold are extracted. When the target grade proportion exceeds the preset standard threshold, the feedback strategy is activated to complete the optimization process through dual-path feedback. Sorting Execution Module: After the defect identification results are determined, the robotic arm in the execution end of the sorting execution system is scheduled to complete the sorting action.

Images