Pearl particle selection method and system based on visual recognition and autonomous learning model

Abstract

The application discloses a pearl particle fine selection method based on visual identification and an autonomous learning model, and comprises the following steps: S1, by a first camera component and a second camera component arranged on the outer edge of a rotating disc, image acquisition of the top and bottom of pearl particles continuously conveyed in the rotating disc groove is carried out respectively, and multi-angle original image data of the pearl particles is obtained; S2, the original image data is subjected to denoising, illumination normalization and background segmentation processing, and morphological features, color features, texture features and surface defect features of the pearl particles are extracted; the morphological features include area, roundness, length-diameter ratio and contour curvature; the color features include RGB color space mean value, hue saturation and color difference value; the texture features include energy, contrast and correlation of a gray level co-occurrence matrix; and the surface defect features include hole number, crack length and concave-convex degree. The application further discloses a pearl particle fine selection system based on visual identification and the autonomous learning model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and automated sorting technology, and in particular to a method and system for selecting pearl particles based on visual recognition and autonomous learning models. It is applicable to the intelligent selection of irregularly shaped, discolored, stone-like, and defective pearl particles during the automated processing of pearl powder. Background Technology

[0002] Freshwater pearls are a high-quality raw material for beauty pearl powder. China is the world's largest producer of freshwater pearls, accounting for more than 95% of global production. In the pearl powder processing, pearl particles must be strictly screened before grinding to remove unqualified products such as irregularly shaped particles, discolored particles, defective particles, conjoined particles, and pearl-like stones, in order to ensure the purity and consistency of the pearl powder, while protecting the fine grinding equipment from damage by irregularly shaped particles.

[0003] Existing pearl particle selection processes have low levels of automation, sometimes requiring manual re-inspection, resulting in overall low efficiency. Therefore, there is an urgent need for a pearl particle selection method and system that can achieve high-precision visual recognition, support autonomous model updating and learning, and possess multi-angle detection capabilities. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for selecting pearl particles based on visual recognition and autonomous learning models, so as to solve the problems of low recognition accuracy, inability of the model to adaptively update, and single detection perspective in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for selecting pearl particles based on visual recognition and autonomous learning models includes the following steps:

[0007] S1. Using a first and a second camera component positioned along the outer edge of the turntable, images of the top and bottom of the pearl particles continuously conveyed in the turntable's grooves are acquired, obtaining multi-angle raw image data of the pearl particles. The turntable is made of a transparent material (such as highly transparent acrylic glass or tempered glass), with grooves evenly distributed around its outer edge for single-row, orderly carrying and conveying of pearl particles. The transparent turntable allows the second camera component to clearly image the bottom of the pearl particles from bottom to top, eliminating blind spots in single-view detection.

[0008] S2. The original image data is denoised (using Gaussian filtering or bilateral filtering), normalized (using the Retinex algorithm to eliminate uneven illumination), and segmented (using adaptive thresholding or the GrabCut algorithm) to extract multi-dimensional features of the pearl particles. Morphological features include area, roundness, aspect ratio, and contour curvature; color features include RGB color space mean, HSV hue saturation, and color difference (ΔE) with the standard color; texture features include the energy, contrast, and correlation of the gray-level co-occurrence matrix (GLCM); surface defect features include the number of pores, crack length, and unevenness.

[0009] Furthermore, the top and bottom images are registered and fused, and an affine transformation method based on SIFT or ORB feature point matching is used to generate a panoramic stitched image of the pearl particles, eliminating the detection blind spot of a single viewpoint.

[0010] S3. Input the extracted features into the pre-trained pearl quality classification model, and output the quality category identifier of the pearl particles. The quality categories include six main types: good quality particles, irregularly shaped particles, discolored particles, defective particles, conjoined particles, and pearl-like stones. The pearl quality classification model adopts a multi-task convolutional neural network (Multi-Task CNN) structure, including a shared feature extraction layer and multiple parallel classification branches. The shared feature extraction layer consists of 5-8 convolutional blocks, each containing a convolutional layer, a batch normalization layer, a ReLU activation function layer, and a max-pooling layer. The parallel classification branches include a morphology classification branch, a color classification branch, a defect classification branch, and a comprehensive quality classification branch. Each branch independently outputs its corresponding classification probability, and the comprehensive quality classification branch merges the results from all branches to output the final quality category. The model also includes an attention mechanism module, including a cascaded channel attention submodule (SE-Net or ECA-Net) and a spatial attention submodule (CBAM-Spatial), enhancing the model's ability to perceive minute defects and subtle color differences on the pearl surface.

[0011] Based on the quality category identifier, the controller calculates the transmission delay of the pearl particles from the camera component to the corresponding discharge mechanism. At the precise moment, it drives the control valve of the discharge mechanism at the corresponding station to open, using compressed air to blow the pearl particles into the corresponding receiving hopper, completing the sorting process. The discharge mechanism uses an air blowing method, which can also clean the turntable groove with air, reducing the accumulation of foreign objects.

[0012] S5 includes the following sub-steps:

[0013] S51. Evaluate the confidence level of the classification probabilities output by the pearl quality classification model in step S3. When the highest classification probability is lower than the preset confidence threshold, the pearl particle is marked as an uncertain sample. The confidence threshold is set differently according to the quality category: 0.92-0.96 for good quality particles, 0.88-0.93 for irregularly shaped particles, 0.85-0.90 for discolored particles, 0.80-0.88 for defective particles, 0.90-0.95 for conjoined particles, and 0.85-0.92 for pearl-like stones. Lower thresholds are set for high-risk categories (such as defective particles and pearl-like stones) to ensure that more suspicious samples are subject to manual review.

[0014] S52 pushes multi-angle images of uncertain samples to the manual review interface, where quality inspectors manually label the quality categories, generating high-quality labeled data. The manual review interface simultaneously displays the model's prediction results and confidence levels, assisting quality inspectors in making quick decisions.

[0015] Before incremental training, S53 performs data augmentation on the labeled data to optimize for the characteristics of pearl particles, including random rotation (0-360°), horizontal flipping, brightness jitter (±15%), contrast adjustment (0.8-1.2 times), and Gaussian noise addition (σ=0.01-0.05), to expand the diversity of training samples and enhance the model's generalization ability.

[0016] The S54 annotation data was added to the training dataset, and the Elastic Weight Consolidation (EWC) method based on the Fisher information matrix was used to fine-tune and update the pearl quality classification model online. The EWC method evaluates the importance of each parameter to the learned task by calculating the Fisher information matrix, adds a regularization term to the loss function to constrain the variation of key parameters, and prevents catastrophic forgetting while learning new samples.

[0017] S55 evaluates the performance of the incrementally trained model on a complete validation set containing historical and new samples. When the classification accuracy is ≥98%, the recall of each class is ≥95%, and the F1 score is ≥96%, the updated model is automatically deployed to replace the current online model using a blue-green deployment strategy: the new model first performs synchronous inference in shadow mode, and then switches to online mode after confirming that the output difference rate is <1%, thus achieving seamless hot updates of the model.

[0018] S6 provides real-time statistics on the quantity and percentage of pearl particles in each quality category, generating a quality distribution report. It calculates the moving average and standard deviation of the percentage for each quality category within a preset time window. When the percentage of any quality category deviates from the historical average by more than three times the standard deviation, an anomaly warning signal is triggered, indicating potential fluctuations in feed quality or equipment malfunction, facilitating timely intervention by operators.

[0019] A pearl particle selection system based on visual recognition and autonomous learning model includes a mechanical execution module, a visual acquisition module, a computational control module, and a human-computer interaction module.

[0020] The mechanical execution module includes a turntable, a feeding mechanism, a first discharging mechanism, and a second discharging mechanism. The turntable is a horizontally arranged transparent disc with a central shaft connected to a motor via a reducer. A V-shaped groove along the outer edge facilitates continuous, single-row conveying of pearl particles. The groove has an inverted trapezoidal cross-section, with a bottom width of 5-12 mm and a stepped structure (3-7 mm high) on the outer side to ensure the stability of the pearl particles during high-speed rotation. The feeding mechanism includes a vibrating disc and a conveying trough, which orderly feeds the pearl particles into the groove in a single row. The first and second discharging mechanisms each include an air inlet inside the groove, a control valve, and a receiving hopper outside the groove, respectively. Compressed air drives the pearl particles into the corresponding receiving hopper.

[0021] The visual acquisition module includes a first camera component, a second camera component, an illumination component, and an infrared sensor. The first camera component takes top-down images of the pearl particles in the groove, while the second camera component takes bottom-up images. Both cameras are vertically aligned with the center line of the groove and are spaced approximately 10 degrees apart. The illumination component is a ring-shaped LED diffused light source with a color temperature of 5000-6500K and a color rendering index ≥90, providing stable diffused backlighting. The infrared sensor is positioned between the feeding mechanism and the first camera component to detect the pearl particle conveying interval. When there are no pearl particles in the groove, it sends a skip signal to reduce invalid acquisition and calculation. The first and second camera components are industrial area scan cameras with a resolution ≥5 megapixels and a frame rate ≥30fps.

[0022] The computational control module includes a controller and a server. The server runs a multi-task CNN pearl quality classification model and a model self-updating learning module. The controller is responsible for: receiving infrared sensor signals to determine the arrival of pearls; triggering the camera component to acquire images; transmitting image data to the server; receiving the quality category identifier output by the server; calculating the transmission delay; and driving the corresponding discharging mechanism to perform sorting. The turntable diameter is 60-120 cm, and the groove linear velocity is 0.5-1.2 m / s. Based on 1 gram of pearls, it can process 1.8-3 kg per minute, exhibiting excellent sorting efficiency.

[0023] The model autonomous update learning module includes a confidence evaluation unit, a labeled data management unit, an incremental training unit, and a model deployment unit. The confidence evaluation unit evaluates the classification probabilities output by the model in real time, marking samples below the confidence threshold as uncertain samples; it also includes a distribution drift detection submodule, which uses a sliding window approach to monitor the feature distribution of online inference data, triggering model retraining when the KL divergence exceeds a preset threshold. The labeled data management unit stores and manages manually labeled data, maintaining version management for the training and validation sets. The incremental training unit uses the EWC method for online fine-tuning. The model deployment unit evaluates the performance of the trained model and automatically replaces the online model once the performance meets the required metrics.

[0024] The human-computer interaction module includes an interface for verifying uncertain samples and an interface for quality monitoring. The verification interface displays multi-angle images of low-confidence samples, model prediction results, and confidence levels, and receives manual annotations. The monitoring interface displays real-time quality distribution trend charts, the proportion of each quality category, sorting speed, and anomaly warning information.

[0025] Compared with existing technologies, this solution has the following advantages:

[0026] This solution employs a multi-task CNN combined with an attention mechanism to comprehensively evaluate pearl particles from four dimensions: morphology, color, texture, and surface defects, achieving significantly higher recognition accuracy than traditional methods. Multi-angle image acquisition eliminates blind spots from single-viewpoint detection, while panoramic stitching technology provides complete particle surface information.

[0027] This solution filters uncertain samples through confidence assessment, and after manual annotation, uses the EWC incremental learning method to update the model online. It can adapt to new defect patterns and raw material changes without downtime for retraining, achieving continuous model evolution. The distribution drift detection mechanism can proactively trigger model retraining when raw material quality changes.

[0028] Different confidence thresholds are set for different quality categories, and stricter screening criteria are used for high-risk categories, balancing the false positive rate and false negative rate to ensure a high recall rate for key categories (such as pearl-like stones and defective particles).

[0029] This solution combines the intelligent triggering mechanism of infrared sensors to reduce unnecessary computational load; the transparent turntable enables simultaneous acquisition from both perspectives; and the air-blowing sorting achieves millisecond-level response, with an overall processing speed of 1.8-3 kg / min.

[0030] Real-time quality statistics and anomaly early warning mechanisms based on statistical process control (SPC) can promptly detect fluctuations in feed quality and equipment malfunctions, ensuring the stability of the production process and the consistency of product quality. Attached Figure Description

[0031] Figure 1 This is a photograph of a typical pearl grain awaiting processing.

[0032] Figure 2 This is a schematic diagram of the overall structure of a preferred embodiment.

[0033] Figure 3 This is a schematic diagram of the internal layout of the turntable.

[0034] Figure 4 This is a schematic diagram of the workstation layout for the turntable.

[0035] Figure 5 This is a top view of the turntable's structure.

[0036] Figure 6 This is a schematic diagram of the groove arrangement structure of the turntable.

[0037] Figure 7 This is a diagram of the actual imaging processing interface.

[0038] Figure 8 To stitch together a panoramic image of pearl particles. Detailed Implementation

[0039] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0040] refer to Figure 1 and Figure 8 This embodiment provides a pearl particle selection system based on visual recognition and autonomous learning model, including a mechanical execution module, a visual acquisition module, a computational control module, and a human-computer interaction module.

[0041] The basic structure consists of a refining device installed at the front end of the grinding process in the automated pearl powder processing production line. It is used to refine the pearl particles to be ground, removing irregularly shaped, discolored, lumpy, and defective particles. A typical example is... Figure 1 As shown, A and B are regular, qualified pearl particles; C is pearl particles with discoloration or pores; D is pearl particles with pores; E is conjoined pearl particles (these particles can easily damage grinding tools during fine grinding); and F is a stone resembling a pearl particle. This process ensures that high-quality pearl particles are used for subsequent grinding operations, preventing defective pearls from being ground and guaranteeing the consistency and purity of the finished pearl powder.

[0042] This selected device is a typical center turntable automated multi-station processing equipment, which includes a turntable 1 and a first camera component 11 arranged on the outer edge of the turntable. The turntable 1 is arranged horizontally, with a rotating shaft arranged at the center. The rotating shaft is connected to a rotating shaft motor through a reducer. Under the control of the controller, the rotating shaft motor can drive the rotating shaft and the turntable to rotate at a uniform speed.

[0043] The outer edge of the turntable 1 is provided with a V-shaped groove 10. The groove 10 is used for continuous conveying of pearl particles. In order to ensure that the pearl particles can be conveyed smoothly and efficiently and to adapt to high-speed visual recognition operation, the main body of the turntable 1 is made of a highly transparent material, such as glass or highly transparent organic glass, so that the pearl particles can be visually imaged and processed from multiple angles.

[0044] To ensure that the pearl particles are stably arranged in the groove 10 and to prevent them from being thrown out due to centrifugal force or shaking during transport when the turntable 1 rotates at high speed (which would be detrimental to visual recognition), in some embodiments, the cross-section of the groove 10 is an inverted trapezoidal shape, the bottom width of the groove 10 is 5-12 mm, and a stepped structure is provided on the outside of the groove 10. The height of the stepped structure relative to the bottom of the groove is 3-7 mm, thereby ensuring that the pearl particles are transported stably and efficiently at the turntable 1.

[0045] In actual setup, the diameter of the turntable is 60-120 cm, and the linear velocity at groove 10 of turntable 1 is 0.5-1.2 m / s. Based on a pearl particle weight of 1 gram, this turntable can process 1.8-3 kg of pearl particles per minute, which has excellent efficiency and is much higher than traditional manual sorting operations.

[0046] To improve the accuracy and comprehensiveness of identification, a first camera component 11 and a second camera component 12 are arranged on the outer edge of the turntable 1. To reduce interference between the two, the first camera component 11 and the second camera component 12 are spaced a certain distance apart, for example, 10 degrees apart from the center of the turntable. The main structures of the first camera component 11 and the second camera component 12 are identical. The first camera component 11 takes pictures of the pearl particles in the groove 10 from top to bottom, and the second camera component 12 takes pictures of the pearl particles in the groove 10 from bottom to top. The center lines of the two camera components are vertically aligned with the center line of the groove. The two camera components are connected to a controller and a server, which can identify the quality category of the pearl particles after the pictures are taken.

[0047] The turntable 1 is provided with a feeding mechanism 13 at the front end of the first camera component 11, and the turntable 1 is provided with a first discharging mechanism 14 and a second discharging mechanism 15 at the rear end of the second camera component 12.

[0048] Specifically, a typical pearl particle feeding mechanism 13 can be a vibratory feeder and a conveying trough structure. The pearl particles to be selected are placed at the vibratory feeder. Through the operation of the vibratory feeder, the pearl particles can be conveyed in a straight line and in an orderly manner at the conveying trough, so that the conveying is coordinated with the rotation of the turntable 1, and the pearl particles are arranged in an orderly and non-overlapping manner at the groove 10.

[0049] The first discharge mechanism 14 and the second discharge mechanism 15 are used to discharge pearl particles identified as defective and good quality, respectively, and to drive the pearl particles away from the groove. Both have similar structures and use an air-blowing method to drive the output of the current pearl particles. Specifically, it includes an air-blowing port arranged inside the groove and a receiving hopper arranged outside the groove. The air-blowing port is connected to a control valve, a compressed air pipeline, and an air source. The control valve is connected to a controller, enabling precise air-blowing to drive the identified pearl particles to the corresponding receiving hopper.

[0050] In some embodiments, to improve the efficiency of the first camera component 11 and the second camera component 12, an infrared sensor 16 is arranged between the feeding mechanism 13 and the first camera component. The infrared sensor 16 includes an infrared transmitter and a receiver. The transmitter emits infrared light in a straight line towards the receiver, with the infrared light oriented in the direction of the turntable's radius and positioned above the turntable. The receiver converts the infrared signal into a digital signal and transmits the data to the controller. The purpose of the infrared sensor 16 is to identify the gaps in the pearl particle transport, reduce the disordered operation of the first camera component 11 and the second camera component 12, reduce the imaging processing of the gaps in the pearl particles, thereby reducing the system's computational load and improving system efficiency.

[0051] To ensure the stability of the turntable and the selection operation, the turntable and related components are arranged inside the detection box. Illumination components are arranged in both the first and second camera components to provide stable imaging background light, reduce the influence of ambient light, and improve equipment efficiency.

[0052] As a visual acquisition module, the first camera takes top-down images of the pearl particles in the groove, while the second camera takes bottom-up images. Both cameras are vertically aligned with the center line of the groove and are spaced approximately 10 degrees apart. The lighting is a ring-shaped LED diffused light source with a color temperature of 5000-6500K and a color rendering index ≥90, providing stable diffused backlighting. An infrared sensor is positioned between the feeding mechanism and the first camera to detect the pearl particle conveying interval. When there are no pearl particles in the groove, a skip signal is sent to reduce invalid acquisition and calculation. Both the first and second cameras are industrial area scan cameras with a resolution ≥5 megapixels and a frame rate ≥30fps.

[0053] The computational control module includes a controller and a server. The server runs a multi-task CNN pearl quality classification model and a model self-updating learning module. The controller is responsible for: receiving infrared sensor signals to determine the position of the pearls; triggering the camera component to acquire images; transmitting the image data to the server; receiving the quality category identifier output by the server; calculating the transmission delay; and driving the corresponding discharging mechanism to perform sorting.

[0054] The model autonomous update learning module includes a confidence evaluation unit, a labeled data management unit, an incremental training unit, and a model deployment unit. The confidence evaluation unit evaluates the classification probabilities output by the model in real time, marking samples below the confidence threshold as uncertain samples; it also includes a distribution drift detection submodule, which uses a sliding window approach to monitor the feature distribution of online inference data, triggering model retraining when the KL divergence exceeds a preset threshold. The labeled data management unit stores and manages manually labeled data, maintaining version management for the training and validation sets. The incremental training unit uses the EWC method for online fine-tuning. The model deployment unit evaluates the performance of the trained model and automatically replaces the online model once the performance meets the required metrics.

[0055] The human-computer interaction module includes an interface for verifying uncertain samples and an interface for quality monitoring. The verification interface displays multi-angle images of low-confidence samples, model prediction results, and confidence levels, and receives manual annotations. The monitoring interface displays real-time quality distribution trend charts, the proportion of each quality category, sorting speed, and anomaly warning information.

[0056] This solution presents a pearl particle selection method based on visual recognition and autonomous learning models, which is applied to the aforementioned selection system.

[0057] Specifically, when the infrared sensor 16 detects that a pearl particle has arrived at the collection station, it sends a trigger signal to the controller. The controller simultaneously triggers the first camera component 11 and the second camera component 12 to perform synchronous acquisition. The first camera component 11 acquires an image of the top of the pearl particle, and the second camera component 12 acquires an image of the bottom of the pearl particle through a transparent turntable.

[0058] After receiving the image data, the server performs the following preprocessing steps:

[0059] (1) Noise removal: Gaussian filtering with σ=1.0 is used to remove sensor noise from the original image;

[0060] (2) Illumination normalization: The multi-scale Retinex algorithm is used to eliminate the influence of uneven illumination and enhance the contrast and details of the image;

[0061] (3) Background segmentation: Adaptive threshold segmentation based on color space is used to separate the pearl particles from the groove background and generate a binary mask;

[0062] (4) Feature extraction: Based on the binary mask, extract morphological features (area S, roundness C=4πS / P², aspect ratio AR, contour curvature κ); calculate color features (RGB mean, HSV value, ΔE color difference with standard good color) within the mask area; calculate gray-level co-occurrence matrix to extract texture features (energy, contrast, correlation, homogeneity); detect surface defect features (calculate the number of holes through connected component analysis, calculate the crack length through edge detection, and calculate the concavity and convexity through surface gradient).

[0063] (5) Panoramic stitching: ORB feature point matching is used on the top and bottom images to calculate the affine transformation matrix, transform the bottom image to the coordinate system of the top image, and stitch them together to generate a panoramic image of the pearl particles, providing complete surface information.

[0064] Pearl quality classification model:

[0065] The pearl quality classification model employs a multi-task convolutional neural network structure, with input being a 224×224×3 RGB image (top or panoramic image), as shown below:

[0066] The shared feature extraction layer contains 6 convolutional blocks:

[0067] Block 1: Conv(3→32,3×3)+BN+ReLU+MaxPool(2×2);

[0068] Block 2: Conv(32→64,3×3)+BN+ReLU+MaxPool(2×2);

[0069] Block 3: Conv(64→128,3×3)+BN+ReLU+MaxPool(2×2);

[0070] Block 4: Conv(128→256,3×3)+BN+ReLU+MaxPool(2×2);

[0071] Block 5: Conv(256→512,3×3)+BN+ReLU+MaxPool(2×2);

[0072] Block 6: Conv(512→512,3×3)+BN+ReLU+AdaptiveAvgPool(7×7).

[0073] Attention mechanism modules are connected in series:

[0074] Channel attention submodule: Global average pooling → FC (512 → 32) → ReLU → FC (32 → 512) → Sigmoid → Channel weighting;

[0075] Spatial attention submodule: Channel dimension MaxPool+AvgPool→Concat→Conv(2→1,7×7)→Sigmoid→Spatial weighting;

[0076] Parallel classification branches (each branch structure: FC(512→256)+ReLU+Dropout(0.5)+FC(256→N)+Softmax):

[0077] Morphological classification branches: N=4 (normal shape, irregular shape, conjoined, stone-like);

[0078] Color classification branches: N=3 (normal color, heterochromatic color, severe heterochromatic color);

[0079] Defect classification branches: N=4 (no defects, minor defects, major defects, and void defects);

[0080] Comprehensive quality classification branches: N=6 (good, irregular, discolored, defective, conjoined, stone), input the logits features of the first three branches.

[0081] The model training uses a multi-branch weighted cross-entropy loss function:

[0082] ;

[0083] Where w1=w2=w3=0.2, w4=0.4.

[0084] The initial training used 10,000 balanced labeled images, employed the Adam optimizer, and had an initial learning rate of 1×10⁻⁶. -4 Batch size 32, training for 100 epochs; incremental learning using a learning rate of 5×10⁻⁶. −5 The batch size is 16, the training lasts for 20 epochs, and an early stopping strategy (patience=5) is used to prevent overfitting. The regularization coefficient λ=5000.

[0085] The autonomous learning and updating of the model in this scheme is specifically as follows:

[0086] During online inference, the model outputs a probability distribution P=[p1,p2, ...,p6] for each pearl particle across six categories for confidence assessment and uncertain sample screening. The confidence assessment unit performs the following judgments:

[0087] (1) Calculate the highest probability p max = max(P);

[0088] (2) Query the corresponding threshold θ based on the predicted category:

[0089] Good product: θ=0.94;

[0090] Irregular shape: θ = 0.90;

[0091] Different colors: θ = 0.88;

[0092] Defect: θ = 0.85;

[0093] Conjoined bodies: θ = 0.92;

[0094] Stone: θ = 0.88;

[0095] (3) If p max < θ, marked as an uncertain sample, along with multi-angle images, model prediction results and the probabilities of each classification, are pushed to the manual review interface;

[0096] (4) Uncertain samples are synchronously stored in the database to be labeled, retaining the original image, feature vector and timestamp.

[0097] 4.2 Distribution Drift Detection

[0098] The distribution drift detection submodule maintains a sliding window (window size N=1000 samples), and calculates the KL divergence between the online data feature distribution and the training data feature distribution every N samples:

[0099] ;

[0100] Where: D KL KL divergence measures the degree of difference between the feature distribution of online inference data and the feature distribution of model training data.

[0101] P train (i): The probability density of the i-th feature interval in the training set;

[0102] P online (i): The probability density of the i-th feature interval within the online inference sliding window;

[0103] N: Total number of characteristic intervals;

[0104] When D KL When the value is ≥δ (preset threshold δ=0.15), it is determined to be a feature distribution drift, triggering incremental retraining of the model. A retraining trigger signal is sent to the incremental training unit, indicating that the model may need to adapt to the new data distribution.

[0105] 4.3 Manual annotation and data management

[0106] The manual review interface displays the top, bottom, and panoramic images of the pearl particles side-by-side, along with the model's predicted categories and probability bar charts for each category. Quality inspectors complete annotations by clicking the corresponding category button, and the system simultaneously records the annotator's ID, annotation time, and annotation confidence level (quality inspectors can annotate as "OK" or "Requires Review"). The annotation data management unit maintains data versions by date and category, supporting data backtracking and auditing.

[0107] 4.4 Incremental Training

[0108] When the accumulated labeled data reaches 500 records or a distribution drift trigger signal is received, the incremental training unit initiates the online fine-tuning process:

[0109] (1) Obtain the latest labeled data from the labeled data management unit and divide it into incremental training set and incremental validation set in an 8:2 ratio;

[0110] (2) Perform data augmentation on the incremental training set (random rotation 0-360°, horizontal flip, brightness ±15% jitter, contrast adjustment 0.8-1.2 times, Gaussian noise σ=0.03), and generate 5 augmented images for each original image;

[0111] (3) Incremental training is performed using the EWC method:

[0112] Calculate the Fisher information matrix F i Evaluate each parameter θ i The importance of the learned task; the calculation formula is:

[0113] ;

[0114] Wherein: F i For parameter θ i The corresponding Fisher information; E is the mathematical expectation based on the training set distribution; logP(y|x;θ) is the log-likelihood function of the model with respect to the training samples; θ i Let i be the i-th parameter of the model.

[0115] The loss function is:

[0116] ;

[0117] Where L new The cross-entropy loss is for the new data, λ=5000 is the regularization coefficient, and θ i The parameter values ​​before the update;

[0118] The Adam optimizer is used with a learning rate of 5×10⁻⁶. -5 (Lower than the initial training), batch size 16, training for 20 epochs;

[0119] Use an early stopping strategy to prevent overfitting.

[0120] 4.5 Model Validation and Hot Deployment

[0121] After incremental training is completed, the model deployment unit performs performance evaluation on the full validation set (including the historical validation set and the newly added validation set):

[0122] Classification accuracy ≥ 98%; recall rate for each category ≥ 95%; precision rate for each category ≥ 93%; F1 score ≥ 96%.

[0123] If all the above indicators are met, execute the blue-green deployment:

[0124] (1) Load the new model onto the shadow server;

[0125] (2) Run in shadow mode for 10 minutes to perform synchronous inference on online traffic without affecting actual sorting;

[0126] (3) Compare the output consistency between the shadow model and the online model. If the difference rate is <1%, perform a switch.

[0127] (4) Upgrade the new model to an online model, downgrade the online model to a backup, keep it for 72 hours and then clean it up.

[0128] If the validation metrics do not meet the requirements, discard this update, retain the current online model, and record the failure log for subsequent analysis.

[0129] Regarding online quality monitoring and anomaly alerts, the specifics are as follows:

[0130] The quality monitoring interface displays the following information in real time:

[0131] (1) Real-time quantity and percentage of each quality category;

[0132] (2) Quality trend line chart for the most recent hour;

[0133] (3) Sorting speed (particles / minute) and throughput (kg / hour);

[0134] (4) Histogram of model confidence distribution;

[0135] (5) List of abnormal warnings.

[0136] The anomaly warning system employs the Statistical Process Control (SPC) method: using a 1-hour time window, it calculates the moving average μ and standard deviation σ of the proportion of each quality category. When the proportion of a certain category exceeds the control limit of μ ± 3σ (e.g., a sudden increase in the defect rate), the system automatically issues an audible and visual warning and displays an anomaly details pop-up window on the interface, prompting operators to check the quality of incoming materials or the status of the equipment.

[0137] Thus, this solution presents a method and system for selecting pearl particles based on visual recognition and autonomous learning models. This method can accurately perform the selection of pearl particles and, through autonomous learning, significantly improve the accuracy of the operation.

Claims

1. A method for selecting pearl particles based on visual recognition and autonomous learning models, characterized in that: Includes the following steps: S1 uses a first camera component and a second camera component located on the outer edge of the turntable to acquire images of the top and bottom of the pearl particles continuously conveyed in the groove of the turntable, thereby obtaining multi-angle raw image data of the pearl particles. S2 performs denoising, illumination normalization, and background segmentation on the original image data, extracting the morphological features, color features, texture features, and surface defect features of the pearl particles; the morphological features include area, roundness, aspect ratio, and contour curvature; the color features include the RGB color space mean, hue saturation, and color difference value; the texture features include the energy, contrast, and correlation of the gray-level co-occurrence matrix; The surface defect characteristics include the number of holes, crack length, and unevenness; S3 inputs the extracted features into the pre-trained pearl quality classification model and outputs the quality category identifier of the pearl particles; the quality categories include good quality particles, irregularly shaped particles, discolored particles, defective particles, conjoined particles, and pearl-like stones. Based on the quality category identification, S4 drives the discharge mechanism of the corresponding station through the controller to blow pearl particles identified as different quality categories into the corresponding receiving hopper, completing the fine sorting.

2. The method according to claim 1, characterized in that: The pearl quality classification model in step S3 adopts a pearl-specific multi-task convolutional neural network structure, including a pearl-specific shared feature extraction layer and multiple parallel classification branches corresponding to the physical properties of pearls. The shared feature extraction layer consists of multiple convolutional layers, batch normalization layers, and activation function layers. The parallel classification branches include morphology classification branches, color classification branches, defect classification branches, and comprehensive quality classification branches. Each branch independently outputs its corresponding classification probability, and the comprehensive quality classification branch merges the results of each branch to output the final quality category.

3. The preferred method according to claim 2, characterized in that: The pearl quality classification model also includes an attention mechanism module for enhancing the perception of minor defects and weak color differences in pearls. The attention mechanism module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule performs adaptive weighting of the feature map output by the shared feature extraction layer according to the channel dimension. The spatial attention submodule performs adaptive weighting of the feature map according to the spatial dimension. The two are connected in series, with channel weighting performed first and then spatial weighting performed, which focuses on enhancing the model's ability to identify minor cracks, pinholes and weak color differences on the pearl surface.

4. The selection method according to claim 1, characterized in that: The method further includes step S5, which comprises the following sub-steps: S51 evaluates the confidence level of the classification probability output by the pearl quality classification model in step S3. When the highest classification probability is lower than the preset confidence threshold, the pearl particle is marked as an uncertain sample. S52 pushes the multi-angle images of the uncertain sample to the manual review interface, where quality inspectors manually annotate the quality category and generate annotation data; S53 adds the labeled data to the training dataset and uses an elastic weight consolidation incremental learning strategy based on the Fisher information matrix to fine-tune and update the pearl quality classification model online. The elastic weight consolidation method evaluates the importance of model parameters by calculating the Fisher information matrix and adds a regularization term to the loss function to constrain the change range of key parameters. While learning new samples, it retains the pearl classification knowledge that has been learned and prevents catastrophic forgetting. S54 evaluates the performance of the incrementally trained model on the validation set. When the classification accuracy, recall, and F1 score all reach the preset thresholds, it automatically replaces the current online model with the updated model using a blue-green deployment strategy.

5. The selection method according to claim 4, characterized in that: The confidence threshold in step S51 is set differently according to the risk level of the pearl particle quality category: the confidence threshold for good quality particles is 0.92-0.96, the confidence threshold for irregularly shaped particles is 0.88-0.93, the confidence threshold for discolored particles is 0.85-0.90, the confidence threshold for defective particles is 0.80-0.88, the confidence threshold for conjoined particles is 0.90-0.95, and the confidence threshold for pearl-like stones is 0.85-0.92; a lower threshold is used for high-risk defect categories to improve the detection rate of suspicious samples.

6. The selection method according to claim 1, characterized in that: The method further includes step S6, which comprises the following sub-steps: S61 provides real-time statistics on the number and percentage of pearl particles in each quality category, generating a quality distribution report. S62 calculates the moving average and standard deviation of the proportion of each quality category according to a preset time window; S63 When the proportion of any quality category deviates from the historical average by more than the preset deviation multiple standard deviation, an abnormal warning signal is triggered, indicating that there may be fluctuations in the quality of the feed or equipment abnormalities.

7. A pearl particle selection system based on visual recognition and autonomous learning model, characterized in that: It includes a mechanical execution module, a vision acquisition module, a computing control module, and a human-computer interaction module; The mechanical execution module includes a turntable, a feeding mechanism, a first discharging mechanism, and a second discharging mechanism. The turntable is a horizontally arranged transparent disc with a rotating shaft motor connected to its center. A V-shaped groove is provided along its outer edge for continuous single-row conveying of pearl particles. The feeding mechanism includes a vibrating disc and a conveying trough, which orderly arranges the pearl particles and feeds them into the groove. The first and second discharging mechanisms each include an air inlet, a control valve, and a receiving hopper, respectively, using compressed air to drive the pearl particles into the corresponding receiving hopper. The visual acquisition module includes a first camera component, a second camera component, and an illumination component; the first camera component takes a top-down photograph of the pearl particles in the groove, and the second camera component takes a bottom-up photograph of the pearl particles in the groove, with the center lines of both cameras vertically aligned with the center line of the groove; the illumination component provides a stable diffused backlight source for the first and second camera components.

8. The selection system according to claim 7, characterized in that: The computing control module includes a controller and a server; the server runs a pearl-specific multi-task CNN quality classification model and a model self-updating learning module; the controller receives image data from the vision acquisition module and transmits it to the server for processing, receives the quality category identifier output by the server and drives the discharge mechanism to perform sorting; The human-computer interaction module includes an uncertain sample verification interface and a quality monitoring interface. The uncertain sample verification interface is used to display low-confidence samples and receive manual annotations. The quality monitoring interface is used to display quality distribution reports and abnormal warning information in real time.

9. The selection system according to claim 7, characterized in that: The system also includes an infrared sensor, which is arranged between the feeding mechanism and the first camera component. The infrared sensor includes an infrared transmitter and a receiver. The transmitter emits infrared light in a straight line towards the receiver. The infrared light is arranged along the radius of the turntable and is attached to the top of the turntable. The infrared sensor is used to detect the conveying interval of the pearl particles. When no pearl particles are detected in the groove, a skip signal is sent to the controller. The controller controls the camera component to pause the acquisition, reducing the unnecessary calculation load.

Images