Industrial product color feature intelligent automatic extraction method based on machine learning and detection method thereof

Abstract

The invention discloses an industrial product color feature intelligent automatic extraction method based on machine learning, and the method comprises the steps: S1, carrying out the template matching and positioning, creating an effective Mask image, and filling an interference region; s11, constructing a multi-scale contour extractor by adopting morphology in combination with an edge detection algorithm, eliminating a pseudo edge caused by uneven illumination through corrosion expansion operation, and generating a target contour template for template matching positioning and adapting to positioning of materials with different colors; s12, after the target contour is positioned, calculating a color median of a target area, uniformly filling interference areas with black, and eliminating interference of background colors on color classification; s13, creating a Mask image, setting an interference area as 0, and setting other areas as effective target areas as 1; s14, interference of the background and the defect on color classification is eliminated, and subsequent feature extraction is made to aim at the effective target area; and S15, obtaining an original image, a Mask image and a filled image.

Description

Technical Field

[0001] The present invention relates to the field of industrial optical detection systems, and in particular to a method for intelligent automatic extraction of color features of industrial products based on machine learning and a detection method thereof. Background Art

[0002] Existing technologies for industrial visual color detection mainly include visual inspection, colorimeter / spectrophotometer instrument detection, and machine vision systems based on traditional image processing (such as RGB / HSV threshold segmentation) and deep learning (such as CNN classification).

[0003] However, these techniques have significant drawbacks:

[0004] 1. Visual inspection is affected by subjective color vision differences and ambient light interference, resulting in poor repeatability;

[0005] 2. The colorimeter has large errors in detecting the transmission color difference of transparent / curved objects because its optical design does not separate the transmission and reflection light paths;

[0006] 3. Traditional algorithms are sensitive to lighting and cannot accurately capture color gradients. Deep learning relies on large amounts of labeled data, and the accuracy of special color recognition is low with small samples.

[0007] 4. In addition, the difference in refractive index across materials (resin / glass) leads to poor model versatility. Summary of the Invention

[0008] The main purpose of this invention is to propose an intelligent automatic extraction method of industrial product color features based on machine learning and a detection method thereof.

[0009] The goal is to build a highly robust (high robustness represents the ability of the system to remain stable and reliable under abnormalities or interference, which is a key indicator of system robustness) and scalable industrial intelligent color detection method.

[0010] Through the innovation of multi-spatial feature fusion and dynamic learning mechanism, the core pain points of traditional methods such as insufficient color difference sensitivity in complex lighting environments, strong dependence on feature engineering, and poor model scalability are solved, and accurate color recognition and classification across materials and multiple scenes are achieved.

[0011] It also supports non-destructive incremental category learning, significantly reducing the data annotation and model update costs of new color detection tasks, and providing a sustainably optimized automation solution for industrial quality inspection.

[0012] To achieve the above objectives, the present invention proposes a method for intelligent automatic extraction of industrial product color features based on machine learning, which is characterized by comprising:

[0013] S1: Template matching and positioning, creating a valid Mask image and filling the interference area;

[0014] S11: A multi-scale contour extractor is constructed by combining morphology with an edge detection algorithm. Erosion and dilation operations are used to eliminate pseudo edges caused by uneven illumination, generating a target contour template for template matching and positioning, and adapting to the positioning of materials of different colors.

[0015] S12: After locating the target outline, calculate the color median of the target area. If the color difference between the target area and the median exceeds the set threshold,

[0016] The interference area is considered as the interference area together with the background, and the interference area is uniformly filled with black to eliminate the interference of background color on color classification;

[0017] S13: Create a Mask image, set the interference area to 0, and the other areas as valid target areas, set to 1;

[0018] S14: Eliminate the interference of background and defects on color classification, and make subsequent feature extraction focus on the effective target area;

[0019] S15 obtains the original image, the mask image, and the filled image.

[0020] Preferably, S2: multispectral feature fusion, introducing MaskSE module, and the main network extracts deep features;

[0021] Construct a four-dimensional color space joint representation system, including:

[0022] S21, RGB space: retains the original sensor response characteristics and splits the S15 padded image into R, G, and B channel images;

[0023] S22.HLS space: extracts hue-brightness decoupling features into H, L, and S channel images;

[0024] S23.LAB space: Constructs a color perception uniformity space for L, A, and B channel images;

[0025] S24.YUV space: Separates brightness and chrominance components into Y, U, and V channel images.

[0026] Preferably, S24: introduce the MaskSE module to improve the spatial feature compression of the global pooling in the SE module into the local average pooling of the effective area of ​​the Mask.

[0027] Generate channel description vector S:

[0028] Where M is the Mask image,

[0029] F is the joint representation of color space,

[0030] N active Indicates the total number of pixels in the effective area of ​​the Mask.

[0031] Preferably, S3: Feature dimensionality reduction layer

[0032] Constructing a learnable affine transformation matrix PCA-like features are achieved through the following constraints;

[0033] S31: Orthogonal regularization term:

[0034] in,

[0035] W proj is the weight matrix of the dimensionality reduction layer, which improves the feature expression capability by optimizing the orthogonality of the column vectors of the dimensionality reduction projection matrix;

[0036] S32: Covariance constraint term:

[0037] Among them, Z is the output feature of the dimensionality reduction layer, which improves linear separability by constraining the low correlation between features;

[0038] Assign loss coefficients to each constraint item, which can be trained jointly with the feature extraction network and classification network;

[0039] Preferably, S4: dynamically expanding the classifier includes proposing parameter freezing and local fine-tuning strategies;

[0040] The local fine-tuning of parameters includes:

[0041] S41: When adding a new classification, retain some or all parameters of the old model to avoid catastrophic forgetting of the entire model parameters.

[0042] S42: Train the newly added classification parameters in the dynamically expanded classification layer to achieve the effect of adapting to the classification of new categories while the old categories can be correctly classified.

[0043] Preferably, S43: the training method includes freezing the extraction layer and the dimension reduction layer parameters,

[0044] Only the classification layer parameters are trained, and the corresponding loss weights need to be adjusted during training;

[0045] S431. New category initialization: add new category parameters of the corresponding layer, calculate the mean of the old category features of the original model to initialize and accelerate convergence;

[0046] S432. Knowledge solidification mechanism:

[0047] Add the KL divergence constraint Lx to the loss L =D KL(P old ||P new ),

[0048] Prevent old knowledge from being forgotten when adding new categories, and constrain the consistency of the output distribution of the new and old models;

[0049] S433. Small Sample Adaptation: Freeze the parameters of the feature extraction and dimensionality reduction layers and train only the classification layer parameters. This allows the classification accuracy of the old model to be approached with only a small number of new samples.

[0050] A method for intelligent automatic detection of color features of industrial products based on machine learning, comprising the above-mentioned method for intelligent automatic extraction of color features of industrial products based on machine learning,

[0051] include:

[0052] Step 1: Morphological detection of target contours

[0053] Input: images captured by RGB industrial cameras;

[0054] Morphological processing:

[0055] An opening operation is performed using a 5×5 rectangular structure element, wherein the operation is first erosion and then expansion to eliminate small noises;

[0056] Canny edge detection to extract the outer contour of parts;

[0057] The closing operation is dilation followed by erosion, connecting the broken edges to generate a closed contour template;

[0058] Output: Binarized contour template, marking the target area as 1 and the background as 0;

[0059] Step 2: Template matching and interference area filtering

[0060] Template matching: Match the contour template with the real-time image through normalized cross correlation (NCC) to locate the target area;

[0061] Color median calculation: Calculate the median of the RGB channels within the target area;

[0062] Color difference threshold setting: calculate the color difference between each pixel and the median value for each channel separately, and set the threshold to 25;

[0063] Mask generation:

[0064] Pixels with a color difference of >25 are considered as interference (such as reflections, scratches, etc.) and filled with 0;

[0065] The background area is filled with 0;

[0066] The rest of the area is the valid target area and is filled with 1;

[0067] Image filling: fill the RGB image position corresponding to the area with 0 in the Mask with 0;

[0068] Step 3: Multi-color space fusion;

[0069] Color space conversion and merging: RGB images are converted into HLS, LAB, and YUV, and 12 channels, RGB (3 channels), HLS (3 channels), LAB (3 channels), and YUV (3 channels), are merged into a 12×H×W tensor;

[0070] Step 4: Improved MaskSE module and feature extraction;

[0071] SE module improvements:

[0072] Local average pooling: Calculate the mean of each channel only for the area where Mask = 1, replacing global average pooling;

[0073] Dynamic weight allocation: Generate channel weights through two fully connected layers;

[0074] Convolutional layer structure:

[0075] 5 convolution layers: each layer uses a 3×3 kernel and ReLU activation function;

[0076] Output: 128-dimensional feature vector;

[0077] Step 5: Feature Dimensionality Reduction and Orthogonality Constraints

[0078] Dimensionality reduction layer: fully connected layer (128→24), weight matrix

[0079] Output: 24-dimensional low-redundancy feature vector Z;

[0080] Regularization constraints: adding terms to the loss function

[0081] Covariance constraint: adding terms to the loss function

[0082] Step 6: Dynamically expand the classification layer

[0083] Classification layer: fully connected (24→N categories),

[0084] Classification layer weight matrix

[0085] Using cross entropy loss

[0086] Add n categories

[0087] Classification layer weight matrix: from Expand to

[0088] New weight initialization: The new weight is initialized to the mean vector of the old category features. The neighborhood of the old category mean can accelerate the convergence of the new category.

[0089] Right now:

[0090] Among them, μ i is the mean of the i-th category feature;

[0091] KL divergence constraint: L KL =D KL (P old ||P new );

[0092] Total loss function: L = L CE +λ1L orth +λ2L decorr +λ3L KL ,

[0093] The cross entropy loss coefficient is 1,

[0094] Other losses are assigned different coefficients,

[0095] The recommended range of coefficients is λ1∈[0, 0.1], λ2∈[0, 0.3], λ3∈[0, 1.2];

[0096] Preferably, step 7: model training

[0097] Initial training:

[0098] Samples: >50 samples per category;

[0099] Loss coefficient: Orthogonal regularization term (λ1), recommended to be set to 0.1, to balance dimensionality reduction efficiency and feature expression ability; feature decoupling constraint (λ2), recommended to be set to 0.3, to suppress high-dimensional feature redundancy;

[0100] KL divergence loss (λ3) should be disabled during initial training and set to 0, which is used during the incremental learning phase;

[0101] New category training (freeze feature extraction and dimensionality reduction layer parameters):

[0102] Sample: Suitable for small sample training, with more than 10 samples of a single category added;

[0103] Loss coefficient: Due to freezing the parameters of the feature extraction layer and feature dimensionality reduction layer,

[0104] Only the classification layer is trained, and the orthogonality regularization term (λ1) and feature decoupling constraint (λ2) are both set to 0;

[0105] KL divergence loss (λ3),

[0106] It is recommended to set it to 1.2 to retain old knowledge by constraining the consistency of the output distribution of the new and old models. The coefficient is set larger to strengthen the retention of old knowledge.

[0107] Preferably, add new categories for training, retain a small number of samples, combine new samples, and tune the overall model:

[0108] Samples: >10 samples of a single category in the old category, >50 samples of a single category in the new category;

[0109] Loss coefficient: Orthogonal regularization term (λ1), recommended to be set to 0.1, to balance dimensionality reduction efficiency and feature expression ability; feature decoupling constraint (λ2), recommended to be set to 0.3, to suppress high-dimensional feature redundancy;

[0110] The KL divergence loss (λ3), set to 0.5, preserves old knowledge by constraining the consistency of the output distribution of the new and old models.

[0111] Beneficial effects of this technical solution:

[0112] 1. Mask-guided multi-spatial channel attention mechanism: The MaskSE module is used to achieve dynamic weight distribution across color spaces, strengthen the response intensity of key color channels (such as the a / b channels in LAB), reduce background and defect interference, and improve adaptability to complex lighting and color difference scenes.

[0113] 2. Differentiable Orthogonal Dimensionality Reduction: This method transforms the mathematical properties of traditional PCA into orthogonal constraints for neural networks, retaining the interpretability of linear dimensionality reduction while also enabling end-to-end optimization for nonlinear fitting and reducing feature redundancy. 3. Incremental Parameter Freezing: This method proposes a dual mechanism of parameter freezing at the classification layer and reuse of feature prototypes. This allows for rapid adaptation to new categories through fine-tuning with a small number of samples while maintaining the accuracy of existing categories when expanding to new categories.

[0114] 4. Hybrid regularization constraint system: Innovatively integrates orthogonality constraints, covariance constraints, and mutual information constraints to construct a theoretically guaranteed feature decoupling space, which increases the classification hyperplane interval by 2.3 times.

[0115] 5. Breakthrough in environmental adaptability and improved special material detection capabilities

[0116] Multispectral feature fusion: RGB, HLS (brightness separation), LAB (color deviation sensitivity), and YUV (chromaticity separation) complement each other;

[0117] Regularized constrained dimensionality reduction: By diagonalizing the covariance, the PCA characteristics are approximated to improve the linear separability of features after dimensionality reduction.

[0118] The above method reduces the sensitivity to lighting changes and solves the universality problem of color detection of special materials (resin / glass).

[0119] 2. Automated feature extraction

[0120] In CNN feature extraction, dynamic attention (MaskSE module) is introduced to eliminate the interference of background, defects, reflections, etc. on color feature extraction, dynamically allocate weights of different color channels, improve the sensitivity to key color channels (such as the a / b channels in LAB), and avoid the limitations of manually designed features (such as color moments and histogram statistics).

[0121] 3. Facilitate incremental expansion and improve small sample training capabilities

[0122] Through the dynamic category expansion mechanism, the color detection system achieves continuous learning capabilities across materials and product lines, solving the problems of model generalization and knowledge transfer in industrial scenarios.

[0123] Traditional methods require redesigning features and training classifiers to add new color categories, and old categories are easily forgotten (catastrophic forgetting).

[0124] This method adopts a method that can dynamically expand the classification layer. When adding a new category, through the KL divergence constraint, only a small number of new category samples are needed for fast training while ensuring the accuracy of the old category, which can achieve the effect of the new category approaching the accuracy of the old category. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1 This is the original image of S1;

[0126] Figure 2 is the S1Mask image;

[0127] Figure 3 The image after filling for S1;

[0128] Figure 4 The RGB space is an image with R, G, and B channels in sequence;

[0129] Figure 5 The HLS space is composed of H, L, and S channel images respectively;

[0130] Figure 6 The 4LAB space is L, A, and B channel images respectively;

[0131] Figure 7 The YUV space is an image with Y, U, and V channels in sequence;

[0132] Figure 8 This is the comparison table of feature redundancy of S3 feature dimensionality reduction layer;

[0133] Figure 9 The process of the present invention Figure 1 ;

[0134] Figure 10 The process of the present invention Figure 2 .

[0135] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0136] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, top, bottom, inside, outside, vertical, horizontal, longitudinal, counterclockwise, clockwise, circumferential, radial, axial...), then the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0137] In addition, if there are descriptions involving "first" or "second" in the embodiments of the present invention, the descriptions of "first" or "second" are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0138] This paper proposes an intelligent color feature processing technology system for industrial products. Through the innovative design of multimodal feature fusion and dynamically scalable architecture, it effectively solves the color recognition problem in complex industrial scenarios. The following is a detailed explanation from the perspectives of technical architecture and innovative features:

[0139] 1. Technical Architecture System

[0140] S1: Template matching and positioning, creating a valid Mask image and filling the interference area;

[0141] Morphology is combined with edge detection algorithm (this algorithm is an existing technology, mainly used to obtain material contours) to construct a multi-scale contour extractor. The pseudo edges caused by uneven lighting are eliminated through corrosion and expansion operations, and a target contour template is generated for template matching and positioning, which can adapt to the precise positioning of materials of different colors.

[0142] After locating the target, the median color of the target area is calculated. Any area within the target area where the color difference from the median exceeds a set threshold is considered an interference area, along with the background. The interference area is uniformly filled with black to eliminate background color interference on color classification. A mask image is created, with the interference area set to 0 and the remaining area, the valid target area, set to 1. This eliminates interference from background and defects on color classification, ensuring that subsequent feature extraction targets only the valid target area.

[0143] Figure 1 The original image (green spots are defect interference and white is the background);

[0144] Figure 2 is the Mask image (white is the effective area, black is the interference area);

[0145] Figure 3 is the image after filling (black is the filled interference, and the others are valid areas);

[0146] S2: Multispectral feature fusion, introducing the MaskSE module, and the main network extracts deep features;

[0147] Construct a four-dimensional color space joint representation system, including:

[0148] 1. RGB space ( Figure 4 ): Preserve the original sensor response characteristics (split the image after filling in step 1 into channels, the following figure shows the R, G, and B channel images in sequence);

[0149] 2.HLS Space( Figure 5 ): Extract hue-brightness decoupling features (H, L, S channel images in sequence);

[0150] 3.LAB space ( Figure 6 ): Construct color perception uniformity space (L, A, B channel images in sequence);

[0151] 4.YUV space ( Figure 7 ): Separate the brightness and chrominance components (Y, U, V channel images in sequence);

[0152] The MaskSE module is introduced to improve the spatial feature compression of global pooling in the SE module (Squeeze-and-Excitation) into local average pooling of the effective area of ​​the Mask.

[0153] Generate channel description vector S:

[0154] Where M is the Mask image,

[0155] F is the joint representation of color space,

[0156] N active Indicates the total number of pixels in the effective area of ​​the Mask.

[0157] In color images, images usually contain multiple channels, such as three channels in RGB (red, green, blue) mode.

[0158] The channel description vector can be used to represent the feature information of each channel.

[0159] For example, the channel description vector of an RGB image may contain information such as the pixel value range of each channel (such as the minimum value of the red channel is 0 and the maximum value is 255, etc.), the relative brightness ratio between channels, etc.

[0160] Two fully connected layers are used to learn the nonlinear relationship between channels and dynamically adjust the weights of each color channel. After assigning the channel weights to the corresponding channels, batch normalization (BN) is introduced to address the issue of spatial distribution differences in different color spaces. The main network is then used for feature extraction.

[0161] S3: Feature dimensionality reduction layer

[0162] Constructing a learnable affine transformation matrix The PCA-like properties are achieved through the following constraints:

[0163] 1. Orthogonal regularization term:

[0164] Among them, W proj is the weight matrix of the dimensionality reduction layer, which improves the feature expression ability by optimizing the orthogonality of the column vectors of the dimensionality reduction projection matrix.

[0165] 2. Covariance constraint:

[0166] Among them, Z is the output feature of the dimensionality reduction layer, which improves linear separability by constraining the low correlation between features.

[0167] Assign loss coefficients to each constraint item, which can be trained jointly with the feature extraction network and classification network.

[0168] Performance impact and tuning verification, adding orthogonality constraints and covariance constraints can significantly improve the robustness of the model while suppressing feature redundancy (such as Figure 8 ).

[0169] S4: Dynamically Extended Classifier

[0170] Propose parameter freezing-local fine-tuning strategy

[0171] Local parameter fine-tuning includes: when adding a new classification, retaining some or all parameters of the old model to avoid catastrophic forgetting of the entire model parameters, and only training the newly added classification parameters in the dynamically expanded classification layer to achieve the effect of adapting to the classification of new categories while the old categories can be correctly classified.

[0172] Training method: Freeze the parameters of the extraction layer and dimensionality reduction layer, and only train the parameters of the classification layer. During training, the weights of the corresponding losses need to be adjusted (such as: Specific implementation - Step 7 - Model training). )

[0173] 1. New category initialization: Add new category parameters to this layer and calculate the mean of the old category features of the original model for initialization to accelerate convergence.

[0174] 2. Knowledge solidification mechanism:

[0175] Add the KL divergence constraint L to the loss KL =D KL (P old ||P new ),

[0176] Prevent old knowledge from being forgotten when new categories are added, and constrain the consistency of the output distribution of the new and old models.

[0177] 3. Small sample adaptation: Freeze the parameters of the feature extraction layer and dimensionality reduction layer, and only train the parameters of the classification layer. Only a small number of new samples are needed to approach the classification accuracy of the old model.

[0178] 2. Technical Effect

[0179] 1. Mask-guided multi-spatial channel attention mechanism: The MaskSE module is used to achieve dynamic weight distribution across color spaces, strengthen the response intensity of key color channels (such as the a / b channels in LAB), reduce background and defect interference, and improve adaptability to complex lighting and color difference scenes.

[0180] 2. Differentiable Orthogonal Dimensionality Reduction: This method converts the mathematical properties of traditional PCA into orthogonal constraints for neural networks, retaining the interpretability of linear dimensionality reduction while also enabling end-to-end optimized nonlinear fitting and reducing feature redundancy.

[0181] 3. Incremental parameter freezing strategy: A dual mechanism of freezing classification layer parameters and reusing feature prototypes is proposed. When expanding new categories, while maintaining the accuracy of old categories, it can quickly adapt to new categories through fine-tuning with a small amount of samples.

[0182] 4. Hybrid regularization constraint system: Innovatively integrates orthogonality constraints, covariance constraints, and mutual information constraints to construct a theoretically guaranteed feature decoupling space, which increases the classification hyperplane interval by 2.3 times.

[0183] 5. Breakthrough in environmental adaptability and improved special material detection capabilities

[0184] Multispectral feature fusion: RGB, HLS (brightness separation), LAB (color deviation sensitivity), YUV (chromaticity separation) complementary

[0185] Regularized constrained dimensionality reduction: By diagonalizing the covariance, the PCA characteristics are approximated to improve the linear separability of features after dimensionality reduction.

[0186] The above method reduces the sensitivity to lighting changes and solves the universality problem of color detection of special materials (resin / glass).

[0187] 2. Automated feature extraction

[0188] In CNN feature extraction, dynamic attention (MaskSE module) is introduced to eliminate the interference of background, defects, reflections, etc. on color feature extraction, dynamically allocate weights of different color channels, improve the sensitivity to key color channels (such as the a / b channels in LAB), and avoid the limitations of manually designed features (such as color moments and histogram statistics).

[0189] 3. Facilitate incremental expansion and improve small sample training capabilities

[0190] Through the dynamic category expansion mechanism, the color detection system achieves continuous learning capabilities across materials and product lines, solving the problems of model generalization and knowledge transfer in industrial scenarios.

[0191] Traditional methods require redesigning features and training classifiers to add new color categories, and old categories are easily forgotten (catastrophic forgetting).

[0192] This method adopts a method that can dynamically expand the classification layer. When adding a new category, through the KL divergence constraint, only a small number of new category samples are needed for fast training while ensuring the accuracy of the old category, which can achieve the effect of the new category approaching the accuracy of the old category.

[0193] Step 1: Morphological detection of target contours

[0194] Input: RGB images captured by industrial cameras.

[0195] Morphological processing:

[0196] Use a 5×5 rectangular structure element to perform an opening operation (erode first and then dilate) to eliminate small noise (such as dust and reflective spots).

[0197] Canny edge detection to extract the outer contour of parts.

[0198] The closing operation (dilation followed by erosion, structuring element 7×7) connects the broken edges and generates a closed contour template.

[0199] Output: Binarized contour template, marking the target area as 1 and the background as 0.

[0200] The first step is to erode, eliminate isolated noise points smaller than structural elements, disconnect narrow connected parts, and shrink the object boundary;

[0201] Then, the image is expanded to restore the approximate shape of the object, but the noise points that have been eroded will not be restored.

[0202] Function: Removes small noise, smoothes object outlines, and disconnects narrow connections. Applicable scenarios: Image denoising, object separation, edge optimization, etc.

[0203] Step 2: Template matching and interference area filtering

[0204] Template matching: The contour template is matched with the real-time image through normalized cross correlation (NCC) to locate the target area.

[0205] Color median calculation: Calculate the median of the RGB channels within the target area.

[0206] Color difference threshold setting: The color difference between each pixel and the median is calculated for each channel separately, and the threshold is set to 25.

[0207] Mask generation:

[0208] Pixels with a color difference of >25 are considered as interference (such as reflections, scratches, etc.) and are filled with 0.

[0209] The background area is filled with 0.

[0210] The remaining area is the valid target area and is filled with 1.

[0211] Image filling: Fill the RGB image positions corresponding to the areas with 0 in the Mask with 0.

[0212] Function: Eliminate the influence of reflective spots on color feature extraction.

[0213] Step 3: Multi-color space fusion

[0214] Color space conversion and merging: RGB images are converted into HLS, LAB, and YUV, and 12 channels, RGB (3 channels), HLS (3 channels), LAB (3 channels), and YUV (3 channels), are merged into a 12×H×W tensor.

[0215] Step 4: Improved MaskSE module and feature extraction

[0216] SE module improvements:

[0217] Local average pooling: Calculate the mean of each channel only for the area where Mask = 1, replacing global average pooling.

[0218] Dynamic weight allocation: Generate channel weights through two fully connected layers.

[0219] Convolutional layer structure:

[0220] 5 convolution layers: Each layer uses a 3×3 kernel and a ReLU activation function.

[0221] Output: 128-dimensional feature vector.

[0222] Step 5: Feature Dimensionality Reduction and Orthogonality Constraints

[0223] Dimensionality reduction layer: fully connected layer (128→24), weight matrix

[0224] Output: 24-dimensional low-redundancy feature vector Z.

[0225] Regularization constraints: adding terms to the loss function

[0226] Covariance constraint: adding terms to the loss function

[0227] Step 6: Dynamically expand the classification layer

[0228] Classification layer: fully connected (24→N categories),

[0229] Classification layer weight matrix

[0230] Using cross entropy loss

[0231] Add n categories

[0232] Classification layer weight matrix: from Expand to

[0233] New weight initialization: The new weight is initialized to the mean vector of the old category features. The neighborhood of the old category mean can accelerate the convergence of the new category.

[0234] Right now:

[0235] (μ i is the feature mean of the i-th category).

[0236] KL divergence constraint: L KL =D KL (Pold ||P new ).

[0237] Total loss function: L = L CE +λ1L orth +λ2L decorr +λ3L KL ,

[0238] The cross entropy loss coefficient is 1,

[0239] Other losses are assigned different coefficients,

[0240] The recommended range of coefficients is λ1∈[0, 0.1], λ2∈[0, 0.3], λ3∈[0, 1.2].

[0241] Step 7: Model Training

[0242] Initial training:

[0243] Samples: >50 samples for each category.

[0244] Loss coefficient: Orthogonality regularization term (λ1), recommended to be set to 0.1, to balance dimensionality reduction efficiency and feature expression capability. Feature decoupling constraint (λ2), recommended to be set to 0.3, to suppress high-dimensional feature redundancy. KL divergence loss (λ3) should be disabled during initial training and set to 0, used during the incremental learning phase.

[0245] New category training (freeze feature extraction and dimensionality reduction layer parameters):

[0246] Sample: Suitable for small sample training, with >10 samples of a single category added.

[0247] Loss coefficient: Due to freezing the parameters of the feature extraction layer and feature dimensionality reduction layer,

[0248] Only the classification layer is trained, and the orthogonality regularization term (λ1) and feature decoupling constraint (λ2) are both set to 0. KL divergence loss (λ3),

[0249] It is recommended to set it to 1.2 to retain old knowledge by constraining the consistency of the output distribution of the new and old models. The coefficient is set larger to strengthen the retention of old knowledge.

[0250] New category training (retain a small number of samples, combine new samples, and tune the overall model):

[0251] Samples: The number of single-class samples of old categories is greater than 10, and the number of single-class samples of new categories is greater than 50.

[0252] Loss coefficients: Orthogonality regularization term (λ1) is recommended to be set to 0.1 to balance dimensionality reduction efficiency and feature expression capability. Feature decoupling constraint (λ2) is recommended to be set to 0.3 to suppress high-dimensional feature redundancy. KL divergence loss (λ3) is set to 0.5 to preserve old knowledge by constraining the consistency of the output distribution of the new and old models.

[0253] The image and original image are image data.

[0254] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformation made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. A method for intelligent automatic extraction of industrial product color features based on machine learning, characterized in that: include: S1: Template matching and positioning, creating a valid Mask image and filling the interference area; S11: A multi-scale contour extractor is constructed by combining morphology with an edge detection algorithm. Erosion and dilation operations are used to eliminate pseudo edges caused by uneven illumination, generating a target contour template for template matching and positioning, and adapting to the positioning of materials of different colors. S12: After locating the target outline, calculate the color median of the target area. If the color difference between the target area and the median exceeds the set threshold, The interference area is considered as the interference area together with the background, and the interference area is uniformly filled with black to eliminate the interference of background color on color classification; S13: Create a Mask image, set the interference area to 0, and the other areas as valid target areas, set to 1; S14: Eliminate the interference of background and defects on color classification, and make subsequent feature extraction focus on the effective target area; S15 obtains the original image, the mask image, and the filled image.

2. The method for intelligent automatic extraction of industrial product color features based on machine learning according to claim 1, characterized in that: S2: Multispectral feature fusion, introducing the MaskSE module, and the main network extracts deep features; Construct a four-dimensional color space joint representation system, including: S21, RGB space: retains the original sensor response characteristics and splits the S15 padded image into R, G, and B channel images; S22.HLS space: extracts hue-brightness decoupling features into H, L, and S channel images; S23.LAB space: Constructs a color perception uniformity space for L, A, and B channel images; S24.YUV space: Separates brightness and chrominance components into Y, U, and V channel images.

3. The method for intelligent automatic extraction of industrial product color features based on machine learning according to claim 2, characterized in that: S24: Introduce the MaskSE module to improve the spatial feature compression of the global pooling in the SE module into the local average pooling of the effective area of ​​the Mask. Generate channel description vector S: Where M is the Mask image, F is the joint representation of color space, N active Indicates the total number of pixels in the effective area of ​​the Mask.

4. The method for intelligent automatic extraction of industrial product color features based on machine learning according to claim 3, characterized in that: S3: Feature dimensionality reduction layer Constructing a learnable affine transformation matrix PCA-like features are achieved through the following constraints; S31: Orthogonal regularization term: in, W proj is the weight matrix of the dimensionality reduction layer, which improves the feature expression capability by optimizing the orthogonality of the column vectors of the dimensionality reduction projection matrix; S32: Covariance constraint term: Among them, Z is the output feature of the dimensionality reduction layer, which improves linear separability by constraining the low correlation between features; Assign loss coefficients to each constraint item, which can be trained jointly with the feature extraction network and classification network.

5. The method for intelligent automatic extraction of industrial product color features based on machine learning according to claim 4, characterized in that: S4: Dynamically expand the classifier including the proposed parameter freezing and local fine-tuning strategies; The local fine-tuning of parameters includes: S41: When adding a new classification, retain some or all parameters of the old model to avoid catastrophic forgetting of the entire model parameters. S42: Train the newly added classification parameters in the dynamically expanded classification layer to achieve the effect of adapting to the classification of new categories while the old categories can be correctly classified.

6. The method for intelligent automatic extraction of industrial product color features based on machine learning according to claim 5, characterized in that: S43: The training method includes freezing the parameters of the extraction layer and the dimensionality reduction layer. Only the classification layer parameters are trained, and the corresponding loss weights need to be adjusted during training; S431. New category initialization: add new category parameters of the corresponding layer, calculate the mean of the old category features of the original model to initialize and accelerate convergence; S432. Knowledge solidification mechanism: Add the KL divergence constraint L to the loss KL =D KL (P old ||P new ), Prevent old knowledge from being forgotten when adding new categories, and constrain the consistency of the output distribution of the new and old models; S433. Small Sample Adaptation: Freeze the parameters of the feature extraction and dimensionality reduction layers and train only the classification layer parameters. This allows the classification accuracy of the old model to be approached with only a small number of new samples.

7. A method for intelligent automatic detection of color features of industrial products based on machine learning, characterized by: A method for intelligent automatic extraction of color features of industrial products based on machine learning according to any one of claims 1 to 6, include: Step 1: Morphological detection of target contours Input: images captured by RGB industrial cameras; Morphological processing: An opening operation is performed using a 5×5 rectangular structure element, wherein the operation is first erosion and then expansion to eliminate small noises; Canny edge detection to extract the outer contour of parts; The closing operation is dilation followed by erosion, connecting the broken edges to generate a closed contour template; Output: Binarized contour template, marking the target area as 1 and the background as 0; Step 2: Template matching and interference area filtering Template matching: Match the contour template with the real-time image through normalized cross correlation (NCC) to locate the target area; Color median calculation: Calculate the median of the RGB channels within the target area; Color difference threshold setting: calculate the color difference between each pixel and the median value for each channel separately, and set the threshold to 25; Mask generation: Pixels with a color difference of >25 are considered as interference (such as reflections, scratches, etc.) and filled with 0; The background area is filled with 0; The rest of the area is the valid target area and is filled with 1; Image filling: fill the RGB image position corresponding to the area with 0 in the Mask with 0; Step 3: Multi-color space fusion; Color space conversion and merging: RGB images are converted into HLS, LAB, and YUV, and 12 channels, RGB (3 channels), HLS (3 channels), LAB (3 channels), and YUV (3 channels), are merged into a 12×H×W tensor; Step 4: Improved MaskSE module and feature extraction; SE module improvements: Local average pooling: Calculate the mean of each channel only for the area where Mask = 1, replacing global average pooling; Dynamic weight allocation: Generate channel weights through two fully connected layers; Convolutional layer structure: 5 convolution layers: each layer uses a 3×3 kernel and ReLU activation function; Output: 128-dimensional feature vector; Step 5: Feature Dimensionality Reduction and Orthogonality Constraints Dimensionality reduction layer: fully connected layer (128→24), weight matrix Output: 24-dimensional low-redundancy feature vector Z; Regularization constraints: adding terms to the loss function Covariance constraint: adding terms to the loss function Step 6: Dynamically expand the classification layer Classification layer: fully connected (24→N categories), Classification layer weight matrix Using cross entropy loss Add n categories Classification layer weight matrix: from Expand to New weight initialization: The new weight is initialized to the mean vector of the old category features. The neighborhood of the old category mean can accelerate the convergence of the new category. Right now: Among them, μ i is the mean of the i-th category feature; KL divergence constraint: L KL =D KL (P old ||P new ); Total loss function: L = L CE +λ1L orth +λ2L decorr +λ3L KL , The cross entropy loss coefficient is 1, Other losses are assigned different coefficients, The recommended range of coefficients is λ1∈[0,0.1], λ2∈[0,0.3], λ3∈[0,1.2].

8. The method for intelligent automatic detection of industrial product color features based on machine learning according to claim 7, characterized in that: Step 7: Model Training Initial training: Samples: >50 samples per category; Loss coefficient: Orthogonal regularization term (λ1), it is recommended to be set to 0.1 to balance the efficiency of dimensionality reduction and the ability to express features; Feature decoupling constraint (λ2), which is recommended to be set to 0.3, suppresses high-dimensional feature redundancy; KL divergence loss (λ3) should be disabled during initial training and set to 0, which is used during the incremental learning phase; New category training (freeze feature extraction and dimensionality reduction layer parameters): Sample: Suitable for small sample training, with more than 10 samples of a single category added; Loss coefficient: Due to freezing the parameters of the feature extraction layer and feature dimensionality reduction layer, Only the classification layer is trained, and the orthogonality regularization term (λ1) and feature decoupling constraint (λ2) are both set to 0; KL divergence loss (λ3), It is recommended to set it to 1.2 to retain old knowledge by constraining the consistency of the output distribution of the new and old models. The coefficient is set larger to strengthen the retention of old knowledge.

9. The method for intelligent automatic detection of industrial product color features based on machine learning according to claim 7, characterized in that: Add new category training, retain a small number of samples, combine new samples, and tune the overall model: Samples: >10 samples of a single category for old categories, >50 samples of a single category for new categories; Loss coefficient: Orthogonal regularization term (λ1), it is recommended to be set to 0.1 to balance the efficiency of dimensionality reduction and the ability to express features; Feature decoupling constraint (λ2), which is recommended to be set to 0.3, suppresses high-dimensional feature redundancy; The KL divergence loss (λ3), set to 0.5, preserves old knowledge by constraining the consistency of the output distribution of the new and old models.

Images