Industrial button color intelligent recognition method and system based on YOLO and SVM fusion

Abstract

This invention discloses an intelligent color recognition method and system for industrial buttons based on the fusion of YOLO and SVM. The invention inputs the image to be detected into the YOLO model to obtain the button bounding box, preliminary classification results, and confidence scores. After cropping the ROI, the mean, standard deviation, and median of the HSV and RGB spaces are extracted to construct a standardized feature vector, which is then input into the SVM to obtain refined classification results and confidence scores. Finally, based on the confidence threshold, an adaptive fusion strategy using YOLO priority, SVM priority, consistent weighting, or weighted voting is employed to obtain the final color recognition result. This invention balances speed and accuracy through a two-stage coarse detection to fine classification pipeline, enhances robustness through multi-color space statistical features, and achieves confidence-driven intelligent decision-making through the adaptive fusion strategy. It can also interface with MES and PLC to complete product release or interception, significantly improving the automation and reliability of industrial button color detection.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and intelligent manufacturing technology, and in particular to an intelligent color recognition method and system for industrial buttons based on the fusion of YOLO and SVM. Background Technology

[0002] Traditional industrial button color recognition methods mainly rely on manual visual inspection, but manual inspection has problems such as low efficiency, high cost, susceptibility to subjective factors, and visual fatigue and errors caused by long-term work.

[0003] With the development of machine vision technology, button color recognition methods based on image processing have emerged. Existing technologies commonly employ color classification methods based on simple threshold segmentation. However, these methods are significantly affected by lighting conditions and struggle to handle complex backgrounds, color gradients, and changes in ambient light, resulting in insufficient robustness.

[0004] Support Vector Machines (SVMs), a classic machine learning method, perform well in classification tasks with small sample sizes and high-dimensional features. By using manually designed color histograms, textures, and other features, SVMs can capture subtle differences in color distribution and achieve some success in color classification. However, SVMs themselves do not have object localization capabilities and rely on accurate region extraction as a prerequisite.

[0005] In recent years, deep learning technology has made significant progress in image classification and object detection. Real-time object detection models, represented by the YOLO (YouOnly Look Once) series, can quickly and accurately locate target objects in images and provide class predictions, offering advantages such as fast detection speed and high localization accuracy. However, in fine-grained color classification tasks, the classification accuracy of deep learning models is often limited by factors such as the number of training samples, the range of color variations, and lighting conditions, making it difficult to achieve accurate tone differentiation.

[0006] Currently, existing technologies typically use only a single detection or classification method, failing to fully leverage the complementary advantages of different methods. For example, while using deep learning alone for end-to-end color recognition has advantages in localization, it may not be as accurate as traditional methods in fine color differentiation; while using traditional methods such as SVM alone, although providing fine classification, requires that the input region has already been accurately segmented and lacks automatic localization capabilities.

[0007] Furthermore, many existing multi-model fusion methods employ simple voting or averaging strategies, failing to adequately consider the differences in the confidence levels of each model's output. When a model delivers a result with high confidence, it should rightfully receive higher decision weights, while results with low confidence should be downweighted or ignored. Current technologies lack adaptive decision-making mechanisms based on confidence levels.

[0008] Furthermore, the statistical features of a single color space are insufficient to fully describe the color characteristics of a button. In real-world industrial scenarios, button colors are influenced by multiple factors such as material, lighting, and angle, requiring the extraction of complementary features from multiple color spaces. Additionally, single-region sampling is susceptible to interference from local anomalies (such as reflections, stains, and shadows), affecting the robustness of the recognition results.

[0009] Therefore, how to comprehensively utilize the advantages of deep learning and traditional machine learning to achieve high-precision and robust intelligent recognition of industrial button colors under complex conditions, and to achieve seamless integration with industrial field equipment, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0010] The purpose of this invention is to provide an intelligent color recognition method and system for industrial buttons based on the fusion of YOLO and SVM, so as to solve the problems of existing technologies where a single model is difficult to balance accuracy and real-time performance and lacks an adaptive fusion mechanism.

[0011] The technical solution adopted in this invention is:

[0012] An intelligent color recognition method for industrial buttons based on the fusion of YOLO and SVM includes the following steps:

[0013] Step S1: Obtain the image to be inspected, which includes the industrial button;

[0014] Step S2: Input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification result, and the corresponding YOLO confidence score;

[0015] Step S3: Based on the bounding box coordinates, crop out the ROI region of the industrial button from the image to be detected;

[0016] Step S4: Extract the statistical features of the ROI region in multiple color spaces and construct a multidimensional feature vector;

[0017] Step S5: Input the multidimensional feature vector into the trained SVM classifier to obtain the fine classification results of SVM and the corresponding SVM confidence.

[0018] Step S6: Based on the adaptive confidence fusion algorithm, according to the YOLO confidence and SVM confidence, perform a fusion decision on the YOLO preliminary classification results and the SVM fine classification results to determine the final button color recognition result;

[0019] Step S7: Compare the final button color recognition result with the preset color configuration, determine whether it is qualified or unqualified, and output the judgment result.

[0020] Furthermore, the YOLO object detection model adopts a lightweight network structure and is trained using an industrial button dataset containing multiple color combinations during the training phase. The color combinations include at least white frame white button, white frame black button, white frame blue button, black frame white button, black frame black button, black frame blue button, blue frame white button, blue frame black button, and blue frame blue button.

[0021] Furthermore, step S4 specifically includes:

[0022] In the HSV color space, the mean, standard deviation, and median of the hue, saturation, and lightness channels within the ROI are calculated, resulting in a total of 9 features.

[0023] In the RGB color space, calculate the mean and standard deviation of the red, green, and blue channels within the ROI, resulting in a total of 6-dimensional features;

[0024] The extracted 9-dimensional features in HSV space are concatenated with the 6-dimensional features in RGB space to construct a 15-dimensional feature vector;

[0025] The 15-dimensional feature vectors are standardized to eliminate dimensional differences.

[0026] Furthermore, the SVM classifier employs the RBF kernel function and undergoes multi-class training for specific color categories of industrial buttons. These specific color categories include at least black-black, black-blue, black-white, blue-black, blue-blue, blue-white, white-black, white-blue, and white-white.

[0027] Furthermore, the adaptive confidence fusion algorithm in step S6 specifically includes the following decision-making strategies:

[0028] YOLO priority strategy: When the YOLO confidence score is higher than the preset high confidence threshold and the SVM confidence score is lower than the preset low confidence threshold, the YOLO preliminary classification result is selected;

[0029] SVM priority strategy: When the SVM confidence score is higher than the preset high confidence threshold, select the SVM fine classification result;

[0030] Consistent Result Strategy: When the initial classification result of YOLO is consistent with the fine classification result of SVM, the confidence scores of the two are weighted and averaged.

[0031] Weighted voting strategy: If none of the above conditions are met, calculate the weighted score of the YOLO preliminary classification result and the SVM fine classification result, and select the result with the highest score as the final recognition result.

[0032] Furthermore, step S7 specifically includes:

[0033] The final button color recognition result is compared with the preset color configuration to determine whether it is qualified or unqualified.

[0034] The judgment result is displayed in the graphical user interface;

[0035] The identification and judgment results are recorded in the local database and sent to the manufacturing execution system and PLC controller;

[0036] If the result is qualified, the product is released through the PLC controller; if the result is unqualified, the PLC controller will trigger an alarm and intercept the product.

[0037] This invention discloses an industrial button color intelligent recognition system based on the fusion of YOLO and SVM, used to execute the above-described method, the system comprising:

[0038] Image acquisition module, used to acquire images of the object to be inspected, including industrial buttons;

[0039] The YOLO detection module is used to input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification results, and the corresponding YOLO confidence scores.

[0040] The region clipping module is used to clip the ROI region of the industrial button based on the bounding box coordinates;

[0041] The feature extraction module is used to extract the statistical features of the ROI region in multiple color spaces and construct a multidimensional feature vector.

[0042] The SVM classification module is used to input multidimensional feature vectors into a trained SVM classifier to obtain fine SVM classification results and corresponding SVM confidence scores.

[0043] The adaptive fusion decision module is used to perform fusion decision on the preliminary classification results of YOLO and the fine classification results of SVM based on the adaptive confidence fusion algorithm, according to the confidence of YOLO and SVM, to determine the final button color recognition result;

[0044] The result output module is used to compare the final button color recognition result with the preset color configuration, generate a qualified or unqualified judgment result, and output it.

[0045] Furthermore, the image acquisition module is an industrial camera.

[0046] Furthermore, the YOLO detection module, region cropping module, feature extraction module, SVM classification module, adaptive fusion decision module, and result output module in the system are deployed on an industrial-grade PC.

[0047] Furthermore, the industrial-grade PC is equipped with a Windows 10 operating system, an Intel Core i7-6700 CPU, a 1TB hard drive, and 8GB of memory.

[0048] The present invention adopts the above technical solution and has the following beneficial technical effects:

[0049] 1. This invention adopts a two-stage fusion architecture of YOLO and SVM to realize a two-stage pipeline from coarse detection to fine classification, overcoming the problem that a single model cannot balance accuracy and real-time performance.

[0050] 2. This invention combines HSV and RGB dual color spaces to construct feature vectors and uses SVM for fine classification, effectively overcoming the impact of complex lighting environments on color recognition.

[0051] 3. This invention designs an adaptive confidence fusion algorithm that dynamically selects the optimal decision strategy based on the output states of YOLO and SVM, overcoming the problem of lack of adaptive decision-making mechanism in multi-model fusion and realizing confidence-driven intelligent decision-making.

[0052] 4. This invention integrates serial communication and MES system interface to achieve seamless integration with industrial PLC control and MES system, providing an intelligent detection and identification solution from algorithm to system. Attached Figure Description

[0053] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0054] Figure 1 This is a schematic diagram of the framework of the identification system of the present invention;

[0055] Figure 2 This is a schematic diagram of the button style;

[0056] Figure 3 This is a schematic diagram of a graphical user interface. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0058] like Figure 1 As shown, this embodiment provides an industrial button color intelligent recognition system based on the fusion of YOLO and SVM. The system includes:

[0059] Image acquisition module: Used to acquire images of the industrial buttons to be inspected. In this embodiment, the module consists of an industrial camera and a light source, such as a Logitech C922 industrial camera and a matching high-uniformity bar light source, used to acquire images of the buttons to be inspected on the production line in real time.

[0060] YOLO detection module: Used to input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification result, and the corresponding YOLO confidence score.

[0061] Region clipping module: Used to clip the ROI (Region of Interest) of industrial buttons based on bounding box coordinates.

[0062] Feature extraction module: used to extract statistical features of ROI regions in multiple color spaces and construct multidimensional feature vectors.

[0063] SVM classification module: Used to input multi-dimensional feature vectors into a trained SVM classifier to obtain fine-grained SVM classification results and corresponding SVM confidence scores.

[0064] Adaptive Fusion Decision Module: Based on the adaptive confidence fusion algorithm, it performs fusion decision on the preliminary classification results of YOLO and the fine classification results of SVM according to the confidence of YOLO and SVM, and determines the final button color recognition result.

[0065] The result output module is used to compare the final button color recognition result with the preset color configuration, generate a qualified or unqualified judgment result, and output it.

[0066] The YOLO detection module, region clipping module, feature extraction module, SVM classification module, adaptive fusion decision module, and result output module in the above system are deployed on an industrial-grade PC (e.g., configured with Windows 10 operating system, Intel Core i7-6700 CPU, 1TB hard drive, 8GB memory), and interact with the PLC controller via serial port and the Manufacturing Execution System (MES) via Ethernet port.

[0067] This embodiment also provides an industrial button color intelligent recognition method based on the fusion of YOLO and SVM. This method can run on the above-mentioned system, and the specific steps include:

[0068] Step S1: Acquire an image of the industrial button to be inspected using an industrial camera.

[0069] Step S2: Input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification result, and the corresponding YOLO confidence score.

[0070] Among them, the YOLO object detection model adopts a lightweight network structure (such as YOLOv5s or YOLOv8) and is trained in the training stage using an industrial button dataset containing various color combinations. As Figure 2 shown, the color combinations at least include: white frame white key, white frame black key, white frame blue key, black frame white key, black frame black key, black frame blue key, blue frame white key, blue frame black key, blue frame blue key. The specific training process is as follows: In the industrial site detection environment, collect pictures of the above nine color combinations as the dataset; use annotation software to perform rectangular box annotation on the collected pictures and convert them into the label type supported by the YOLO algorithm; shuffle the pictures and the corresponding labels and divide them into training set, validation set and test set according to a certain ratio; use the YOLO algorithm model for training and enhance the generalization ability of the model through data augmentation.

[0071] Step S3: According to the bounding box coordinates, crop out the ROI region of the industrial button from the image to be detected.

[0072] Step S4: Extract the statistical features of the ROI region in multiple color spaces and construct a multi-dimensional feature vector.

[0073] In this step, color feature extraction is performed on the ROI region obtained in step S3. Specifically, it includes calculating statistical features in the HSV and RGB color spaces respectively, then splicing them into a 15-dimensional feature vector and performing normalization processing. The detailed process is as follows:

[0074] First, convert each pixel in the ROI region from the RGB color space to the HSV color space. The conversion formula is as follows (assuming r, g, b ∈ [0, 1]):

[0075] Let max = max(r, g, b), min = min(r, g, b);

[0076] Hue H:

[0077] If max = min, then H = 0;

[0078] If max = r and g ≥ b, then H = 60° × (g - b) / (max - min) + 0°;

[0079] If max = r and g < b, then H = 60° × (g - b) / (max - min) + 360°;

[0080] If max = g, then H = 60° × (b - r) / (max - min) + 120°;

[0081] If max = b, then H = 60° × (r - g) / (max - min) + 240°; <00Saturation S: If max=0, then S=0; otherwise, S=(max-min) / max;

[0083] Brightness V: V = max;

[0084] After the conversion is complete, calculate the mean values ​​of the H, S, and V channels in the HSV color space for the entire ROI region. ), standard deviation ( ) and median ( ), a total of 9-dimensional features; simultaneously calculate the mean values ​​of the R, G, and B channels in the RGB color space ( ) and standard deviation ( The data consists of 6 dimensions. The mean and standard deviation were calculated using conventional statistical methods. The median was calculated by sorting all pixel values ​​in the channel from smallest to largest. If the total number of pixels N is odd, the value at the middle position is taken; if N is even, the average of the two middle values ​​is taken.

[0085] The above 15 statistics are concatenated to form a 15-dimensional feature vector. :

[0086] ;

[0087] Finally, the eigenvectors are standardized to eliminate the influence of different eigenvalue units. The specific standardization process is as follows:

[0088] Suppose there are m training samples, and the following is the feature matrix formed by the samples:

[0089] ;

[0090] Since the eigenvalues ​​have different dimensions, they are standardized according to the following expression:

[0091] The element in the i-th row and j-th column Let j represent the j-th feature value of the i-th sample, and calculate the mean of each feature dimension j. and standard deviation :

[0092]

[0093]

[0094]

[0095] in, Characteristic matrix element The standardized value.

[0096] Step S5: Input the multidimensional feature vector

[0097] The trained SVM classifier is used to obtain the fine-grained SVM classification results and the corresponding SVM confidence scores.

[0098] Regarding the training of the SVM classifier: The SVM classifier uses the RBF kernel function and performs multi-class training for specific color categories of industrial buttons during training. The specific color categories include at least: black-black, black-blue, black-white, blue-black, blue-blue, blue-white, white-black, white-blue, and white-white. The specific training process is as follows: Training samples are collected, and a 15-dimensional standardized feature vector is extracted for each training sample; the above color categories are encoded; the multi-class SVM model is trained using the standardized feature matrix and the encoded labels. After training, for the input 15-dimensional feature vector, the SVM classifier outputs a refined classification result and the corresponding confidence score.

[0099] Step S6: Based on the adaptive confidence fusion algorithm, the YOLO preliminary classification result and the SVM fine classification result are fused according to the YOLO confidence and SVM confidence to determine the final button color recognition result.

[0100] In this step, the adaptive confidence fusion algorithm specifically includes the following decision strategies:

[0101] YOLO priority strategy: When the YOLO confidence score is higher than the preset high confidence threshold and the SVM confidence score is lower than the preset low confidence threshold, the YOLO preliminary classification result is selected;

[0102] SVM priority strategy: When the SVM confidence score is higher than the preset high confidence threshold, select the SVM fine classification result;

[0103] Consistent Result Strategy: When the initial classification result of YOLO is consistent with the fine classification result of SVM, the confidence scores of the two are weighted and averaged.

[0104] Weighted voting strategy: If none of the above conditions are met, calculate the weighted score of the YOLO preliminary classification result and the SVM fine classification result, and select the result with the highest score as the final recognition result.

[0105] Step S7: Compare the final button color recognition result with the preset color configuration, determine whether it is qualified or unqualified, and output the judgment result.

[0106] This step compares the final color recognition result obtained in step S6 with the preset color configuration to determine whether it is qualified or unqualified. Specifically, it includes:

[0107] The determination result is displayed on the graphical user interface (GUI), such as... Figure 3 As shown.

[0108] The identification and judgment results are recorded in the local database and sent to the Manufacturing Execution System (MES) and PLC controller.

[0109] If the result is qualified, the product is released through the PLC controller; if the result is unqualified, the PLC controller will trigger an alarm and intercept the product.

[0110] After the above steps, the intelligent recognition and quality assessment of the industrial button colors are completed.

[0111] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Without conflict, the embodiments and features described and illustrated herein can be combined with each other. The components of the embodiments of the present invention generally described and illustrated in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A method for intelligent color recognition of industrial buttons based on the fusion of YOLO and SVM, characterized in that, Includes the following steps: Step S1: Obtain the image to be inspected, which includes the industrial button; Step S2: Input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification result, and the corresponding YOLO confidence score; Step S3: Based on the bounding box coordinates, crop out the ROI region of the industrial button from the image to be detected; Step S4: Extract the statistical features of the ROI region in multiple color spaces and construct a multidimensional feature vector; Step S5: Input the multidimensional feature vector into the trained SVM classifier to obtain the fine classification results of SVM and the corresponding SVM confidence. Step S6: Based on the adaptive confidence fusion algorithm, according to the YOLO confidence and SVM confidence, perform a fusion decision on the YOLO preliminary classification results and the SVM fine classification results to determine the final button color recognition result; Step S7: Compare the final button color recognition result with the preset color configuration, determine whether it is qualified or unqualified, and output the judgment result.

2. The industrial button color intelligent recognition method based on the fusion of YOLO and SVM as described in claim 1, characterized in that: The YOLO object detection model adopts a lightweight network structure and is trained using an industrial button dataset containing multiple color combinations during the training phase. The color combinations include at least white frame white button, white frame black button, white frame blue button, black frame white button, black frame black button, black frame blue button, blue frame white button, blue frame black button, and blue frame blue button.

3. The industrial button color intelligent recognition method based on the fusion of YOLO and SVM as described in claim 1, characterized in that: Step S4 specifically includes: In the HSV color space, the mean, standard deviation, and median of the hue, saturation, and lightness channels within the ROI are calculated, resulting in a total of 9 features. In the RGB color space, calculate the mean and standard deviation of the red, green, and blue channels within the ROI, resulting in a total of 6-dimensional features; The extracted 9-dimensional features in HSV space are concatenated with the 6-dimensional features in RGB space to construct a 15-dimensional feature vector; The 15-dimensional feature vectors are standardized to eliminate dimensional differences.

4. The industrial button color intelligent recognition method based on the fusion of YOLO and SVM as described in claim 1, characterized in that: The SVM classifier uses the RBF kernel function and is trained for multiple classifications for specific color categories of industrial buttons. The specific color categories include at least black-black, black-blue, black-white, blue-black, blue-blue, blue-white, white-black, white-blue, and white-white.

5. The industrial button color intelligent recognition method based on the fusion of YOLO and SVM as described in claim 1, characterized in that: The adaptive confidence fusion algorithm in step S6 specifically includes the following decision-making strategies: YOLO priority strategy: When the YOLO confidence score is higher than the preset high confidence threshold and the SVM confidence score is lower than the preset low confidence threshold, the YOLO preliminary classification result is selected; SVM priority strategy: When the SVM confidence score is higher than the preset high confidence threshold, select the SVM fine classification result; Consistent Result Strategy: When the initial classification result of YOLO is consistent with the fine classification result of SVM, the confidence scores of the two are weighted and averaged. Weighted voting strategy: If none of the above conditions are met, calculate the weighted score of the YOLO preliminary classification result and the SVM fine classification result, and select the result with the highest score as the final recognition result.

6. The industrial button color intelligent recognition method based on the fusion of YOLO and SVM as described in claim 1, characterized in that: Step S7 specifically includes: The final button color recognition result is compared with the preset color configuration to determine whether it is qualified or unqualified. The judgment result is displayed in the graphical user interface; The identification and judgment results are recorded in the local database and sent to the manufacturing execution system and PLC controller; If the result is qualified, the product is released through the PLC controller; if the result is unqualified, the PLC controller will trigger an alarm and intercept the product.

7. An industrial button color intelligent recognition system based on the fusion of YOLO and SVM, characterized in that, The system for performing the method according to any one of claims 1 to 6 comprises: Image acquisition module, used to acquire images of the object to be inspected, including industrial buttons; The YOLO detection module is used to input the image to be detected into the trained YOLO object detection model to obtain the bounding box coordinates of the industrial button, the preliminary YOLO classification results, and the corresponding YOLO confidence scores. The region clipping module is used to clip the ROI region of the industrial button based on the bounding box coordinates; The feature extraction module is used to extract the statistical features of the ROI region in multiple color spaces and construct a multidimensional feature vector. The SVM classification module is used to input multidimensional feature vectors into a trained SVM classifier to obtain fine SVM classification results and corresponding SVM confidence scores. The adaptive fusion decision module is used to perform fusion decision on the preliminary classification results of YOLO and the fine classification results of SVM based on the adaptive confidence fusion algorithm, according to the confidence of YOLO and SVM, to determine the final button color recognition result; The result output module is used to compare the final button color recognition result with the preset color configuration, generate a qualified or unqualified judgment result, and output it.

8. The industrial button color intelligent recognition system based on the fusion of YOLO and SVM as described in claim 7, characterized in that, The image acquisition module is an industrial camera.

9. The industrial button color intelligent recognition system based on the fusion of YOLO and SVM as described in claim 7, characterized in that, The YOLO detection module, region cropping module, feature extraction module, SVM classification module, adaptive fusion decision module, and result output module in the system are deployed on an industrial-grade PC.

10. The industrial button color intelligent recognition system based on the fusion of YOLO and SVM as described in claim 9, characterized in that, The industrial-grade PC is equipped with a Windows 10 operating system, an Intel Core i7-6700 CPU, a 1TB hard drive, and 8GB of memory.

Images