Image recognition and classification system and method based on machine learning

Abstract

The invention discloses an image recognition and classification system and method based on machine learning, and relates to the field of industrial image processing, and the method comprises the steps: collecting image data of an industrial product, and marking related information; the method comprises the following steps: detecting an acquired industrial product image through a self-adaptive method, removing noise in the image, and classifying industrial product materials based on feature extraction; dynamically adjusting the weight of a defect detection task by sharing a CNN backbone network according to the importance of a product type; constructing a multi-task model to evaluate the severity of the defect; constructing a physical parameter calculation function, setting a score by combining a multi-task model output result, performing weighted fusion on the function and the model result, and classifying the severity of product defects according to a fusion result; through adaptive noise removal and material classification, adaptability to a complex industrial environment is enhanced, and through combination of a multi-task model and physical parameters, comprehensive evaluation of defects from multiple dimensions is realized, and classification precision is improved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial image processing, and specifically relates to an image recognition and classification system and method based on machine learning. Background Art

[0002] With the development of industrial automation and intelligence, the quality control of industrial products has put forward higher requirements for image recognition and defect detection technology. Traditional industrial inspection methods usually rely on manual visual inspection or simple image processing algorithms, and have problems such as low efficiency, insufficient accuracy, and poor robustness.

[0003] Existing machine learning-based image recognition technology has made significant progress in the field of industrial defect detection, but it still has certain limitations: factors such as lighting changes and equipment vibration in industrial environments can easily lead to image noise. Traditional methods lack the ability to adaptively process noise, which affects the accuracy of subsequent defect detection. Material classification and defect detection are usually processed as independent tasks, lacking multi-task collaborative optimization, resulting in wasted computing resources and low detection efficiency. The detection tasks for different product types and defect types vary greatly. Traditional methods find it difficult to dynamically adjust detection strategies based on product types, resulting in more important defects being ignored. In addition, they usually rely on a single model output and do not combine physical characteristics for comprehensive evaluation, resulting in incomplete evaluation results.

[0004] Therefore, it is crucial to develop an image recognition and classification system and method based on machine learning, which can adapt to a variety of industrial products, work stably in different industrial environments, perform dynamic defect detection and classification, and build a multi-task model and comprehensive physical parameter scoring for defect classification, which has extremely important practical significance. Summary of the Invention

[0005] The present invention proposes an image recognition and classification system and method based on machine learning to solve the problems of insufficient adaptive processing capability for noise, which affects the accuracy of subsequent defect detection. Material classification and defect detection are usually processed as independent tasks, lacking multi-task collaborative optimization, resulting in waste of computing resources and low detection efficiency. The detection tasks for different product types and defect types vary greatly. Traditional methods find it difficult to dynamically adjust the detection strategy according to the product type, resulting in the possibility that more important defects may be ignored. In addition, they usually rely on a single model output without combining physical characteristics for comprehensive evaluation.

[0006] A first aspect of the present invention provides an image recognition and classification system based on machine learning, comprising the following modules:

[0007] Image acquisition and annotation module: collects industrial product image data through multiple high-definition industrial cameras and annotates the collected industrial product images;

[0008] Adaptive noise monitoring and material classification module: This module detects and removes noise from collected industrial product images through adaptive methods, and classifies the materials of industrial products based on feature extraction.

[0009] Dynamic defect detection and classification module: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs.

[0010] Multi-task model defect assessment module: This module extracts defect areas using the ROI function and constructs a feature extraction network. Based on different defect types, it designs multiple independent assessment heads to output the defect severity category for the corresponding defect type. The F.softmax function is used to normalize the output to obtain the probability distribution of defect severity.

[0011] Scoring fusion classification module: Construct a physical parameter calculation function, combine the output results of the multi-task model, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

[0012] Preferably, the image acquisition and annotation module includes:

[0013] Install multiple high-definition industrial cameras on the industrial production line to shoot industrial products from four angles: front, side, top and bottom;

[0014] Transmit image data from multiple cameras to the data acquisition server in real time via industrial Ethernet;

[0015] Use professional image annotation software to annotate images. The annotation information includes: industrial product type information, defect information, key features and other information.

[0016] Preferably, the image annotation using professional image annotation software includes the following steps:

[0017] Industrial product type information: Assign a unique label to each industrial product type, 0 for high-precision parts, 1 for ordinary parts, and 2 for low-precision parts;

[0018] Defect information: Mark the type, code, and location of the defect, and assign a unique label to each defect type: A for no defect, B for scratches, C for dents, D for cracks, E for rust, F for deformation, G for bubbles, and H for color difference;

[0019] Additional information: production batch, line number, and timestamp.

[0020] Preferably, the adaptive noise monitoring and material classification module includes:

[0021] For each pixel, the mean, median, and standard deviation within a 3*3 pixel window are calculated. A noise intensity index S is defined. If S exceeds a preset threshold, the pixel is marked as a candidate noise point. If the noise ratio within the window exceeds 50%, a multi-scale sliding window is used for recursive detection until non-noise pixels dominate. The pixel value of the noise point is replaced with the median value within the window. The noise detection threshold is dynamically adjusted based on the type of industrial product.

[0022] According to the adaptive denoising image, the industrial product materials are classified based on texture, color and emission characteristics. According to the results of material classification, corresponding lighting correction, edge sharpening, texture filtering and geometric correction are performed.

[0023] Preferably, the dynamic defect detection and classification module includes:

[0024] Collect annotated industrial product image datasets and divide the dataset into training set, validation set, and training set;

[0025] The industrial product images are preprocessed, including image size adjustment, normalization, and defect sensitivity enhancement. The defect sensitivity enhancement is as follows: the image is defect-sensitively enhanced with a probability of 50%;

[0026] Use a shared convolutional neural network as the backbone network to extract image features. Add a fully connected layer on top of the backbone network to output the predicted product type, and add another fully connected layer on top to output the predicted defect type.

[0027] According to the importance of product types, the weight adjustment rules are determined and the weight of defect detection tasks is dynamically adjusted.

[0028] Preferably, determining the weight adjustment rule includes the following steps:

[0029] Define a weight coefficient for each product type based on its importance:

[0030] High-precision parts: the corresponding defect detection importance is 80%, and the weight is set to 0.8; ordinary parts: the corresponding defect detection importance is 60%, and the weight is set to 0.5; low-precision parts: the corresponding defect detection importance is 40%, and the weight is set to 0.3;

[0031] Dynamically adjust the weight of defect detection tasks based on product type labels;

[0032] Introducing dynamic weight coefficients into the loss function to adjust the weight of the defect detection task according to the product type;

[0033] Use the Adam optimizer, set the learning rate, train the model using the training set, and dynamically adjust the weight value based on the product category label.

[0034] Preferably, the step of saving the trained model as a file and deploying it according to actual needs includes the following steps:

[0035] Save the trained model as a file and select a deployment platform based on actual needs;

[0036] Build an inference service: Write code to load the model and process input data;

[0037] Integrate the inference service into existing production systems, add logging capabilities, and use monitoring tools to monitor system status in real time;

[0038] In actual operation, the model's prediction results and true labels are collected to build a feedback dataset. If the model input does not meet expectations, an error message is returned and logged.

[0039] Preferably, the multi-task model defect assessment module includes:

[0040] A multi-task model based on a convolutional neural network was constructed. The defect region was extracted from the original image using the ROI function. The cropped image region (i.e., the defect region) and the defect type identifier were input. The cropped defect region served as the main input of the model, and the defect type identifier was used to dynamically select the evaluation head. A convolutional layer was set with 3 input channels, 32 output channels, and a convolution kernel size of 3*3. The Swish activation function was used. A maximum pooling layer was set with a pooling window size of 2*2. A residual block was set with 32 input channels and 64 output channels.

[0041] According to different defect types, multiple independent evaluation heads are designed, and each evaluation head outputs the defect severity category of the corresponding defect type;

[0042] Through the feature extraction network, a feature map is obtained. According to the input defect type identifier, the corresponding evaluation head is dynamically selected for calculation: the feature map is globally averaged pooled to obtain the corresponding feature vector, which is input into the corresponding evaluation head to obtain the classification result of the severity of the defect type. The output is normalized using the F.softmax function to obtain the probability distribution.

[0043] Preferably, the scoring fusion classification module includes:

[0044] Design corresponding physical evaluation indicators and calculation methods according to different defect types;

[0045] Construct a physical parameter calculation function: The input is the mask of the defect area and the defect type identifier. Based on the defect type, the corresponding physical parameter calculation method is called to perform the calculation and the calculated physical parameters are stored in a dictionary, where the key is the defect type and the value is the corresponding physical parameter. The defect area is extracted from the original image and the extracted defect area and the corresponding identifier are input into the multi-task model to obtain the defect severity category.

[0046] Set different severity scores based on the defect severity categories of different product categories;

[0047] The results of the multi-task model and the physical parameter calculation function are weighted and integrated to obtain the final score;

[0048] Classify the severity of product defects according to the scoring range, and set corresponding processing methods for the classified results.

[0049] A second aspect of the present invention provides an image recognition and classification method based on machine learning, comprising the following steps:

[0050] S1: Collect industrial product image data through multiple high-definition industrial cameras and annotate the collected industrial product images;

[0051] S2: Detect and remove noise from the collected industrial product images through adaptive methods, and classify the materials of industrial products based on feature extraction;

[0052] S3: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs.

[0053] S4: Extract the defect area through the ROI function and build a feature extraction network. According to different defect types, design multiple independent evaluation heads to output the defect severity category of the corresponding defect type. Use the F.softmax function to normalize the output and obtain the probability distribution of the defect severity.

[0054] S5: Construct a physical parameter calculation function, combine the multi-task model output results, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

[0055] Beneficial effects of the present invention:

[0056] The present invention processes images under different lighting and environments through an adaptive noise detection and removal method, thereby improving robustness, and realizes automatic classification of industrial product materials based on feature extraction material classification, performs targeted preprocessing according to the classification results, and reduces manual intervention.

[0057] The present invention improves the computational efficiency of the model by sharing the CNN backbone network. At the same time, by dynamically adjusting the weights of defect detection tasks, it can optimize the detection performance according to the importance of product types and construct a multi-task model for defect assessment: the defect area is extracted through the ROI function, and the analysis of the defective part is focused on to improve the accuracy of the assessment. The design of multiple independent assessment heads enables the model to perform refined assessments for different defect types, and through the combination of the physical parameter calculation function and the output results of the multi-task model, it realizes the assessment of defects in multiple dimensions, and also improves the accuracy and comprehensiveness of the classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of the method flow of the present invention;

[0059] Figure 2 It is a schematic diagram of the module flow of the present invention;

[0060] Figure 3 This is a flow chart of the dynamic defect detection and classification module of the present invention;

[0061] Figure 4 This is a flow chart of the scoring fusion classification module of the present invention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] See also Figure 1 As shown, the present invention is an image recognition and classification system based on machine learning, which includes the following modules:

[0064] Image acquisition and annotation module: collects industrial product image data through multiple high-definition industrial cameras and annotates the collected industrial product images;

[0065] Adaptive noise monitoring and material classification module: This module detects and removes noise from collected industrial product images through adaptive methods, and classifies the materials of industrial products based on feature extraction.

[0066] Dynamic defect detection and classification module: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs.

[0067] Multi-task model defect assessment module: This module extracts defect areas using the ROI function and constructs a feature extraction network. Based on different defect types, it designs multiple independent assessment heads to output the defect severity category for the corresponding defect type. The F.softmax function is used to normalize the output to obtain the probability distribution of defect severity.

[0068] Scoring fusion classification module: Construct a physical parameter calculation function, combine the output results of the multi-task model, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

[0069] Specifically, use an industrial Ethernet cable to connect the camera to the data acquisition server, assign a unique IP address to each camera, ensure normal network communication, install the acquisition software or driver provided by the camera manufacturer on the data acquisition server, test the image transmission function of each camera, and ensure that the image is displayed on the server in real time; determine the accuracy of the parts according to the quality standards of the industrial products, observe the image, and determine whether there are defects. If there are defects, select the corresponding label according to the defect type, and use the annotation tool to mark the defect location; define the noise intensity index: S = σ-|μ-median|, if S exceeds the preset threshold Tnoise, it is marked as a candidate noise point; if the candidate noise point If the noise ratio in the window of the sound point exceeds 50%, the window size is expanded and the noise intensity index S in the new window is recursively calculated until the non-noise pixels dominate; Tnoise is dynamically adjusted according to the type of industrial product, the grayscale co-occurrence matrix or local binary pattern features of the image are calculated, texture parameters such as contrast, correlation, and energy are extracted, converted to HSV or LAB color space, hue, saturation, and brightness features are extracted, color histograms or color moments are calculated, and machine learning models are used to classify materials. Based on the material classification results, the image is processed in a targeted manner; a shared convolutional neural network is used as the backbone network to extract image features; two fully connected networks are added on top of the backbone network. Layer, one for outputting the product type prediction result, and the other for outputting the defect type prediction result; save the trained model, weights, and configuration files, load the model according to the framework, preprocess the input data, receive the input data, call the model for prediction, and return the result. If the input data does not meet the expectations, return an error message and log it; record the input, output, and exception information of each request, integrate the inference service into the existing system as a microservice or module, and update the model regularly; use the ROI function to extract the defect area from the original image, pair the cropped defect area with the defect type identifier to form a data set; design multiple independent A separate evaluation head is created to convert the dataset into a format suitable for model input, and the model is trained using the fit method. The trained model is loaded, the prediction results are normalized using F.softmax to obtain the probability distribution of defect severity. For different defect types, corresponding physical evaluation indicators are designed, and a calculation formula or algorithm is designed for each physical indicator. The mask of the defect area and the defect type identifier are input, and the corresponding calculation method is called according to the defect type, and the calculated physical parameters are stored in a dictionary. Weights are set for the physical parameters and model results, and then weighted fusion is performed to obtain the final score. The scoring interval is defined for classification, and the corresponding processing method is set according to the classification results.

[0070] In one embodiment of the present invention, the image acquisition and annotation module includes:

[0071] Install multiple high-definition industrial cameras on the industrial production line to shoot industrial products from four angles: front, side, top and bottom;

[0072] Transmit image data from multiple cameras to the data acquisition server in real time via industrial Ethernet;

[0073] Use professional image annotation software to annotate images. The annotation information includes: industrial product type information, defect information, key features and other information.

[0074] Specifically, according to the product characteristics and inspection requirements, determine the installation position and angle of the camera. The front is used to capture defects on the front of the product, the side is used to capture defects on the side of the product, the top is used to capture defects on the top of the product, and the bottom is used to capture defects on the bottom of the product. Ensure that the camera's field of view covers the entire area of ​​the product to avoid obstructions or blind spots; use a bracket or fixture to install the camera in the predetermined position, adjust the camera's angle and focal length to ensure that the captured image is clear and without distortion, number each camera, set the camera resolution, frame rate, exposure time and other parameters to ensure that the image quality meets the inspection requirements; use an industrial Ethernet cable to connect the camera to the data acquisition server, assign a unique IP address to each camera, ensure normal network communication, install the acquisition software or driver provided by the camera manufacturer on the data acquisition server, test the image transmission function of each camera, and ensure that the image is displayed on the server in real time; select professional image annotation software, annotate the product category, annotate the type and location of the defect, and annotate the key features of the product, classify and store the annotation files by camera angle and product category, and store the annotated images and annotation files on the data acquisition server.

[0075] In one embodiment of the present invention, the step of annotating an image using professional image annotation software includes the following steps:

[0076] Industrial product type information: Assign a unique label to each industrial product type, 0 for high-precision parts, 1 for ordinary parts, and 2 for low-precision parts;

[0077] Defect information: Mark the type, code, and location of the defect, and assign a unique label to each defect type: A for no defect, B for scratches, C for dents, D for cracks, E for rust, F for deformation, G for bubbles, and H for color difference;

[0078] Additional information: production batch, line number, and timestamp.

[0079] Specifically, the accuracy of parts is determined according to the quality standards of industrial products. High-precision parts are those that meet international standards, such as ISO 2768-mK, AS9100, and GJB9001; ordinary parts are those that meet general industry standards, such as ISO 2768-c and GB / T; low-precision parts are those that meet basic standards, such as ISO 2768-u and enterprise standards; observe the image to determine whether there are defects. If there are defects, select the corresponding label according to the defect type, for example, if the defect is a scratch, mark it as B, if there is no defect, mark it as A; use annotation tools to mark the defect location; obtain the current batch number from the production management system, obtain the production line number based on the camera position or production environment, and record the image acquisition time in the format of YYYY-MM-DDHH:MM:SS.

[0080] In one embodiment of the present invention, the adaptive noise monitoring and material classification module includes:

[0081] For each pixel, the mean, median, and standard deviation within a 3*3 pixel window are calculated. A noise intensity index S is defined. If S exceeds a preset threshold, the pixel is marked as a candidate noise point. If the noise ratio within the window exceeds 50%, a multi-scale sliding window is used for recursive detection until non-noise pixels dominate. The pixel value of the noise point is replaced with the median value within the window. The noise detection threshold is dynamically adjusted based on the type of industrial product.

[0082] According to the adaptive denoising image, the industrial product materials are classified based on texture, color and emission characteristics. According to the results of material classification, corresponding lighting correction, edge sharpening, texture filtering and geometric correction are performed.

[0083] Specifically, for each pixel point, a 3*3 window is taken with it as the center, and the mean μ, median median and standard deviation σ of the pixels in the window are calculated. The noise intensity index is defined as: S = σ-|μ-median|. If S exceeds the preset threshold Tnoise, it is marked as a candidate noise point; if the noise ratio in the window of the candidate noise point exceeds 50%, the window size is expanded, such as 5*5 or 7*7, and the noise intensity index S in the new window is recursively calculated until the non-noise pixels dominate; and Tnoise is dynamically adjusted according to the type of industrial product: for high-precision parts, Tnoise is set to a lower threshold, such as 10, for Strict denoising; for ordinary parts, Tnoise sets a medium threshold, such as 20; for low-precision parts, Tnoise sets a higher threshold and performs ordinary denoising; calculate the grayscale co-occurrence matrix or local binary pattern features of the image, extract texture parameters such as contrast, correlation, and energy, convert to HSV or LAB color space, extract hue, saturation, and brightness features, and calculate the color histogram or color moment; if it is a metal material, calculate the reflectivity or glossiness; if it is a plastic or composite material, analyze the surface gloss and texture regularity; use a machine learning model to classify the material, and perform targeted image processing based on the material classification results.

[0084] In one embodiment of the present invention, the dynamic defect detection and classification module includes:

[0085] Collect annotated industrial product image datasets and divide the dataset into training set, validation set, and training set;

[0086] The industrial product images are preprocessed, including image size adjustment, normalization, and defect sensitivity enhancement. The defect sensitivity enhancement is as follows: the image is defect-sensitively enhanced with a probability of 50%;

[0087] Use a shared convolutional neural network as the backbone network to extract image features. Add a fully connected layer on top of the backbone network to output the predicted product type, and add another fully connected layer on top to output the predicted defect type.

[0088] According to the importance of product types, the weight adjustment rules are determined and the weight of defect detection tasks is dynamically adjusted.

[0089] Specifically, ensure that each image contains the following annotation information, including product type and defect type, and divide the dataset into training set, validation set and test set in proportion; uniformly resize all images to a fixed size, such as 224*224 or 256*256, normalize the pixel values ​​to the range of [0, 1], and perform defect sensitivity enhancement on the images with a probability of 50%: random brightness adjustment, such as ±10%, random contrast adjustment, such as ±10%; use a shared convolutional neural network as the backbone network to extract image features; add two fully connected layers on top of the backbone network, one for outputting product type prediction results, and the other for outputting defect type prediction results; define the loss function and optimizer: both product type classification loss and defect type classification loss are: categorical_crossentropy, and the optimizer is: Adam; dynamically adjust the weight of the defect detection task according to the importance of the product type.

[0090] In one embodiment of the present invention, determining the weight adjustment rule includes the following steps:

[0091] Define a weight coefficient for each product type based on its importance:

[0092] High-precision parts: the corresponding defect detection importance is 80%, and the weight is set to 0.8; ordinary parts: the corresponding defect detection importance is 60%, and the weight is set to 0.5; low-precision parts: the corresponding defect detection importance is 40%, and the weight is set to 0.3;

[0093] Dynamically adjust the weight of defect detection tasks based on product type labels;

[0094] Introducing dynamic weight coefficients into the loss function to adjust the weight of the defect detection task according to the product type;

[0095] Use the Adam optimizer, set the learning rate, train the model using the training set, and dynamically adjust the weight value based on the product category label.

[0096] Specifically, the weight adjustment rule is: High-precision parts: defect detection weight W defect =0.8; Ordinary parts: defect detection weight W defect =0.5; low-precision parts: defect detection weight W defect = 0.3; map the product type label to the corresponding weight value; the total loss consists of two parts, product type classification loss and defect detection loss. The weight of the defect detection loss is dynamically adjusted according to the product type. The total loss formula is:

[0097] L total =L product +W defect ×Ldefect

[0098] Among them, L total is the total loss, L product is the product type classification loss, L defect is the defect detection loss, W defect is the corresponding weight coefficient;

[0099] During training, the weight of the defect detection loss is dynamically adjusted according to the product type label of each sample.

[0100] In one embodiment of the present invention, the trained model is saved as a file and deployed according to actual needs, including the following steps:

[0101] Save the trained model as a file and select a deployment platform based on actual needs;

[0102] Build an inference service: Write code to load the model and process input data;

[0103] Integrate the inference service into existing production systems, add logging capabilities, and use monitoring tools to monitor system status in real time;

[0104] In actual operation, the model's prediction results and true labels are collected to build a feedback dataset. If the model input does not meet expectations, an error message is returned and logged.

[0105] Specifically, save the trained model, weights, and configuration files, load the model according to the framework, preprocess the input data, receive input data, call the model for prediction, and return the result. If the input data does not meet expectations, return an error message and log it; record the input, output, and exception information of each request, and integrate the inference service into the existing system as a microservice or module; and use Prometheus, Grafana, or ELK Stack to monitor the system status, start Prometheus and Grafana, configure the monitoring panel, and trigger an alarm when the error rate or response time exceeds the threshold; record the results and true labels of each prediction in the inference service, regularly extract feedback data from the log or database, use the feedback data set to update the model, and replace the old model.

[0106] In one embodiment of the present invention, the multi-task model defect assessment module includes:

[0107] A multi-task model based on a convolutional neural network was constructed. The defect region was extracted from the original image using the ROI function. The cropped image region (i.e., the defect region) and the defect type identifier were input. The cropped defect region served as the main input of the model, and the defect type identifier was used to dynamically select the evaluation head. A convolutional layer was set with 3 input channels, 32 output channels, and a convolution kernel size of 3*3. The Swish activation function was used. A maximum pooling layer was set with a pooling window size of 2*2. A residual block was set with 32 input channels and 64 output channels.

[0108] According to different defect types, multiple independent evaluation heads are designed, and each evaluation head outputs the defect severity category of the corresponding defect type;

[0109] Through the feature extraction network, a feature map is obtained. According to the input defect type identifier, the corresponding evaluation head is dynamically selected for calculation: the feature map is globally averaged pooled to obtain the corresponding feature vector, which is input into the corresponding evaluation head to obtain the classification result of the severity of the defect type. The output is normalized using the F.softmax function to obtain the probability distribution.

[0110] Specifically, the ROI function is used to extract the defect area from the original image, and the cropped defect area is paired with the defect type identifier to form a dataset; the necessary libraries are imported to define the feature extraction network, including the convolution layer, Swish activation function, maximum pooling layer and residual block, and the residual block input channel is defined as 32 and the output channel is defined as 64; multiple independent evaluation heads are designed according to the defect type, and the appropriate loss function and optimizer are used to convert the dataset into a format suitable for model input, and the model is trained using the fit method; the trained model is loaded, the cropped defect area and defect type identifier are input, the prediction results are obtained, and the output is normalized using F.softmax to obtain the probability distribution.

[0111] In one embodiment of the present invention, the scoring fusion classification module includes:

[0112] Design corresponding physical evaluation indicators and calculation methods according to different defect types;

[0113] Construct a physical parameter calculation function: The input is the mask of the defect area and the defect type identifier. Based on the defect type, the corresponding physical parameter calculation method is called to perform the calculation and the calculated physical parameters are stored in a dictionary, where the key is the defect type and the value is the corresponding physical parameter. The defect area is extracted from the original image and the extracted defect area and the corresponding identifier are input into the multi-task model to obtain the defect severity category.

[0114] Set different severity scores based on the defect severity categories of different product categories;

[0115] The results of the multi-task model and the physical parameter calculation function are weighted and integrated to obtain the final score;

[0116] Classify the severity of product defects according to the scoring range, and set corresponding processing methods for the classified results.

[0117] Specifically, for different defect types, corresponding physical evaluation indicators are designed, and a calculation formula or algorithm is designed for each physical indicator. For example, for scratches, the length and width are calculated by the pixel distance of the mask; for dents, the diameter and depth are calculated by grayscale value or height map; and for cracks, the length and width are calculated by topological analysis. The mask (binary image) of the defect area and the defect type identifier are input, and the corresponding calculation method is called according to the defect type, and the calculated physical parameters are stored in a dictionary. The defect severity category is obtained from the output results of the above multi-task model, and different scores are set according to the product category and defect severity category. Weights are set for the physical parameters and model results, with weights of 0.4 and 0.6 respectively, and the physical parameters and model scores are weighted and fused to obtain the final score. The scoring range is defined for classification, and the corresponding processing method is set according to the classification result. For example, if the final score is between 0 and 35 points, the defect has little impact on the product, and processing is performed according to different product categories. If the final score is between 36 and 65 points, the defect has a certain degree of impact on the product, and the defect is repaired manually or automatically according to different product categories, or returned to the production line for reprocessing or adjustment.

[0118] See also Figure 2 As shown, the present invention is an image recognition and classification method based on machine learning, comprising the following steps:

[0119] S1: Collect industrial product image data through multiple high-definition industrial cameras and annotate the collected industrial product images;

[0120] S2: Detect and remove noise from the collected industrial product images through adaptive methods, and classify the materials of industrial products based on feature extraction;

[0121] S3: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs.

[0122] S4: Extract the defect area through the ROI function and build a feature extraction network. According to different defect types, design multiple independent evaluation heads to output the defect severity category of the corresponding defect type. Use the F.softmax function to normalize the output and obtain the probability distribution of the defect severity.

[0123] S5: Construct a physical parameter calculation function, combine the multi-task model output results, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

[0124] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An image recognition and classification system based on machine learning, characterized in that: Includes the following modules: Image acquisition and annotation module: collects industrial product image data through multiple high-definition industrial cameras and annotates the collected industrial product images; Adaptive noise monitoring and material classification module: This module detects and removes noise from collected industrial product images through adaptive methods, and classifies the materials of industrial products based on feature extraction. Dynamic defect detection and classification module: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs. Multi-task model defect assessment module: This module extracts defect areas using the ROI function and constructs a feature extraction network. Based on different defect types, it designs multiple independent assessment heads to output the defect severity category for the corresponding defect type. The F.softmax function is used to normalize the output to obtain the probability distribution of defect severity. Scoring fusion classification module: Construct a physical parameter calculation function, combine the output results of the multi-task model, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

2. The image recognition and classification system based on machine learning according to claim 1, characterized in that: The image acquisition and annotation module includes: Install multiple high-definition industrial cameras on the industrial production line to shoot industrial products from four angles: front, side, top and bottom; Transmit image data from multiple cameras to the data acquisition server in real time via industrial Ethernet; Use professional image annotation software to annotate images. The annotation information includes: industrial product type information, defect information, key features and other information.

3. The image recognition and classification system based on machine learning according to claim 2, characterized in that: The image annotation using professional image annotation software includes: Industrial product type information: Assign a unique label to each industrial product type, 0 for high-precision parts, 1 for ordinary parts, and 2 for low-precision parts; Defect information: Mark the type, code, and location of the defect, and assign a unique label to each defect type: A for no defect, B for scratches, C for dents, D for cracks, E for rust, F for deformation, G for bubbles, and H for color difference; Additional information: production batch, line number, and timestamp.

4. The image recognition and classification system based on machine learning according to claim 1, characterized in that: The adaptive noise monitoring and material classification module includes: For each pixel, the mean, median, and standard deviation within a 3*3 pixel window are calculated. A noise intensity index S is defined. If S exceeds a preset threshold, the pixel is marked as a candidate noise point. If the noise ratio within the window exceeds 50%, a multi-scale sliding window is used for recursive detection until non-noise pixels dominate. The pixel value of the noise point is replaced with the median value within the window. The noise detection threshold is dynamically adjusted based on the type of industrial product. According to the adaptive denoising image, the industrial product materials are classified based on texture, color and emission characteristics. According to the results of material classification, corresponding lighting correction, edge sharpening, texture filtering and geometric correction are performed.

5. The image recognition and classification system based on machine learning according to claim 1, characterized in that: The dynamic defect detection and classification module includes: Collect annotated industrial product image datasets and divide the dataset into training set, validation set, and training set; The industrial product images are preprocessed, including image size adjustment, normalization, and defect sensitivity enhancement. The defect sensitivity enhancement is as follows: the image is defect-sensitively enhanced with a probability of 50%; Use a shared convolutional neural network as the backbone network to extract image features. Add a fully connected layer on top of the backbone network to output the predicted product type, and add another fully connected layer on top to output the predicted defect type. According to the importance of product types, the weight adjustment rules are determined and the weight of defect detection tasks is dynamically adjusted.

6. The image recognition and classification system based on machine learning according to claim 5, characterized in that: Determining the weight adjustment rule includes the following steps: Define a weight coefficient for each product type based on its importance: High-precision parts: the corresponding defect detection importance is 80%, and the weight is set to 0.8; ordinary parts: the corresponding defect detection importance is 60%, and the weight is set to 0.5; low-precision parts: the corresponding defect detection importance is 40%, and the weight is set to 0.3; Dynamically adjust the weight of defect detection tasks based on product type labels; Introducing dynamic weight coefficients into the loss function to adjust the weight of the defect detection task according to the product type; Use the Adam optimizer, set the learning rate, train the model using the training set, and dynamically adjust the weight value based on the product category label.

7. The image recognition and classification system based on machine learning according to claim 1, characterized in that: Saving the trained model as a file and deploying it according to actual needs includes the following steps: Save the trained model as a file and select a deployment platform based on actual needs; Build an inference service: Write code to load the model and process input data; Integrate the inference service into existing production systems, add logging capabilities, and use monitoring tools to monitor system status in real time; In actual operation, the model's prediction results and true labels are collected to build a feedback dataset. If the model input does not meet expectations, an error message is returned and logged.

8. The image recognition and classification system based on machine learning according to claim 1, characterized in that: The multi-task model defect assessment module includes: A multi-task model based on a convolutional neural network was constructed. The defect region was extracted from the original image using the ROI function. The cropped image region (i.e., the defect region) and the defect type identifier were input. The cropped defect region served as the main input of the model, and the defect type identifier was used to dynamically select the evaluation head. A convolutional layer was set with 3 input channels, 32 output channels, and a convolution kernel size of 3*3. The Swish activation function was used. A maximum pooling layer was set with a pooling window size of 2*2. A residual block was set with 32 input channels and 64 output channels. According to different defect types, multiple independent evaluation heads are designed, and each evaluation head outputs the defect severity category of the corresponding defect type; Through the feature extraction network, a feature map is obtained. According to the input defect type identifier, the corresponding evaluation head is dynamically selected for calculation: the feature map is globally averaged pooled to obtain the corresponding feature vector, which is input into the corresponding evaluation head to obtain the classification result of the severity of the defect type. The output is normalized using the F.softmax function to obtain the probability distribution.

9. The image recognition and classification system based on machine learning according to claim 1, characterized in that: The scoring fusion classification module includes: Design corresponding physical evaluation indicators and calculation methods according to different defect types; Construct a physical parameter calculation function: The input is the mask of the defect area and the defect type identifier. Based on the defect type, the corresponding physical parameter calculation method is called to perform the calculation and the calculated physical parameters are stored in a dictionary, where the key is the defect type and the value is the corresponding physical parameter. The defect area is extracted from the original image and the extracted defect area and the corresponding identifier are input into the multi-task model to obtain the defect severity category. Set different severity scores based on the defect severity categories of different product categories; The results of the multi-task model and the physical parameter calculation function are weighted and integrated to obtain the final score; Classify the severity of product defects according to the scoring range, and set corresponding processing methods for the classified results.

10. An image recognition and classification method based on machine learning, characterized in that: The following steps are involved: S1: Collect industrial product image data through multiple high-definition industrial cameras and annotate the collected industrial product images; S2: Detect and remove noise from the collected industrial product images through adaptive methods, and classify the materials of industrial products based on feature extraction; S3: By sharing the CNN backbone network, two fully connected layers are added to output prediction results for product type and defect type. The weight of the defect detection task is dynamically adjusted based on the importance of the product type. The trained model is saved as a file and deployed according to actual needs. S4: Extract the defect area through the ROI function and build a feature extraction network. According to different defect types, design multiple independent evaluation heads to output the defect severity category of the corresponding defect type. Use the F.softmax function to normalize the output and obtain the probability distribution of the defect severity. S5: Construct a physical parameter calculation function, combine the multi-task model output results, set the corresponding severity score, perform weighted fusion of the function calculation and the model output results to obtain the final score, and classify the severity of product defects according to the score range.

Images