Image Semantics-Based Product Appearance Defect Detection System and Method

Abstract

This invention discloses a product appearance defect detection system and method based on image semantics, belonging to the field of defect detection technology. The system includes: an image acquisition module for acquiring real-time images of the product appearance; an image processing module for processing the real-time images of the product appearance; an image semantics module for extracting features and performing semantic segmentation on the real-time images of the product appearance to generate a semantic image of the product appearance; and a defect detection module for detecting defects in the product appearance, determining whether defects exist, and providing early warnings and visual displays of defect conditions. This invention solves the problem of existing methods that cannot effectively detect product appearance defects based on image semantics, resulting in poor defect detection performance. This invention can effectively detect product appearance defects based on image semantics and can provide timely early warnings based on defect detection results, thus improving the overall defect detection effect.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, specifically to a product appearance defect detection system and method based on image semantics. Background Technology

[0002] Product appearance defect detection refers to the use of a series of technical means to detect whether there are defects or undesirable phenomena on the surface of a product. These defects may cause the product to fail to meet the reasonable safety requirements necessary for consumption or use. Therefore, it is necessary to detect defects in the product appearance.

[0003] Chinese Patent Publication No. CN116883370B discloses an agricultural product appearance quality inspection system, including an image filtering module for filtering the grayscale image corresponding to the appearance image of the agricultural product. The system calculates the filter contrast value of pixels in the grayscale image; obtains pixels belonging to noise in the grayscale image based on the filter contrast value; calculates the range parameter of the noise pixels; and performs median filtering on the noise pixels based on the range parameter to obtain a filtered image. This allows the filtering range to automatically change with the attributes of the noise pixels, significantly reducing the probability of overly large pixel values ​​after filtering. This facilitates obtaining more accurate median filtering results, improving the accuracy of the obtained image features, and making the appearance quality inspection results more accurate. However, this patent has the following drawbacks:

[0004] Existing technologies cannot effectively detect product appearance defects based on image semantics, nor can they provide timely warnings based on the detected product appearance defects, resulting in poor product appearance defect detection performance. Summary of the Invention

[0005] The purpose of this invention is to provide a product appearance defect detection system and method based on image semantics, which can effectively detect product appearance defects based on image semantics and provide timely warnings based on the product appearance defect detection results, thereby improving the product appearance defect detection effect and solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] Image semantics-based product appearance defect detection system includes:

[0008] Image acquisition module, used to acquire real-time images of product appearance based on computer vision;

[0009] The image processing module is used to process real-time images of product appearance based on computer vision.

[0010] The image semantic module is used to extract features and perform semantic segmentation on real-time images of product appearance to generate semantic images of product appearance.

[0011] The defect detection module is used to detect defects in the product appearance based on machine learning technology, determine whether there are defects in the product appearance, and provide early warnings and visual displays of the defects.

[0012] Preferably, the image acquisition module includes:

[0013] The product fixing unit is used to fix the product to be tested.

[0014] Place the product to be inspected on the product appearance defect inspection station and fix it in place to ensure stability.

[0015] The acquisition and adjustment unit is used to adjust the acquisition angle and light intensity of the product to be tested;

[0016] When acquiring images of the product to be inspected, the acquisition angle and light intensity of the product to be inspected are adjusted based on the intelligent adjustment device;

[0017] The product acquisition unit is used to acquire real-time images of the product's appearance.

[0018] Based on computer vision technology, machine vision equipment is used to monitor and continuously collect images of the product under inspection in real time, and to obtain product images of the product under inspection under different acquisition angles and lighting conditions, thereby determining the real-time image of the product appearance based on computer vision.

[0019] Preferably, the acquisition and adjustment unit further performs the following steps:

[0020] Extract the initial light intensity value for each element in the initial light intensity group for the product;

[0021] After the test product is fixed, the test product is photographed according to each initial light intensity value to obtain the product test image corresponding to each initial light intensity value;

[0022] The grayscale value of the pixels in the product test image corresponding to each initial light intensity value is compared with a preset grayscale threshold to obtain the area of ​​the target region formed by pixels whose grayscale value exceeds the preset grayscale threshold.

[0023] The intensity modulation coefficient is obtained by using the target area of ​​the product image corresponding to each initial illumination intensity value;

[0024] The light intensity modulation coefficient is obtained by the following formula:

[0025]

[0026] Where L represents the light intensity modulation factor; n represents the number of initial light intensity values ​​included in the initial light intensity group; B i B represents the i-th initial illumination intensity value contained in the initial illumination intensity group; i+1 S represents the (i+1)th initial illumination intensity value contained in the initial illumination intensity group; i S represents the area of ​​the target region of the product test image corresponding to the i-th initial illumination intensity value in the initial illumination intensity group; i+1 B represents the area of ​​the target region of the product test image corresponding to the (i+1)th initial illumination intensity value in the initial illumination intensity group; b S represents the standard deviation of the n initial light intensity values ​​contained in the initial light intensity group; b This represents the standard deviation of the target area of ​​the product test image corresponding to the n initial light intensity values ​​contained in the initial light intensity group;

[0027] The light intensity of the image acquisition of the product to be tested is adjusted using the light intensity modulation coefficient.

[0028] Preferably, the intensity modulation coefficient adjusts the illumination intensity of the image acquisition of the product to be tested, including:

[0029] The initial light intensity value corresponding to the minimum area of ​​the target region in the product test image is retrieved as the first light intensity.

[0030] The initial light intensity value corresponding to the maximum area of ​​the target region in the product test image is retrieved as the second light intensity.

[0031] A reference value for illumination intensity is obtained using the first and second illumination intensities.

[0032] The reference value for light intensity is obtained by the following formula:

[0033]

[0034] Among them, B c B represents the reference value for light intensity. p B represents the average value of the initial illumination intensities contained in the initial illumination intensity group; min Indicates the first light intensity; B max Indicates the second light intensity; n represents the number of initial light intensity values ​​included in the initial light intensity group; H zi H represents the median grayscale value corresponding to the area of ​​the target region corresponding to the i-th initial illumination intensity value; ki This represents the center grayscale value of the product test image corresponding to the i-th initial illumination intensity value;

[0035] Retrieve the light intensity modulation coefficient;

[0036] The light intensity of the product to be tested is obtained using the light intensity modulation coefficient and the light intensity reference value.

[0037] The light intensity is obtained by the following formula:

[0038]

[0039] Among them, B t Indicates light intensity; B c represents the baseline value of light intensity; L represents the light intensity adjustment factor.

[0040] Preferably, the image processing module includes:

[0041] The image denoising unit is used to denoise real-time images of product appearance based on computer vision.

[0042] Acquire real-time images of product appearance based on computer vision, convert the real-time images of product appearance based on computer vision into grayscale images, and perform noise reduction processing on the real-time images of product appearance based on computer vision using a median filter.

[0043] Specifically, the window size of the median filter is set based on the noise type and intensity, as well as the image details and size, in the real-time image of the product appearance based on computer vision.

[0044] When the noise in the real-time image of a product appearance based on computer vision is salt-and-pepper noise or random noise, and the noise intensity is high, a large window is used; when the noise intensity is low, a small window is used.

[0045] When an image contains important details or edge information, use a small window; when the image size is large, use a small window or a block-based processing method.

[0046] After setting the window size of the median filter, for each pixel in the real-time image of the product appearance based on computer vision, the median of the gray values ​​of all pixels in the neighborhood of the surrounding window size is calculated, and the gray value of the pixel is replaced with the median. This is used to remove or reduce noise in the real-time image of the product appearance based on computer vision, and the denoised real-time image of the product appearance is saved or displayed.

[0047] Preferably, the image processing module further includes:

[0048] The image adjustment unit is used to adjust the brightness and contrast of the real-time image of the product appearance after noise reduction.

[0049] Brightness adjustment: Obtain the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising. Increase the brightness by increasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising, and decrease the brightness by decreasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising.

[0050] Contrast Adjustment: Based on histogram equalization, the contrast of the denoised real-time product appearance image is adjusted. By redistributing the gray values ​​of the real-time product appearance image, the gray distribution of the real-time product appearance image is adjusted to a uniform distribution, increasing the dynamic range of pixel gray values, thereby adjusting the contrast of the real-time product appearance image.

[0051] Preferably, the image semantic module includes:

[0052] The extraction and matching unit is used to extract and match features from the processed real-time image of the product appearance.

[0053] Based on a convolutional neural network deep learning model, feature extraction is performed on the processed real-time product appearance image. The features of the real-time product appearance image are automatically learned, and highly discriminative feature vectors are extracted, including shape features, texture features, and color features. A similarity algorithm is used to match the extracted feature vectors with features in an existing feature library.

[0054] The semantic segmentation unit is used to perform semantic segmentation on real-time product appearance images, dividing the real-time product appearance images into different semantic regions. Each semantic region has the same category label. The extracted feature vectors and semantic segmentation results are fused together, and the feature vectors and semantic labels are concatenated to generate a unified product appearance semantic image.

[0055] Preferably, the defect detection module includes:

[0056] The model training unit is used to train a product appearance defect detection model based on image semantics.

[0057] Based on the requirements for product appearance defect detection using image semantics, collect historical images of product appearance.

[0058] The collected historical images of product appearance are divided to determine the training set and the test set;

[0059] Based on the training set, the machine learning model is trained and iterated to enable the machine learning model to automatically learn product appearance defect detection and determine the product appearance defect detection model based on image semantics.

[0060] Based on the test set, the performance of the image semantic-based product appearance defect detection model is tested to determine whether the image semantic-based product appearance defect detection model can achieve the expected results and to determine the optimal image semantic-based product appearance defect detection model.

[0061] Preferably, the defect detection module further includes:

[0062] The defect detection unit is used to detect defects in the product's appearance.

[0063] Obtain the optimal product appearance defect detection model based on image semantics;

[0064] Deploy the optimal image semantic-based product appearance defect detection model in a real product appearance defect detection environment;

[0065] Based on the optimal image semantics-based product appearance defect detection model, image semantics recognition is performed on the product appearance semantic image to check whether there are defects in the product appearance and to determine the product appearance defect detection result based on image semantics.

[0066] The early warning display unit is used to provide early warnings and visual displays of product appearance defects.

[0067] Obtain product appearance defect detection results based on image semantics;

[0068] When a defect is detected in the product's appearance, an early warning alarm will be automatically issued to notify the management personnel in a timely manner.

[0069] Based on product information, defects in product appearance are marked to generate a product appearance defect inspection report, which is then presented to managers in a visual format.

[0070] According to another aspect of the present invention, a product appearance defect detection method based on image semantics is provided, implemented based on the image semantics-based product appearance defect detection system as described above, comprising the following steps:

[0071] S1: Place the product to be inspected on the product appearance defect inspection station. Based on computer vision technology, the product to be inspected is collected in real time under different acquisition angles and light intensity conditions to determine the real-time image of the product appearance based on computer vision.

[0072] S2: Based on the median filter, perform noise reduction processing on the real-time product appearance image based on computer vision, remove or reduce noise in the real-time product appearance image based on computer vision, and save or display the denoised real-time product appearance image.

[0073] S3: Adjust the brightness of the real-time product appearance image by increasing or decreasing the grayscale value of all pixels in the real-time product appearance image, and adjust the contrast of the real-time product appearance image by increasing the dynamic range of pixel grayscale values ​​based on histogram equalization.

[0074] S4: Based on a convolutional neural network deep learning model, feature extraction is performed on real-time product appearance images to extract highly discriminative feature vectors. A similarity algorithm is then used to match the extracted feature vectors with features in an existing feature library. Semantic segmentation is performed on the real-time product appearance images, and the extracted feature vectors and semantic segmentation results are fused to generate a semantic image of the product appearance.

[0075] S5: Based on the optimal image semantic-based product appearance defect detection model, perform image semantic recognition on the product appearance semantic image to determine whether there are defects in the product appearance. When defects are detected, an early warning alarm is automatically issued to notify the management personnel in a timely manner. Based on the product information, the product appearance defect is marked and a product appearance defect detection report is generated. The product appearance defect detection report is presented to the management personnel in a visual form.

[0076] Compared with the prior art, the beneficial effects of the present invention are:

[0077] 1. This invention, based on computer vision technology, performs real-time acquisition of the product to be inspected under different acquisition angles and lighting intensities to determine a real-time image of the product's appearance based on computer vision. A median filter is used to denoise the real-time image, removing or reducing noise. The brightness and contrast of the denoised image are then adjusted. A convolutional neural network deep learning model is used to extract features from the real-time image, extracting highly discriminative feature vectors. A similarity algorithm is used to match the extracted feature vectors with features in an existing feature library. Semantic segmentation is then performed on the real-time image, and the extracted feature vectors and semantic segmentation results are fused to generate a semantic image of the product's appearance, facilitating effective defect detection in subsequent product appearance inspections.

[0078] 2. This invention uses an optimal image semantic-based product appearance defect detection model to perform image semantic recognition on product appearance semantic images to determine whether there are defects in the product appearance. When a defect is detected, an early warning alarm is automatically issued to promptly notify management personnel. Based on product information, the defect is marked to generate a product appearance defect detection report, which is presented to management personnel in a visual format. This facilitates timely management of products with appearance defects. It can effectively detect product appearance defects based on image semantics and provide timely early warnings based on the detected defects, thereby improving the product appearance defect detection effect. Attached Figure Description

[0079] Figure 1 This is a block diagram of the product appearance defect detection system based on image semantics of the present invention.

[0080] Figure 2 This is a flowchart illustrating the operation of the product appearance defect detection method based on image semantics according to the present invention. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] To address the shortcomings of existing technologies, which cannot effectively detect product appearance defects based on image semantics and cannot provide timely warnings based on defect detection results, leading to poor product appearance defect detection performance, please refer to [link to relevant documentation]. Figures 1-2 This embodiment provides the following technical solution:

[0083] Example 1

[0084] The product appearance defect detection system based on image semantics includes: an image acquisition module, an image processing module, an image semantics module, and a defect detection module.

[0085] It should be noted that through the interactive communication between the image acquisition module, image processing module, image semantic module, and defect detection module, defects in the product appearance can be effectively detected based on image semantics, and timely warnings can be issued based on the defect detection results, thereby improving the product appearance defect detection effect.

[0086] Specifically, the image acquisition module acquires real-time images of the product under different acquisition angles and lighting conditions to determine the real-time image of the product appearance. The image processing module processes the real-time image of the product appearance to remove noise and adjust the brightness and contrast. The image semantic module extracts features and performs semantic segmentation on the real-time image of the product appearance to generate a semantic image of the product appearance. The defect detection module detects defects in the product appearance to determine whether there are defects, and provides early warnings and visual displays of the defects.

[0087] In this embodiment, as a preferred technical solution of the present invention, the image acquisition module includes:

[0088] The product fixing unit is used to fix the product to be tested.

[0089] Place the product to be inspected on the product appearance defect inspection station and fix it in place to make it stable and facilitate the collection of the product's appearance data.

[0090] The acquisition and adjustment unit is used to adjust the acquisition angle and light intensity of the product to be tested;

[0091] When acquiring images of the product to be inspected, the acquisition angle and light intensity of the product to be inspected are adjusted based on the intelligent adjustment device, so as to acquire appearance images of the product to be inspected under different acquisition angles and light intensities.

[0092] The product acquisition unit is used to acquire real-time images of the product's appearance.

[0093] Based on computer vision technology, machine vision equipment is used to monitor and continuously collect images of the product under inspection in real time, and to obtain product images of the product under inspection under different acquisition angles and lighting conditions, thereby determining the real-time image of the product appearance based on computer vision.

[0094] It should be noted that computer vision refers to using cameras and computers to replace human eyes for target recognition, tracking, and measurement, and further processing the images to make them more suitable for human observation or transmission to instruments for detection. Machine vision, on the other hand, uses machines to replace human eyes for measurement and judgment. A machine vision system uses machine vision products to convert the captured target into image signals, which are then transmitted to a dedicated image processing system to obtain the target's shape information. Based on pixel distribution, brightness, color, and other information, the image system converts these signals into digital signals. The image system then performs various calculations on these signals to extract the target's features, and then controls the on-site equipment based on the judgment results.

[0095] Therefore, based on computer vision technology, machine vision equipment can be used to monitor and continuously collect data on the product under inspection in real time, thereby obtaining real-time images of the product's appearance under different acquisition angles and lighting conditions, providing data support for the detection of product appearance defects.

[0096] Specifically, the acquisition and adjustment unit also performs the following steps:

[0097] Extract the initial light intensity value for each element in the initial light intensity group for the product;

[0098] After the test product is fixed, the test product is photographed according to each initial light intensity value to obtain the product test image corresponding to each initial light intensity value;

[0099] The grayscale value of the pixels in the product test image corresponding to each initial light intensity value is compared with a preset grayscale threshold to obtain the area of ​​the target region formed by pixels whose grayscale value exceeds the preset grayscale threshold.

[0100] The intensity modulation coefficient is obtained by using the target area of ​​the product image corresponding to each initial illumination intensity value;

[0101] The light intensity modulation coefficient is obtained by the following formula:

[0102]

[0103] Where L represents the light intensity modulation factor; n represents the number of initial light intensity values ​​included in the initial light intensity group; B i B represents the i-th initial illumination intensity value contained in the initial illumination intensity group; i+1 S represents the (i+1)th initial illumination intensity value contained in the initial illumination intensity group; i S represents the area of ​​the target region of the product test image corresponding to the i-th initial illumination intensity value in the initial illumination intensity group; i+1 B represents the area of ​​the target region of the product test image corresponding to the (i+1)th initial illumination intensity value in the initial illumination intensity group; b S represents the standard deviation of the n initial light intensity values ​​contained in the initial light intensity group; b This represents the standard deviation of the target area of ​​the product test image corresponding to the n initial light intensity values ​​contained in the initial light intensity group;

[0104] The light intensity of the image acquisition of the product to be tested is adjusted using the light intensity modulation coefficient.

[0105] The technical effects of the above solution are as follows: By setting a set of initial illumination intensities and photographing the product under each intensity, product images under different illumination conditions can be obtained. This method can systematically explore the impact of illumination intensity on product image quality. By comparing the grayscale values ​​of pixels with preset grayscale thresholds, the size of specific areas (such as overly bright or overly dark areas) in the product image under different illumination conditions can be determined, providing crucial information for subsequent illumination intensity adjustment. The illumination intensity adjustment coefficient (L) is calculated using a formula that comprehensively considers the changes in the initial illumination intensity and the target area area in the product test image. This method avoids the subjectivity of manual judgment and improves the accuracy and repeatability of illumination adjustment. The formula includes the standard deviation (Bb) of the initial illumination intensity and the standard deviation (Sb) of the target area area, which helps to reflect the fluctuations in illumination intensity and image quality, thereby allowing for more precise adjustment of illumination intensity. Using the calculated illumination intensity adjustment coefficient, the illumination intensity for image acquisition of the product to be tested can be adjusted. This helps ensure consistent product image quality under varying lighting conditions, thereby improving the accuracy of image analysis, product inspection, and quality control. This technical solution reduces image quality issues and inspection errors caused by improper lighting through automated and precise adjustment of light intensity, thus improving production efficiency and product quality. Furthermore, this technology can be applied to various scenarios requiring image acquisition and processing, such as machine vision, quality inspection, and product classification, demonstrating broad application prospects.

[0106] In summary, this technical solution optimizes image acquisition quality and improves production efficiency and product quality through precise light intensity adjustment and automated light intensity scaling factor calculation, demonstrating significant technical effects and practical application value.

[0107] Specifically, the intensity modulation coefficient adjusts the illumination intensity of the image acquisition of the product to be tested, including:

[0108] The initial light intensity value corresponding to the minimum area of ​​the target region in the product test image is retrieved as the first light intensity.

[0109] The initial light intensity value corresponding to the maximum area of ​​the target region in the product test image is retrieved as the second light intensity.

[0110] A reference value for illumination intensity is obtained using the first and second illumination intensities.

[0111] The reference value for light intensity is obtained by the following formula:

[0112]

[0113] Among them, B cB represents the reference value for light intensity. p B represents the average value of the initial illumination intensities contained in the initial illumination intensity group; min Indicates the first light intensity; B max Indicates the second light intensity; n represents the number of initial light intensity values ​​included in the initial light intensity group; H zi H represents the median grayscale value corresponding to the area of ​​the target region corresponding to the i-th initial illumination intensity value; ki This represents the center grayscale value of the product test image corresponding to the i-th initial illumination intensity value;

[0114] Retrieve the light intensity modulation coefficient;

[0115] The light intensity of the product to be tested is obtained using the light intensity modulation coefficient and the light intensity reference value.

[0116] The light intensity is obtained by the following formula:

[0117]

[0118] Among them, B t Indicates light intensity; B c represents the baseline value of light intensity; L represents the light intensity adjustment factor.

[0119] The technical effect of the above solution is as follows: By retrieving the initial illumination intensity values ​​(i.e., the first illumination intensity and the second illumination intensity) corresponding to the minimum and maximum target area in the product test image, the illumination conditions that cause extreme changes in image quality can be identified. This provides an important reference for further determining the illumination intensity benchmark value. The calculation formula for the illumination intensity benchmark value comprehensively considers the average value of the initial illumination intensity (Bp), the first illumination intensity (Bmin), the second illumination intensity (Bmax), and the center gray value (Hz i) of the target area corresponding to each initial illumination intensity. (Note that the definition of Hz i seems somewhat ambiguous in the original text, as the center gray value usually does not change with the initial illumination intensity; here it may refer to a gray value related to illumination intensity). This comprehensive calculation method helps to obtain a more scientific and reasonable illumination intensity benchmark value, providing a reliable basis for subsequent adjustment of the illumination intensity of the product under test. By retrieving the previously calculated intensity modulation coefficient, it can be combined with the illumination intensity benchmark value to further calculate the optimal illumination intensity (Bt) of the product under test. The intensity adjustment factor reflects the impact of changes in light intensity on product image quality, thus its utilization helps achieve more precise light intensity adjustment. Precise light intensity adjustment ensures the image acquisition quality of the product under inspection, reducing problems such as image blurring, overexposure, or underexposure caused by improper lighting. This helps improve the accuracy of product inspection and reduce the risk of false positives and false negatives. Automated and intelligent light intensity adjustment processes reduce manual intervention and repetitive adjustments, thereby optimizing production processes and improving efficiency. This technical solution is not only applicable to product inspection but can also be extended to machine vision, image processing, image recognition, and other fields, providing these fields with precise and efficient light intensity adjustment methods.

[0120] In summary, this technical solution achieves precise adjustment of the light intensity of the product under test by accurately calculating the baseline value of light intensity and effectively utilizing the light intensity adjustment coefficient. This improves the image acquisition quality and product testing accuracy, optimizes the production process and efficiency, and has broad application prospects and significant technical effects.

[0121] In this embodiment, as a preferred technical solution of the present invention, the image processing module includes:

[0122] The image denoising unit is used to denoise real-time images of product appearance based on computer vision.

[0123] Acquire real-time images of product appearance based on computer vision, convert the real-time images of product appearance based on computer vision into grayscale images, and perform noise reduction processing on the real-time images of product appearance based on computer vision using a median filter.

[0124] Specifically, the window size of the median filter is set based on the noise type and intensity, as well as the image details and size, in the real-time image of the product appearance based on computer vision.

[0125] When the noise in the real-time image of a product appearance based on computer vision is salt-and-pepper noise or random noise, and the noise intensity is high, a large window is used; when the noise intensity is low, a small window is used.

[0126] When an image contains important details or edge information, use a small window; when the image size is large, use a small window or a block-based processing method.

[0127] After setting the window size of the median filter, for each pixel in the real-time image of the product appearance based on computer vision, the median of the gray values ​​of all pixels in the neighborhood of the surrounding window size is calculated, and the gray value of the pixel is replaced with the median. This is used to remove or reduce noise in the real-time image of the product appearance based on computer vision, and the denoised real-time image of the product appearance is saved or displayed.

[0128] In this embodiment, the median filter can use a 3*3 window to calculate the median of the gray values ​​of all pixels in the 3*3 window neighborhood around it, and replace the gray value of the pixel with the median to remove or reduce noise in the real-time image of the product appearance based on computer vision.

[0129] In this embodiment, as a preferred technical solution of the present invention, the image processing module further includes:

[0130] The image adjustment unit is used to adjust the brightness and contrast of the real-time image of the product appearance after noise reduction.

[0131] Brightness adjustment: Obtain the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising. Increase the brightness by increasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising, and decrease the brightness by decreasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising.

[0132] Contrast Adjustment: Based on histogram equalization, the contrast of the denoised real-time product appearance image is adjusted. By redistributing the gray values ​​of the real-time product appearance image, the gray distribution of the real-time product appearance image is adjusted to a uniform distribution, increasing the dynamic range of pixel gray values, thereby adjusting the contrast of the real-time product appearance image.

[0133] In this embodiment, as a preferred technical solution of the present invention, the image semantic module includes:

[0134] The extraction and matching unit is used to extract and match features from the processed real-time image of the product appearance.

[0135] Based on a convolutional neural network deep learning model, feature extraction is performed on the processed real-time product appearance image. The features of the real-time product appearance image are automatically learned, and highly discriminative feature vectors are extracted, including shape features, texture features, and color features. A similarity algorithm is used to match the extracted feature vectors with features in an existing feature library.

[0136] The semantic segmentation unit is used to perform semantic segmentation on real-time product appearance images, dividing the real-time product appearance images into different semantic regions. Each semantic region has the same category label. The extracted feature vectors and semantic segmentation results are fused together, and the feature vectors and semantic labels are concatenated to generate a unified product appearance semantic image.

[0137] It should be noted that semantic segmentation is an image processing technique that aims to classify each pixel in a real-time image of a product's appearance into a specific semantic category. It does not distinguish between different individuals within the same category, but rather focuses on the semantic content to which the pixel belongs.

[0138] In this embodiment, as a preferred technical solution of the present invention, the defect detection module includes:

[0139] The model training unit is used to train a product appearance defect detection model based on image semantics.

[0140] Based on the requirements for product appearance defect detection using image semantics, collect historical images of product appearance.

[0141] The collected historical images of product appearance are divided to determine the training set and the test set;

[0142] Based on the training set, the machine learning model is trained and iterated to enable the machine learning model to automatically learn product appearance defect detection and determine the product appearance defect detection model based on image semantics.

[0143] Based on the test set, the performance of the image semantic-based product appearance defect detection model is tested to determine whether the image semantic-based product appearance defect detection model can achieve the expected results and to determine the optimal image semantic-based product appearance defect detection model.

[0144] In this embodiment, as a preferred technical solution of the present invention, the defect detection module further includes:

[0145] The defect detection unit is used to detect defects in the product's appearance.

[0146] Obtain the optimal product appearance defect detection model based on image semantics;

[0147] Deploy the optimal image semantic-based product appearance defect detection model in a real product appearance defect detection environment;

[0148] Based on the optimal image semantics-based product appearance defect detection model, image semantics recognition is performed on the product appearance semantic image to check whether there are defects in the product appearance and to determine the product appearance defect detection result based on image semantics.

[0149] The early warning display unit is used to provide early warnings and visual displays of product appearance defects.

[0150] Obtain product appearance defect detection results based on image semantics;

[0151] When a defect is detected in the product's appearance, an early warning alarm will be automatically issued to notify the management personnel in a timely manner.

[0152] Based on product information, defects in product appearance are marked to generate a product appearance defect inspection report, which is then presented to managers in a visual format.

[0153] Example 2

[0154] To better illustrate the product appearance defect detection process based on image semantics, this embodiment provides a product appearance defect detection method based on image semantics, implemented based on the aforementioned product appearance defect detection system based on image semantics, and includes the following steps:

[0155] S1: Place the product to be inspected on the product appearance defect inspection station. Based on computer vision technology, the product to be inspected is collected in real time under different acquisition angles and light intensity conditions to determine the real-time image of the product appearance based on computer vision.

[0156] S2: Based on the median filter, perform noise reduction processing on the real-time product appearance image based on computer vision, remove or reduce noise in the real-time product appearance image based on computer vision, and save or display the denoised real-time product appearance image.

[0157] S3: Adjust the brightness of the real-time product appearance image by increasing or decreasing the grayscale value of all pixels in the real-time product appearance image, and adjust the contrast of the real-time product appearance image by increasing the dynamic range of pixel grayscale values ​​based on histogram equalization.

[0158] S4: Based on a convolutional neural network deep learning model, feature extraction is performed on real-time product appearance images to extract highly discriminative feature vectors. A similarity algorithm is then used to match the extracted feature vectors with features in an existing feature library. Semantic segmentation is performed on the real-time product appearance images, and the extracted feature vectors and semantic segmentation results are fused to generate a semantic image of the product appearance.

[0159] S5: Based on the optimal image semantic-based product appearance defect detection model, perform image semantic recognition on the product appearance semantic image to determine whether there are defects in the product appearance. When a defect is detected, an early warning alarm is automatically issued to notify the management personnel in a timely manner. Based on the product information, the product appearance defect is marked and a product appearance defect detection report is generated. The product appearance defect detection report is presented to the management personnel in a visual form, which facilitates the timely management of products with appearance defects.

[0160] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0161] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A product appearance defect detection system based on image semantics, characterized in that, include: Image acquisition module, used to acquire real-time images of product appearance based on computer vision; The image processing module is used to process real-time images of product appearance based on computer vision. The image semantic module is used to extract features and perform semantic segmentation on real-time images of product appearance to generate semantic images of product appearance. The defect detection module is used to detect defects in the product appearance based on machine learning technology, determine whether there are defects in the product appearance, and provide early warnings and visual displays of the defects in the product appearance. The image acquisition module includes: The product fixing unit is used to fix the product to be tested. Place the product to be inspected on the product appearance defect inspection station and fix it in place to ensure stability. The acquisition and adjustment unit is used to adjust the acquisition angle and light intensity of the product to be tested; When acquiring images of the product to be inspected, the acquisition angle and light intensity of the product to be inspected are adjusted based on the intelligent adjustment device; The product acquisition unit is used to acquire real-time images of the product's appearance. Based on computer vision technology, machine vision equipment is used to monitor and continuously collect images of the product under inspection in real time, and to obtain product images of the product under inspection under different collection angles and light intensity conditions, thereby determining the real-time image of the product appearance based on computer vision. The acquisition and adjustment unit also specifically performs the following steps: Extract the initial light intensity value for each element in the initial light intensity group for the product; After the test product is fixed, the test product is photographed according to each initial light intensity value to obtain the product test image corresponding to each initial light intensity value; The grayscale value of the pixels in the product test image corresponding to each initial light intensity value is compared with a preset grayscale threshold to obtain the area of ​​the target region formed by pixels whose grayscale value exceeds the preset grayscale threshold. The intensity modulation coefficient is obtained by using the target area of ​​the product image corresponding to each initial illumination intensity value; The light intensity modulation coefficient is obtained by the following formula: Where L represents the light intensity modulation factor; n represents the number of initial light intensity values ​​included in the initial light intensity group; B i B represents the i-th initial illumination intensity value contained in the initial illumination intensity group; i+1 S represents the (i+1)th initial illumination intensity value contained in the initial illumination intensity group; i S represents the area of ​​the target region of the product test image corresponding to the i-th initial illumination intensity value in the initial illumination intensity group; i+1 B represents the area of ​​the target region of the product test image corresponding to the (i+1)th initial illumination intensity value in the initial illumination intensity group; b S represents the standard deviation of the n initial light intensity values ​​contained in the initial light intensity group; b This represents the standard deviation of the target area of ​​the product test image corresponding to the n initial light intensity values ​​contained in the initial light intensity group; The light intensity of the image acquisition of the product to be tested is adjusted using the light intensity modulation coefficient.

2. The product appearance defect detection system based on image semantics as described in claim 1, characterized in that, Adjusting the illumination intensity of the image acquired from the product to be tested using the intensity modulation coefficient includes: The initial light intensity value corresponding to the minimum area of ​​the target region in the product test image is retrieved as the first light intensity. The initial light intensity value corresponding to the maximum area of ​​the target region in the product test image is retrieved as the second light intensity. A reference value for illumination intensity is obtained using the first and second illumination intensities. The reference value for light intensity is obtained by the following formula: Among them, B c B represents the reference value for light intensity. p B represents the average value of the initial illumination intensities contained in the initial illumination intensity group; min Indicates the first light intensity; B max Indicates the second light intensity; n represents the number of initial light intensity values ​​included in the initial light intensity group; H zi H represents the median grayscale value corresponding to the area of ​​the target region corresponding to the i-th initial illumination intensity value; ki This represents the center grayscale value of the product test image corresponding to the i-th initial illumination intensity value; Retrieve the light intensity modulation coefficient; The light intensity of the product to be tested is obtained using the light intensity modulation coefficient and the light intensity reference value. The light intensity is obtained by the following formula: Among them, B t Indicates light intensity; B c represents the baseline value of light intensity; L represents the light intensity adjustment factor.

3. The product appearance defect detection system based on image semantics as described in claim 2, characterized in that, The image processing module includes: The image denoising unit is used to denoise real-time images of product appearance based on computer vision. Acquire real-time images of product appearance based on computer vision, convert the real-time images of product appearance based on computer vision into grayscale images, and perform noise reduction processing on the real-time images of product appearance based on computer vision using a median filter. Specifically, the window size of the median filter is set based on the noise type and intensity, as well as the image details and size, in the real-time image of the product appearance based on computer vision. When the noise in the real-time image of a product appearance based on computer vision is salt-and-pepper noise or random noise, and the noise intensity is high, a large window is used; when the noise intensity is low, a small window is used. When an image contains important details or edge information, use a small window; when the image size is large, use a small window or a block-based processing method. After setting the window size of the median filter, for each pixel in the real-time image of the product appearance based on computer vision, the median of the gray values ​​of all pixels in the neighborhood of the surrounding window size is calculated, and the gray value of the pixel is replaced with the median. This is used to remove or reduce noise in the real-time image of the product appearance based on computer vision, and the denoised real-time image of the product appearance is saved or displayed.

4. The product appearance defect detection system based on image semantics as described in claim 3, characterized in that, The image processing module further includes: The image adjustment unit is used to adjust the brightness and contrast of the real-time image of the product appearance after noise reduction. Brightness adjustment: Obtain the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising. Increase the brightness by increasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising, and decrease the brightness by decreasing the grayscale values ​​of all pixels in the real-time image of the product appearance after denoising. Contrast Adjustment: Based on histogram equalization, the contrast of the denoised real-time product appearance image is adjusted. By redistributing the gray values ​​of the real-time product appearance image, the gray distribution of the real-time product appearance image is adjusted to a uniform distribution, increasing the dynamic range of pixel gray values, thereby adjusting the contrast of the real-time product appearance image.

5. The product appearance defect detection system based on image semantics as described in claim 4, characterized in that, The image semantic module includes: The extraction and matching unit is used to extract and match features from the processed real-time image of the product appearance. Based on a convolutional neural network deep learning model, feature extraction is performed on the processed real-time product appearance image. The features of the real-time product appearance image are automatically learned, and highly discriminative feature vectors are extracted, including shape features, texture features, and color features. A similarity algorithm is used to match the extracted feature vectors with features in an existing feature library. The semantic segmentation unit is used to perform semantic segmentation on real-time product appearance images, dividing the real-time product appearance images into different semantic regions. Each semantic region has the same category label. The extracted feature vectors and semantic segmentation results are fused together, and the feature vectors and semantic labels are concatenated to generate a unified product appearance semantic image.

6. The product appearance defect detection system based on image semantics as described in claim 5, characterized in that, The defect detection module includes: The model training unit is used to train a product appearance defect detection model based on image semantics. Based on the requirements for product appearance defect detection using image semantics, collect historical images of product appearance. The collected historical images of product appearance are divided to determine the training set and the test set; Based on the training set, the machine learning model is trained and iterated to enable the machine learning model to automatically learn product appearance defect detection and determine the product appearance defect detection model based on image semantics. Based on the test set, the performance of the image semantic-based product appearance defect detection model is tested to determine whether the image semantic-based product appearance defect detection model can achieve the expected results and to determine the optimal image semantic-based product appearance defect detection model.

7. The product appearance defect detection system based on image semantics as described in claim 6, characterized in that, The defect detection module further includes: The defect detection unit is used to detect defects in the product's appearance. Obtain the optimal product appearance defect detection model based on image semantics; Deploy the optimal image semantic-based product appearance defect detection model in a real product appearance defect detection environment; Based on the optimal image semantics-based product appearance defect detection model, image semantics recognition is performed on the product appearance semantic image to check whether there are defects in the product appearance and to determine the product appearance defect detection result based on image semantics. The early warning display unit is used to provide early warnings and visual displays of product appearance defects. Obtain product appearance defect detection results based on image semantics; When a defect is detected in the product's appearance, an early warning alarm will be automatically issued to notify the management personnel in a timely manner. Based on product information, defects in product appearance are marked to generate a product appearance defect inspection report, which is then presented to managers in a visual format.

8. A product appearance defect detection method based on image semantics, implemented based on the product appearance defect detection system based on image semantics as described in claim 7, characterized in that, Includes the following steps: S1: Place the product to be inspected on the product appearance defect inspection station. Based on computer vision technology, the product to be inspected is collected in real time under different acquisition angles and light intensity conditions to determine the real-time image of the product appearance based on computer vision. S2: Based on the median filter, perform noise reduction processing on the real-time product appearance image based on computer vision, remove or reduce noise in the real-time product appearance image based on computer vision, and save or display the denoised real-time product appearance image. S3: Adjust the brightness of the real-time product appearance image by increasing or decreasing the grayscale value of all pixels in the real-time product appearance image, and adjust the contrast of the real-time product appearance image by increasing the dynamic range of pixel grayscale values ​​based on histogram equalization. S4: Based on a convolutional neural network deep learning model, feature extraction is performed on real-time product appearance images to extract highly discriminative feature vectors. A similarity algorithm is then used to match the extracted feature vectors with features in an existing feature library. Semantic segmentation is performed on the real-time product appearance images, and the extracted feature vectors and semantic segmentation results are fused to generate a semantic image of the product appearance. S5: Based on the optimal image semantic-based product appearance defect detection model, perform image semantic recognition on the product appearance semantic image to determine whether there are defects in the product appearance. When defects are detected, an early warning alarm is automatically issued to notify the management personnel in a timely manner. Based on the product information, the product appearance defect is marked and a product appearance defect detection report is generated. The product appearance defect detection report is presented to the management personnel in a visual form.

Images