Tableware surface dirt detection method and system based on image recognition

Abstract

The present application relates to the field of image processing technology, and in particular to a method and system for detecting stains on tableware surfaces based on image recognition. The method comprises acquiring a tableware image, obtaining a first tableware suspicion level indicating that each pixel is a tableware region based on the brightness and saturation of each pixel in the tableware image in the HSI color space, performing reflection suppression preprocessing on the tableware image to obtain an enhanced image, obtaining a second tableware suspicion level indicating that the pixel is a tableware region based on the chromaticity of each pixel in the enhanced image in the LAB color space, obtaining a grayscale image of the enhanced image, obtaining a third tableware suspicion level indicating that each pixel is a tableware region based on the grayscale of each pixel, and obtaining a comprehensive suspicion level, and marking the stain region in the tableware image based on the comprehensive suspicion level of each pixel. The present application improves the accuracy of tableware stain detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and system for detecting dirt on the surface of tableware based on image recognition. Background Art

[0002] With the increasing digitalization and intelligence of industry, the catering industry has gradually introduced some automated equipment, such as industrial dishwashers, intelligent cooking robots, and intelligent vegetable cutters. Industrial dishwashers are used to clean tableware, which then needs to be stored and disinfected. To ensure the cleanliness of dishware before reuse, dishwasher-processed tableware is often inspected manually through visual inspection before storage and disinfection to reduce the risk of stains. Although manual visual inspection is relatively simple, it is highly subjective and has low detection efficiency. Therefore, some intelligent detection methods based on image processing technology are gradually being applied to tableware stain detection. For example, threshold segmentation is an image segmentation algorithm that sets one or more fixed thresholds based on the image and then divides the image into multiple components based on the thresholds. For example, pixels in the image with grayscale values ​​below the threshold are considered stains, while the rest are considered background.

[0003] Traditional stain detection algorithms offer simple methods and fast processing speeds. However, in practice, images may experience grayscale variations due to external lighting conditions. Highly reflective areas of ceramic tableware can obscure stains, or textures on the tableware can resemble stains. This can lead to misidentification or missed detections. Consequently, traditional stain detection algorithms exhibit low accuracy when faced with complex environmental factors (e.g., tableware material, external lighting, and tableware patterns). Summary of the Invention

[0004] In order to solve the problem of low accuracy of stain detection in traditional tableware stain detection methods, the present application provides a tableware surface stain detection method and system based on image recognition.

[0005] In the first aspect, the present application provides a tableware surface dirt detection method based on image recognition, which adopts the following technical solutions:

[0006] A tableware surface dirt detection method based on image recognition comprises the following steps: obtaining a tableware image, obtaining a first tableware suspicion degree for each pixel in the tableware image based on the brightness and saturation of each pixel in the tableware image in the HSI color space; performing reflection suppression preprocessing on the tableware image to obtain an enhanced image; obtaining a second tableware suspicion degree for the pixel in the tableware area based on the chromaticity of each pixel in the enhanced image in the LAB color space; obtaining a grayscale image of the enhanced image, and obtaining a third tableware suspicion degree for each pixel based on the grayscale of each pixel; binarizing the first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree according to a preset threshold to obtain a first standard stain value, a second standard stain value, and a third standard stain value; taking the weighted sum of the first standard stain value, the second standard stain value, and the third standard stain value as a comprehensive suspicion degree, and marking the stain area in the tableware image based on the comprehensive suspicion degree of each pixel.

[0007] The beneficial effects are as follows: In this method, the pixel features in the tableware image are analyzed in HSI, LAB and grayscale space respectively, and three types of tableware suspicion (first tableware suspicion, second tableware suspicion and third tableware suspicion) are constructed. A multi-dimensional judgment standard is established from the three perspectives of brightness, chromaticity and texture, and then the comprehensive suspicion is calculated through a fusion mechanism and the stain area is marked. Compared with the traditional threshold segmentation method for detecting stains, this method reduces the problem that the traditional threshold segmentation is based only on grayscale information, resulting in the final detection result being susceptible to false detection and missed detection due to interference from light and patterns. At the same time, the multi-channel information fusion in this method improves the stability and robustness of recognition, enabling the system to adapt to tableware images of different materials, different colors and different cleaning states, accurately judge the stain area and highlight it, significantly improve the detection accuracy, and reduce the burden of manual detection. It is a multimodal stain recognition strategy with a clear structure, light calculation and reliable results.

[0008] Optionally, the step of obtaining a first tableware suspicion degree that each pixel is a tableware area based on the brightness and saturation of each pixel in the tableware image in the HSI color space includes: obtaining the brightness level of the neighborhood of each pixel, and taking the ratio of the brightness of the pixel to the average brightness of the corresponding neighborhood as the reflectance of the pixel; calculating a brightness compensation value based on the saturation of each pixel, where the saturation is positively correlated with the brightness compensation value; and taking the sum of the brightness compensation value and the reflectance as the first tableware suspicion degree.

[0009] The beneficial effect is that the I channel of the tableware image in the HSI color space represents the brightness of the tableware image. For stained areas on the tableware, due to their weak reflection, the brightness of these areas in the tableware image is low. Therefore, the first tableware suspicion degree is obtained based on the brightness of the pixel points. The lower the pixel brightness, the more likely it is a stain. Similarly, in the tableware image, areas with patterns also appear to have low brightness, even though these areas are not actually stains. Therefore, saturation is used to compensate for the brightness of the low-brightness pixels to reduce the possibility of misjudgment.

[0010] Optionally, the average of the brightness values ​​of all pixels in the neighborhood of the pixel point is used as the brightness level of the neighborhood of the pixel point.

[0011] The beneficial effect is that the brightness ratio is determined based on the neighborhood corresponding to the pixel point, thereby reducing the situation where the brightness at different positions in the tableware image is uneven due to external factors such as lighting, thereby reducing the misjudgment of stains.

[0012] Optionally, the step of calculating the brightness compensation value based on the saturation of each pixel includes: setting an adjustment coefficient, and taking the product of the normalized result of the pixel saturation and the adjustment coefficient as the brightness compensation value.

[0013] The beneficial effect is that in the tableware image, the higher the saturation, the stronger the color of the area, which may be a decorative pattern, so higher brightness compensation is performed. Conversely, if the saturation is lower, it may be a stained area, so less compensation is performed, or no compensation is performed.

[0014] Optionally, an adjustment coefficient and an enhancement coefficient are set, and the product of the enhancement coefficient and the normalized result of the pixel saturation is used as the enhancement degree, the product of the enhancement degree and the adjustment coefficient is used as the adjustment index, and the normalized result of the adjustment index and the saturation is used as the brightness compensation value.

[0015] The beneficial effect is that a nonlinear model is constructed in this method, so that the final brightness compensation value is more sensitive to the change of saturation.

[0016] Optionally, the step of obtaining a second tableware suspicion degree in which the pixel is a tableware area based on the chromaticity of each pixel in the enhanced image in the LAB color space includes: taking the average of the red and green channel values ​​of each pixel in the enhanced image as the first chromaticity level of the enhanced image; taking the average of the yellow and blue channel values ​​of each pixel in the enhanced image as the second chromaticity level of the enhanced image; for each pixel, taking the absolute difference between the red and green channel values ​​of the pixel and the first chromaticity level as the first deviation, and taking the absolute difference between the yellow and blue channel values ​​of the pixel and the second chromaticity level as the second deviation, and fusing the first deviation and the second deviation to obtain the second tableware suspicion degree.

[0017] The beneficial effect is that the stained area is often gray or neutral, and its chromaticity value is close to the average of the entire image; while the pattern area has a significant color cast, so its difference from the average is greater. Therefore, the stained area is distinguished by the first deviation and the second deviation.

[0018] Optionally, the step of obtaining the third tableware suspicion degree of each pixel point based on the grayscale of each pixel point includes: obtaining the average grayscale value of the pixels in the neighborhood corresponding to each pixel point; obtaining the grayscale difference based on the difference between the grayscale value of the pixel point and the average grayscale value of the pixels in the corresponding neighborhood, and taking the average of the grayscale differences as the third tableware suspicion degree.

[0019] The beneficial effect is: in the grayscale image, the pattern on the surface of the tableware is usually accompanied by high-frequency grayscale changes, while the contaminated area is usually distributed in patches, and the grayscale in the area is smooth. Therefore, the stained area is distinguished here by the difference in the mean grayscale of the pixel point and its corresponding neighborhood.

[0020] Optionally, the step of fusing the first deviation and the second deviation to obtain the second tableware suspicion level includes: taking the sum of the squares of the first deviation and the second deviation as the total deviation, and taking the arithmetic square root of the total deviation as the second tableware suspicion level.

[0021] Optionally, the preset threshold includes a first threshold, a second threshold and a third threshold; the step of binarizing the first tableware suspicion degree, the second tableware suspicion degree and the third tableware suspicion degree includes: for the first tableware suspicion degree, in response to the first tableware suspicion degree being less than the first threshold, 1 is used as the value of the first standard stain value, otherwise 0 is used as the value of the first standard stain value; for the second tableware suspicion degree, in response to the second tableware suspicion degree being less than the second threshold, 1 is used as the value of the second standard stain value, otherwise 0 is used as the value of the second standard stain value; for the third tableware suspicion degree, in response to the third tableware suspicion degree being less than the third threshold, 1 is used as the value of the third standard stain value, otherwise 0 is used as the value of the third standard stain value.

[0022] This is mainly used to identify stain areas under different channels, such as the first tableware suspicion level. Based on the above analysis, the greater the tableware suspicion level, the more likely the pixel is to be tableware. Therefore, a first threshold is used to distinguish pixels. For pixels below the first threshold, the corresponding first standard stain value is defined as 1, and the remaining pixels are defined as 0. A first standard stain value of 1 indicates that the pixel is a stained area.

[0023] In a second aspect, the present application provides a tableware surface dirt detection system based on image recognition, which adopts the following technical solutions:

[0024] The tableware surface dirt detection system based on image recognition includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the tableware surface dirt detection method based on image recognition is implemented.

[0025] The beneficial effect is that the above-mentioned tableware surface dirt detection method based on image recognition is generated into a computer program and stored in a memory so as to be loaded and executed by a processor, thereby making a system based on the memory and the processor for easy use.

[0026] This application has the following technical effects:

[0027] The stain features under multiple channels are analyzed and differentiated, and the tableware suspicion degree under each channel is calculated. The multiple tableware suspicion degrees are then integrated to obtain a comprehensive result, improving the accuracy and robustness of stain detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a method flow chart of a tableware surface dirt detection method based on image recognition in an embodiment of the present application.

[0029] Figure 2 This is a method flow chart of step S2 in the tableware surface dirt detection method based on image recognition in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiment of the present application discloses a method for detecting dirt on the surface of tableware based on image recognition, which obtains a tableware image and obtains the tableware suspicion degree of each pixel point (a collective term for the first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree) based on the different features of the pixel points in different color spaces of the tableware image. The tableware suspicion degrees of each pixel point in different color spaces are then fused to obtain a comprehensive suspicion degree, and the stain area is identified in the image based on the comprehensive suspicion degree to complete the detection of the tableware. By fusing three types of information sources, a multi-dimensional redundant judgment mechanism is implemented at the feature layer to avoid misjudgments and missed detections caused by a single channel, and finally the suspected stain area is marked and identified based on the fusion result. This method can effectively adapt to different tableware materials (such as ceramics, stainless steel), complex patterns, and uneven lighting conditions.

[0031] Reference Figure 1 The tableware surface dirt detection method based on image recognition includes steps S1 to S7.

[0032] S1: Acquire tableware images.

[0033] Build an image acquisition device consisting of at least one industrial camera (such as a CMOS camera) with a fixed focal length and high resolution. The industrial camera is mounted above the inspection station with its lens axis perpendicular to the surface where the tableware is placed. The industrial camera is secured with a bracket and communicates with an image processing terminal (such as an industrial computer, edge computing unit, or embedded board). The camera's resolution must be at least 1920×1080 to ensure sufficient imaging accuracy for stain details. The camera captures the tableware, generating an image of the tableware, which is stored in an uncompressed format in a local cache.

[0034] S2: Obtain a first tableware suspicion degree of each pixel being a tableware area based on the brightness and saturation of each pixel in the tableware image in the HSI color space.

[0035] The dishware image is displayed in the HSI color space. In the HSI color space, H represents the color frequency, known as hue; I represents intensity or brightness, referred to as brightness in this embodiment; and S represents the color depth, known as saturation. The image captured by the image sampling device in this embodiment is in RGB format, so the RGB dishware image needs to be converted to the HSI color space. This step is conventional in the art and will not be detailed here.

[0036] Reference Figure 2 The step of obtaining a first tableware suspicion degree of each pixel point being a tableware area includes: step S21 and step S22.

[0037] S21: Obtain the brightness level of the neighborhood of each pixel, and use the ratio of the brightness of the pixel to the average brightness of the corresponding neighborhood as the reflectance of the pixel.

[0038] The brightness value of each pixel in the image is obtained (which can also be understood as the brightness channel component of each pixel in the tableware image, and is simply referred to as the brightness value in this embodiment for ease of description).

[0039] Based on each pixel point, the neighborhood is divided into each pixel point, and the brightness level in the neighborhood of each pixel point is calculated. In this embodiment, the size is set to The sliding window is used to calculate the average brightness value of the pixels in the sliding window during the process of sliding the window pixel by pixel. The average brightness value is used as the brightness level of the neighborhood of the pixel corresponding to the center of the sliding window. It can also be understood as constructing a neighborhood with each pixel as the center. The average brightness of the window is calculated as the brightness level.

[0040] For any pixel, the ratio of the brightness level of the pixel to the corresponding neighborhood is used as the brightness ratio. Clean areas on the surface of tableware (such as smooth ceramics) usually appear as mirror reflections with uniform and high brightness; while stains (such as oil, water, and residue) usually absorb light or have low reflectivity, and will appear darker in the image. This ratio reflects the reflectivity of the current pixel relative to the surrounding environment. If the brightness is relatively low, it means that the current pixel is in a relatively dark area and may be a stain. In addition, the denominator of the ratio is the brightness level of the pixel's neighborhood. Different pixels correspond to different neighborhood brightness levels. It can also be understood that the brightness ratio is adaptive to the different pixel positions. Therefore, it can handle the uneven lighting conditions at different positions in the tableware image and improve the robustness of the detection system.

[0041] S22: Calculating a brightness compensation value based on the saturation of each pixel point, where the saturation is positively correlated with the brightness compensation value; and taking the sum of the brightness compensation value and the reflectance as the first tableware suspicion level.

[0042] Considering the impact of surface patterns (such as green, blue, and pink) on light reflection during inspection, different colors have different reflectances: white areas reflect nearly all wavelengths, resulting in a high grayscale value; dark blue / green areas absorb most red and yellow wavelengths, reflecting only short-wavelength blue / green light, resulting in a significant decrease in grayscale value; pink areas partially absorb blue light, reflecting red and green wavelengths, resulting in medium to high brightness; black or dark patterns strongly absorb and reflect little, resulting in a significantly lower brightness value, similar to oil stains. Therefore, even clean dark patterned areas may appear as low-brightness areas in the image, confusing them with dark areas caused by oil stains, leading to false detections.

[0043] Therefore, in order to further improve the accuracy of stain recognition based on brightness and avoid the tableware pattern area being misjudged as stains due to low brightness, the system introduces a saturation compensation mechanism to construct a more reasonable first tableware suspicion estimation.

[0044] In the process of calculating the brightness compensation value based on the saturation of each pixel, first, the saturation component of each pixel is obtained, and the saturation of the pixel is normalized.

[0045] In one embodiment, the step of calculating a brightness compensation value based on the saturation of each pixel includes setting an adjustment coefficient and normalizing the adjustment coefficient with the pixel saturation as the brightness compensation value. The brightness compensation value and the brightness ratio are used as the first tableware suspicion level. In this method, higher saturation indicates lower brightness in the image, indicating that the area is likely not stained. Therefore, the brightness of the pixels in this area is compensated to prevent areas with high patterns from being identified as stained, thereby improving the robustness of the detection system.

[0046] In another embodiment, the step of calculating the brightness compensation value based on the saturation of each pixel includes: taking the average of the red and green channel values ​​of each pixel in the enhanced image as the first chromaticity level of the enhanced image; taking the average of the yellow and blue channel values ​​of each pixel in the enhanced image as the second chromaticity level of the enhanced image; for each pixel, taking the absolute difference between the red and green channel values ​​of the pixel and the first chromaticity level as the first deviation, taking the absolute difference between the yellow and blue channel values ​​of the pixel and the second chromaticity level as the second deviation, and fusing the first deviation and the second deviation to obtain a second tableware suspicion level.

[0047] Specifically, the calculation of the brightness compensation value can be expressed by the following formula:

[0048] Where, Indicates the suspicion level of the second tableware; represents the enhancement coefficient; represents the adjustment coefficient; Indicates the normalized result of the pixel's saturation.

[0049] In this method, a nonlinear compensation model is constructed through enhancement coefficients and adjustment coefficients, making the final brightness compensation value more sensitive and more adjustable.

[0050] S3: Perform reflection suppression preprocessing on the tableware image to obtain an enhanced image.

[0051] For some ceramic tableware or stainless steel tableware, highlight areas are easily generated during the image acquisition process. In order to reduce the impact of highlight areas on subsequent image processing, the tableware image is pre-processed to suppress reflections.

[0052] The specific reflection suppression pretreatment steps include:

[0053] S31: Histogram equalization: First, perform histogram equalization on the tableware image to enhance the overall contrast of the image.

[0054] S32: Converting the RGB color space into the LAB space: Converting the captured tableware image in the RGB color space into the LAB space to obtain an L component, an A component, and a B component.

[0055] S33: Contrast Limited Adaptive Histogram Equalization: The A and B components remain unchanged, and the luminance L component is subjected to Contrast Limited Adaptive Histogram Equalization (CLAHE) to obtain a remapped luminance L component.

[0056] S34: LAB space conversion to RGB color space: The A component, the B component and the enhanced brightness component L are spatially converted to the RGB color space to obtain an enhanced image.

[0057] Histogram equalization and contrast-limited adaptive histogram equalization methods are conventional technical means in this field and will not be described in detail here.

[0058] S4: Obtain a second tableware suspicion degree where the pixel is a tableware area based on the chromaticity of each pixel in the enhanced image in the LAB color space.

[0059] The enhanced image is converted to the LAB color space. In the LAB color space, the L channel mainly represents the black, white and gray areas in the image, which is called brightness; the A channel mainly contains the areas from green to magenta information in the image, which is called red-green; the B channel mainly contains the areas from blue to yellow information in the image, which is called yellow-blue.

[0060] In the step of obtaining the second tableware suspicion degree of the pixel being the tableware area, first, the average of the red and green channel values ​​of each pixel in the enhanced image is used as the first chromaticity level of the enhanced image; and the average of the yellow and blue channel values ​​of each pixel in the enhanced image is used as the second chromaticity level of the enhanced image.

[0061] For each pixel, the absolute difference between the red and green channel values ​​of the pixel and the first chromaticity level is used as the first deviation, and the absolute difference between the yellow and blue channel values ​​of the pixel and the second chromaticity level is used as the second deviation. The first deviation and the second deviation are combined to obtain the second tableware suspicion level.

[0062] In this step, the overall background reference chromaticity (first chromaticity level and second chromaticity level) of the image is calculated. Then, each pixel is compared with the overall background reference chromaticity to reflect the degree of color deviation of the pixel (i.e., first deviation and second deviation).

[0063] After the first deviation and the second deviation are calculated, the second tableware suspicion degree is obtained by fusing the first deviation and the second deviation.

[0064] Specifically, the formula for calculating the second tableware suspicion degree by using the first deviation and the second deviation can be expressed as: Where, Indicates the suspicion level of the second tableware; Indicates the first deviation; Indicates the second deviation.

[0065] in the formula The larger the portion, the more the pixel deviates from the main background color of the image in color, which means that the pixel has a significant color and is more likely to belong to the pattern on the tableware, which makes the second tableware more suspicious.

[0066] S5: Obtain a grayscale image of the enhanced image, and obtain a third tableware suspicion degree of each pixel based on the grayscale of each pixel.

[0067] Obtain the mean grayscale value of each pixel in the neighborhood corresponding to the pixel; obtain the grayscale difference based on the difference between the grayscale value of the pixel and the mean grayscale value of the pixels in the corresponding neighborhood, and use the mean grayscale difference as the third tableware suspicion degree.

[0068] In this embodiment, the square of the difference between the grayscale value of a pixel and the mean grayscale value of pixels in its corresponding neighborhood is used as the grayscale difference. This improves the sensitivity of calculating the third tableware suspicion level while maintaining a positive grayscale difference for ease of subsequent calculations. The grayscale difference here represents the fluctuation in grayscale within the local area where the pixel is located. Greater grayscale fluctuation indicates greater grayscale variation, a closer match to the pattern characteristics, and, consequently, a higher third tableware suspicion level.

[0069] S6: Binarize the first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree according to a preset threshold value to obtain a first standard stain value, a second standard stain value, and a third standard stain value, and take the weighted sum of the first standard stain value, the second standard stain value, and the third standard stain value as the comprehensive suspicion degree.

[0070] The first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree are binarized based on a preset threshold value to obtain a first standard stain value, a second standard stain value, and a third standard stain value, and a weighted sum of the first standard stain value, the second standard stain value, and the third standard stain value is used as the comprehensive suspicion degree.

[0071] The preset thresholds include a first threshold, a second threshold, and a third threshold.

[0072] For the first tableware suspicion level, 1 is set as the value of the first standard stain value in response to the first tableware suspicion level being less than the first threshold, and otherwise 0 is set as the value of the first standard stain value.

[0073] In this embodiment, the first threshold is an empirical value within the range of [0.75, 0.85]. In this embodiment, the first threshold is set to 0.8. Clean tableware has a smooth surface and strong specular reflectivity, with a reflectance typically ranging from 0.9 to 1.1. Therefore, in this embodiment, the first threshold is set to 0.8. For pixels whose first tableware suspicion score is less than the first threshold, the likelihood that the pixel is a stained area is greater, so 1 is used as the first standard stain value for that pixel.

[0074] For the second tableware suspicion level, 1 is set as the value of the second standard soil value in response to the second tableware suspicion level being less than the second threshold, and otherwise 0 is set as the value of the second standard soil value.

[0075] The calculation formula for the second threshold is: Where, represents the second threshold, Indicates the adjustment factor, which is set by the staff and can be initially set to 1.5. It is mainly used to set parameters for active intervention by the staff; Represents the standard deviation of the red and green channel values ​​of each pixel in the image; Represents the standard deviation of the yellow and blue channel values ​​of each pixel in the image.

[0076] In this method, a dynamic threshold, namely the second threshold, is constructed; the second threshold can change dynamically following the changes in the image, thereby improving the robustness of the detection of different types of tableware.

[0077] The principle of assigning the second standard stain value is the same as that of assigning the first standard stain value, and will not be repeated here.

[0078] For the third tableware suspicion level, 1 is set as the value of the third standard stain value in response to the third tableware suspicion level being less than the third threshold value, and otherwise 0 is set as the value of the third standard stain value.

[0079] The calculation formula of the third threshold can be expressed as: Where, represents the third threshold; represents the mean value of the suspicion of the third tableware in the image; Represents the adjustment index, with an initial value of 1.0, which can be adjusted according to the image complexity of the tableware; Indicates the standard deviation of the third tableware suspicion of each pixel in the image.

[0080] Tableware patterns have large grayscale variations, so a fixed threshold is not suitable. This formula dynamically adjusts the judgment threshold based on the actual distribution of image texture and has good generalization capabilities.

[0081] After the above steps, in an image, the first standard stain value, the second standard stain value, or the third standard stain value each has only two possibilities: 0 or 1. Thus, a first binary image, a second binary image, and a third binary image are formed, respectively. The first binary image corresponds to the first standard stain value, the second binary image corresponds to the second standard stain value, and the third binary image corresponds to the third standard stain value.

[0082] The first standard stain value, the second standard stain value, and the third standard stain value of the pixel point at the same position are weightedly fused based on the same weight to obtain the comprehensive suspicion.

[0083] Specifically, the calculation formula of the comprehensive weight can be expressed as:

[0084] Where, Indicates the comprehensive suspicion of the pixel; represents the first weight; represents the second weight; Indicates the first standard stain value corresponding to the pixel; Indicates the second standard stain value corresponding to the pixel; In this embodiment, the weights of the first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree are the same, that is, the first weight is equal to the second weight and the third weight is equal to. .

[0085] S7: Marking the stain area in the tableware image based on the comprehensive suspicion of each pixel point.

[0086] A stain threshold is set. When the pixel's overall suspicion score exceeds the threshold, the pixel's overall suspicion score is set to 1; otherwise, it is set to 0. This yields a fused binary image corresponding to the overall suspicion score. Dilation and erosion operations are then performed on the fused binary image to improve the connectivity of the image's contours. Connected regions are then merged and holes are filled. The processed fused binary image is then mapped back to the original image, and its borders are annotated to complete the tableware stain detection.

[0087] An embodiment of the present application also discloses a tableware surface dirt detection system based on image recognition, including a processor and a memory, wherein the memory stores computer program instructions. When the computer program instructions are executed by the processor, the tableware surface dirt detection method based on image recognition according to the present application is implemented.

[0088] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface. The configuration and functions of these components are known in the art and will not be described in detail here.

[0089] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A tableware surface dirt detection method based on image recognition, characterized in that: Acquire a tableware image, and obtain a first tableware suspicion degree of each pixel in the tableware image as a tableware region based on the brightness and saturation of each pixel in the tableware image in the HSI color space; perform reflection suppression preprocessing on the tableware image to obtain an enhanced image; Obtaining a second tableware suspicion degree where the pixel is a tableware area based on the chromaticity of each pixel in the enhanced image in the LAB color space; A grayscale image of the enhanced image is obtained, and a third tableware suspicion degree of each pixel is obtained based on the grayscale of each pixel; the first tableware suspicion degree, the second tableware suspicion degree, and the third tableware suspicion degree are binarized according to a preset threshold value to obtain a first standard stain value, a second standard stain value, and a third standard stain value; a weighted sum of the first standard stain value, the second standard stain value, and the third standard stain value is used as a comprehensive suspicion degree, and a stained area is marked in the tableware image based on the comprehensive suspicion degree of each pixel.

2. The tableware surface dirt detection method based on image recognition according to claim 1, characterized in that: The step of obtaining a first tableware suspicion degree for each pixel in the tableware image as a tableware area based on the brightness and saturation of each pixel in the HSI color space includes: obtaining the brightness level of the neighborhood of each pixel, and taking the ratio of the brightness of the pixel to the average brightness of the corresponding neighborhood as the reflectance of the pixel; calculating a brightness compensation value based on the saturation of each pixel, where the saturation is positively correlated with the brightness compensation value; and taking the sum of the brightness compensation value and the reflectance as the first tableware suspicion degree.

3. The tableware surface dirt detection method based on image recognition according to claim 2, characterized in that: The average brightness value of all pixels in the pixel neighborhood is taken as the brightness level of the pixel neighborhood.

4. The tableware surface dirt detection method based on image recognition according to claim 2, characterized in that: The step of calculating the brightness compensation value based on the saturation of each pixel includes: setting an adjustment coefficient, and taking the product of the normalized result of the pixel saturation and the adjustment coefficient as the brightness compensation value.

5. The tableware surface dirt detection method based on image recognition according to claim 2, characterized in that: Set the adjustment coefficient and enhancement coefficient, use the product of the enhancement coefficient and the normalized result of the pixel saturation as the enhancement degree, use the product of the enhancement degree and the adjustment coefficient as the adjustment index, and use the normalized result of the adjustment index and the saturation as the brightness compensation value.

6. The tableware surface dirt detection method based on image recognition according to claim 1, characterized in that: The step of obtaining a second tableware suspicion degree indicating that the pixel is a tableware area based on the chromaticity of each pixel in the enhanced image in the LAB color space includes: taking the average of the red and green channel values ​​of each pixel in the enhanced image as the first chromaticity level of the enhanced image; taking the average of the yellow and blue channel values ​​of each pixel in the enhanced image as the second chromaticity level of the enhanced image; for each pixel, taking the absolute difference between the red and green channel values ​​of the pixel and the first chromaticity level as the first deviation, taking the absolute difference between the yellow and blue channel values ​​of the pixel and the second chromaticity level as the second deviation, and fusing the first deviation and the second deviation to obtain the second tableware suspicion degree.

7. The tableware surface dirt detection method based on image recognition according to claim 1, characterized in that: The step of obtaining the third tableware suspicion degree of each pixel point based on the grayscale of each pixel point includes: obtaining the average grayscale value of the pixels in the neighborhood corresponding to each pixel point; obtaining the grayscale difference based on the difference between the grayscale value of the pixel point and the average grayscale value of the pixels in the corresponding neighborhood, and taking the average of the grayscale differences as the third tableware suspicion degree.

8. The tableware surface dirt detection method based on image recognition according to claim 1, characterized in that: The step of fusing the first deviation and the second deviation to obtain the second tableware suspicion level includes: taking the sum of the squares of the first deviation and the second deviation as the total deviation, and taking the arithmetic square root of the total deviation as the second tableware suspicion level.

9. The tableware surface dirt detection method based on image recognition according to claim 1, characterized in that: The preset thresholds include a first threshold, a second threshold and a third threshold; the steps of binarizing the first tableware suspicion degree, the second tableware suspicion degree and the third tableware suspicion degree include: for the first tableware suspicion degree, in response to the first tableware suspicion degree being less than the first threshold, 1 is used as the value of the first standard stain value, otherwise 0 is used as the value of the first standard stain value; for the second tableware suspicion degree, in response to the second tableware suspicion degree being less than the second threshold, 1 is used as the value of the second standard stain value, otherwise 0 is used as the value of the second standard stain value; for the third tableware suspicion degree, in response to the third tableware suspicion degree being less than the third threshold, 1 is used as the value of the third standard stain value, otherwise 0 is used as the value of the third standard stain value.

10. A tableware surface dirt detection system based on image recognition, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the tableware surface dirt detection method based on image recognition according to any one of claims 1 to 9 is implemented.

Images