Method and system for detecting residual stains on tableware after disinfection based on deep learning

Abstract

This invention relates to the fields of image processing and pattern recognition technology, specifically disclosing a method and system for detecting residual stains on tableware after disinfection based on deep learning. The invention first performs format conversion and size normalization on the initial image of the disinfected tableware, followed by illumination correction and filtering to enhance image quality. Next, it utilizes convolution and pooling operations combined with an attention mechanism to extract multi-level features, improving the model's ability to represent complex stains. A pre-trained deep learning model is used for forward propagation and threshold segmentation to generate stain detection results, and a visual report is generated based on transparency blending rendering technology. It also features an automatic alarm function; when the stain area exceeds a preset threshold, an alarm signal is immediately triggered and an alarm notification is generated. This invention effectively solves the problems of poor environmental adaptability and insufficient feature extraction in traditional detection methods, achieving unmanned, high-throughput, and standardized operation for tableware cleanliness detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing and pattern recognition technology, and relates to a method and system for detecting residual stains on tableware after disinfection based on deep learning. Background Technology

[0002] In the catering service and centralized tableware disinfection industry, ensuring the cleanliness and hygiene of tableware is a crucial aspect of protecting public health. Whether food residue, oil stains, water stains, or other contaminants remain on the surface of tableware after washing and high-temperature disinfection is a core indicator of disinfection quality. Image recognition technology, as a core branch of computer vision, simulates the human visual system to perceive and understand image content, providing a technical approach for automating and objectively evaluating the cleanliness of tableware.

[0003] The patent with publication number CN114764866A discloses a method and analyzer for analyzing the state of tableware contamination. The method uses a pre-trained deep neural network to analyze the state of tableware contamination. The camera of the tableware contamination analyzer takes pictures of tableware under ultraviolet light. The trained deep neural network can quickly and accurately distinguish whether the tableware surface is clean by judging whether the captured image is a dirty tableware image.

[0004] The patent with publication number CN118505680B discloses a tableware cleanliness detection method and system based on image processing, including: dividing a first detection image by region growing, removing the tableware area and conveyor belt area in the image, the remaining area is the area suspected of being stained, and further distinguishing based on grayscale thresholds based on the characteristics of stains and water stains to identify whether stains exist, and marking the stains in the image.

[0005] Compared with this invention, the existing technology has the following shortcomings: 1. It has stringent requirements for the detection environment and poor robustness. Some technologies are highly dependent on special physical environments, which increases hardware costs and deployment limitations. Moreover, it lacks an adaptive preprocessing mechanism for uneven lighting and sensor noise, and its anti-interference ability is weak under complex backgrounds and environmental changes; 2. The feature extraction and model generalization capabilities are insufficient. Existing methods mostly rely on traditional region growing algorithms or manually set fixed grayscale thresholds for differentiation, lacking deep semantic understanding of multi-level features and dynamic attention focusing mechanisms. When dealing with complex stains of different shapes, light colors, or extremely low contrast with the background of tableware, they are easily affected by water stains or reflections, resulting in missed detections and false alarms; 3. The detection results lack fine-grained quantification and intuitive visualization evaluation. Most existing technologies can only provide coarse-grained overall classification conclusions, and cannot achieve pixel-level accurate stain positioning, semi-transparent high-brightness rendering, and quantitative analysis based on precise area ratio. It is difficult to form an automated closed-loop management system from intuitive quality traceability to quantitative over-limit early warning. Summary of the Invention

[0006] In view of this, in order to solve the problems mentioned in the background technology, a method and system for detecting residual stains after tableware disinfection based on deep learning is proposed.

[0007] The objective of this invention can be achieved through the following technical solution: The first aspect of this invention provides a method for detecting residual stains after tableware disinfection based on deep learning, including: S1, acquiring an initial image of the tableware after disinfection, performing format conversion and size normalization processing on the initial image, and generating original image data.

[0008] S2. Perform illumination intensity analysis, illumination correction, and filtering and denoising processing on the original image data in sequence to generate preprocessed image data.

[0009] S3. Perform convolution and pooling operations on the preprocessed image data to extract low-level and mid-level features, and then use an attention mechanism to weighted fuse them to generate image feature data.

[0010] S4. Input the image feature data into the pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results.

[0011] S5. Based on the stain detection results and preprocessed image data, perform transparency blending rendering to generate a visual report.

[0012] S6. Encode the data in the visualization report and transmit it to the display device for decoding and output.

[0013] S7. Calculate the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, trigger an alarm signal and generate an alarm notification.

[0014] The second aspect of the present invention provides a deep learning-based system for detecting residual stains after tableware disinfection, comprising: an original image data generation module, which acquires an initial image of the tableware after disinfection, performs format conversion and size normalization processing on the initial image, and generates original image data.

[0015] The preprocessed image data generation module sequentially performs illumination intensity analysis, illumination correction, and filtering and denoising on the original image data to generate preprocessed image data.

[0016] The image feature data generation module performs convolution and pooling operations on preprocessed image data to extract low-level and mid-level features, and then uses an attention mechanism to weightedly fuse them to generate image feature data.

[0017] The stain detection result generation module inputs image feature data into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results.

[0018] The visualization report generation module generates a visualization report by performing transparency blending rendering based on stain detection results and preprocessed image data.

[0019] The visualization report output module encodes the visualization report data and transmits it to the display device for decoding and output.

[0020] The alarm signal triggering module calculates the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, an alarm signal is triggered and an alarm notification is generated.

[0021] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects: (1) The present invention effectively eliminates the interference of uneven lighting and image noise in the shooting environment by performing illumination correction and filtering denoising on the original image, providing a high-quality image source for subsequent feature extraction and improving the initial signal-to-noise ratio of detection. Combined with the recognition ability of the subsequent deep learning model, the robustness of the entire detection system to environmental changes is enhanced, and stable detection performance can be maintained under different lighting conditions and under the background of tableware reflection, reducing the false alarm rate and false detection rate.

[0022] This invention proposes a feature extraction scheme that integrates low-level and mid-level features with an attention mechanism. Low-level feature maps retain detailed information such as the edges and textures of stains, while mid-level feature maps summarize the contours and shapes of stains. Adaptive weighted fusion via an attention mechanism allows the model to dynamically focus on the most discriminative features, thereby enhancing its ability to represent stains of varying shapes and sizes. This method improves the effectiveness and richness of features, laying a solid foundation for subsequent accurate identification.

[0023] This invention establishes a complete automated process from image acquisition, preprocessing, feature extraction, stain recognition to alarm notification. By comparing the stain detection results with preset thresholds, it automatically triggers alarm signals and subsequent physical sorting actions, while simultaneously generating visual reports and structured data records. This closed-loop system effectively replaces traditional manual visual inspection, achieving unmanned, high-throughput, and standardized operation of tableware cleanliness detection, improving production efficiency and quality control levels. Furthermore, the generated detection data provides a reliable basis for quality traceability and process optimization. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the method steps of the present invention.

[0026] Figure 2 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Please see Figure 1 The first aspect of the present invention provides a method for detecting residual stains after tableware disinfection based on deep learning, comprising: S1, acquiring an initial image of the tableware after disinfection, performing format conversion and size normalization processing on the initial image to generate original image data.

[0029] In a specific embodiment of the present invention, the initial image is processed by format conversion and size normalization to generate original image data, including: controlling the image acquisition device to photograph the surface of the tableware after disinfection to obtain the initial image.

[0030] The initial image is decoded and re-encoded into a standard format image using a format conversion function.

[0031] The scaling factor is calculated based on the preset target width and target height. The pixel coordinates of the standard format image are mapped to the target coordinate system using a resampling interpolation algorithm to generate original image data of fixed size.

[0032] Specifically, the process of acquiring surface images of sterilized tableware and generating raw image data begins with high-fidelity photography of the tableware using an image acquisition device. An image acquisition device is a device that converts optical images into electrical signals to record visual information about the tableware surface. This photography directly generates the initial image. The initial image is an unprocessed digital file directly output by the sensor of the image acquisition device, and its data format may be directly related to the device hardware.

[0033] Next, to ensure compatibility and consistency in subsequent processing, the initial image needs to be converted to a standard format image. A standard format image is an image file with a universal file structure that can be recognized and processed by various software systems, such as PNG or TIFF formats. This conversion process decodes the initial image from its device-specific encoding and re-encodes it to a universal standard. This conversion relationship can be expressed as: in, Represents the initial image. This represents a standard format image generated after format conversion. It is a format conversion function that reorganizes and encodes the pixel array, color space, and metadata of the initial image according to the target file format standard.

[0034] Finally, the standard format images are normalized to generate the original image data. Size normalization unifies standard format images obtained under different shooting conditions to a fixed size, thus providing consistent input for subsequent feature extraction and model analysis. This process generates the final original image data, which is a size-normalized image data matrix that can be directly used for subsequent analysis. Size normalization is achieved through a resampling algorithm, the core of which is mapping the pixel coordinates of the standard format images to a new coordinate system in the original image data. The coordinate mapping relationship is as follows: Furthermore, the pixel value of any point in the original image data can be obtained by interpolation calculation on a standard format image. The entire normalization operation can be expressed as: In this formula, Represents the generated original image data. This represents a standard format image input. These are the pixel coordinates in the original image data. This represents a resampling interpolation function, such as the bicubic interpolation algorithm, used to calculate pixel values ​​at non-integer coordinates. and These are the scaling factors for width and height, respectively. Width of standard format images target width of the original image data Calculations show that The height of a standard format image Target height relative to the original image data Calculated. Target width Altitude of the target The setting is based on statistical analysis of the effective surface area of ​​300 different types of tableware (including bowls, plates, and dishes), and is set to 1024 pixels. This setting can achieve a balance between preserving sufficient stain details and controlling computational resource consumption.

[0035] S2. Perform illumination intensity analysis, illumination correction, and filtering and denoising processing on the original image data in sequence to generate preprocessed image data.

[0036] In a specific embodiment of the present invention, the original image data is sequentially subjected to illumination intensity analysis, illumination correction and filtering denoising processing to generate preprocessed image data, including: using a two-dimensional Gaussian smoothing kernel function to perform a convolution operation on the original image data to estimate the background illumination generation illumination distribution parameters.

[0037] The illumination equalization coefficient is calculated based on the ratio of the global average value of the illumination distribution parameters to the local illumination intensity. The illumination equalization coefficient is then multiplied pixel by pixel with the original image data to generate the adjusted image.

[0038] The pixel values ​​of the adjusted image are linearly stretched to the target grayscale display range to generate the corrected image.

[0039] The local neighborhood window of the corrected image is denoised using a median filter to generate preprocessed image data.

[0040] Specifically, after obtaining the raw image data generated in step S1, image preprocessing is required to eliminate interference such as uneven lighting and sensor noise in the shooting environment, in order to generate clear and standardized preprocessed image data. This process first performs illumination intensity analysis on the raw image data, thereby generating illumination distribution parameters that quantify the overall illumination characteristics of the image. Illumination intensity analysis establishes a mathematical model describing illumination changes by evaluating the brightness information of each pixel in the image. In practice, this analysis process estimates the background illumination by applying a large-scale smoothing filter to the raw image data; the smoothed image is the illumination distribution parameter, which reflects the slowly changing illumination components in the image.

[0041] This process can be represented as: In this formula, Represents coordinates The pixel values ​​of the original image data. (Symbol) This represents a two-dimensional convolution operation. It is a two-dimensional Gaussian smoothing kernel function, which is used to blur images to extract low-frequency lighting background information. It is generated in coordinates The illumination distribution parameters at a given location are essentially a background illumination estimate map with the same dimensions as the original image data. The standard deviation of the Gaussian kernel is set based on the statistical results of 500 sets of tableware images containing stains of different sizes, with a set value of 50. This value ensures that stain details are smoothed out while only background illumination information is preserved.

[0042] Next, illumination correction is performed based on the generated illumination distribution parameters to produce a corrected image. The goal of illumination correction is to compensate for uneven illumination, making the background brightness of the image more uniform. This correction process first calculates an illumination equalization coefficient based on the illumination distribution parameters. The illumination equalization coefficient is a multiplicative correction factor applied to each pixel, used to adjust the brightness of local areas to the average level of the entire image. Its calculation method is as follows: in, In coordinates The illumination uniformity coefficient at that location. Light distribution parameters The average value across the entire image represents the target's balanced brightness. This is the local illumination intensity at that coordinate point. To prevent... When the denominator approaches zero, we introduce a very small positive number. Based on the experimental analysis of 1000 sets of images under different lighting conditions, Setting it to 1 effectively avoids calculation overflow while having a negligible impact on the result. This formula does not involve physical dimensions and is a purely numerical calculation.

[0043] After calculating the illumination equalization coefficient, it is applied to the pixel values ​​of the original image data to generate an adjusted image with adjusted pixel values. The adjustment process involves point-by-point multiplication. in, Represents the adjusted image in coordinates The pixel values ​​at that location. Through this step, the pixel values ​​of darker areas in the original image data are increased, and the pixel values ​​of brighter areas are decreased, thereby achieving equalization of illumination.

[0044] After equalizing the lighting, to further highlight the difference between the stains and the background of the tableware, the adjusted image needs to undergo contrast enhancement processing to generate a corrected image. Contrast enhancement is achieved by linearly stretching the pixel values ​​of the image to the entire display grayscale range, making the brightness differences between different objects in the image clearer. The processing formula is as follows: in, The generated corrected image in coordinates The pixel value at that location. and These are the adjusted images. The minimum and maximum pixel values ​​in the range. This is the maximum possible value for the image data type; for an 8-bit grayscale image, it is set to 255. This operation will adjust the grayscale range of the resulting image from... Mapped to This improves the visual contrast of the image.

[0045] Finally, the corrected image undergoes filtering and denoising to eliminate random noise introduced during image acquisition or processing, ultimately generating preprocessed image data. This step employs a median filter, which works by replacing the value of each pixel with the median value of its neighboring pixels. This effectively removes isolated noise points such as salt-and-pepper noise while preserving the edge details of the image. This operation can be represented as: in, That is, the final generated preprocessed image data in coordinates The pixel value at that location. Indicated by The center is a neighborhood window, with a size set to 3x3 pixels. This is a common setting that strikes a balance between noise reduction and detail preservation. Its effectiveness has been verified by testing 200 sets of industrial sensor images.

[0046] S3. Perform convolution and pooling operations on the preprocessed image data to extract low-level and mid-level features, and then use an attention mechanism to weighted fuse them to generate image feature data.

[0047] In a specific embodiment of the present invention, convolution and pooling operations are performed on preprocessed image data to extract low-level and mid-level features, and weighted fusion is performed through an attention mechanism to generate image feature data, including: performing convolution operations on preprocessed image data using multiple convolution kernels and applying a nonlinear activation function to generate a low-level feature map.

[0048] Max pooling is performed on the low-level feature map to generate the mid-level feature map.

[0049] The low-level feature map is concatenated with the upsampled mid-level feature map along the channel dimension, input into the convolutional layer, and the sigmoid function is applied to generate the attention weight matrix.

[0050] The low-level feature map is weighted using an attention weight matrix, and the upsampled mid-level feature map is weighted using complementary weights of the attention weight matrix. The two are then added element by element to generate a fused feature map.

[0051] The fused feature map is subjected to layer normalization in the channel dimension to generate image feature data.

[0052] Specifically, after generating preprocessed image data in step S2, in order to extract effective information from the image that can distinguish stains from the background of tableware, feature extraction needs to be performed on the preprocessed image data to generate image feature data for subsequent recognition. This process begins with a convolution operation on the preprocessed image data, aiming to extract low-level features and generate low-level feature maps. A convolution operation is a process that uses a small, learnable filter (convolution kernel) to slide across an image to detect local patterns. Low-level features refer to the most basic constituent elements of an image, such as edges, corners, color blocks, and textures. Through this operation, the original pixel value matrix is ​​transformed into a series of low-level feature maps, where each map corresponds to the response intensity of a specific low-level feature in the original image.

[0053] The mathematical expression for this convolution operation is as follows: In this formula, Indicates the first A low-level feature map in coordinates The value at that location. It is a non-linear activation function, such as a modified linear unit, which introduces non-linearity, enabling the model to learn more complex features. The input preprocessed image data in the channel ,coordinate The pixel value at that location. It is the first Each convolutional kernel in the channel ,Location The weight value at that location. Is with the first The bias term associated with each convolution kernel. The number of convolution kernels. and size It is pre-set, based on training experience with 10,000 samples of tableware stains, and sets 32 convolutional kernels with a size of 3x3 pixels, which can effectively capture details while controlling computational complexity.

[0054] After extracting the low-level feature map, pooling is performed to reduce dimensionality and generalize the features, thereby extracting mid-level features and generating a mid-level feature map. Pooling reduces the spatial size of the feature map, making the features more robust to small positional changes. Mid-level features are more discernible local shapes or textures composed of combinations of low-level features. The mid-level feature map generated by this operation is smaller in size but contains higher-level semantic information. Max pooling is a commonly used technique in this operation, which selects the maximum value within a local region of the feature map as the output.

[0055] The process can be described as follows: in, It is the first The intermediate feature map is in coordinates The value at that location. The function represents the operation of finding the maximum value. Indicates output coordinates Correspondingly, in the input low-level feature map A local window on top. The window size and stride are set to 2x2 pixels and 2, respectively. This setting halves the feature map size, which is a common industry practice to balance reducing computational cost and preserving feature information.

[0056] Subsequently, the low-level feature map containing fine texture and the mid-level feature map containing contour information are fused together and weighted using an attention mechanism to generate the final image feature data. Attention weighting is a process that simulates human visual attention, allowing the system to dynamically assign different importance weights to different features, thereby focusing on the image region most relevant to the stain. This process first requires calculating the weight distribution of the low-level and mid-level feature maps to generate attention weights. The attention weights are a weight matrix of the same size as the feature maps, and their values ​​reflect the importance of the features at the corresponding locations. To achieve size matching, the mid-level feature map needs to be upsampled to the same size as the low-level feature map.

[0057] The attention weights are calculated as follows: In this formula, This represents the generated attention weights. It is the Sigmoid function, which maps any real value to the (0,1) interval and is used to generate gated weights. This represents a 1x1 convolutional layer used to integrate and transform features along the channel dimension. This indicates that the low-level feature map With upsampling function Processed intermediate feature map Concatenation is performed along the channel dimension. Upsampling function. This can be achieved using bilinear interpolation.

[0058] After obtaining the attention weights, the feature maps are weighted and fused according to these weights to generate a fused feature map. Weighted fusion is achieved by applying the attention weights to the original feature maps, which enhances important features and suppresses unimportant features.

[0059] in, It is the generated fused feature map. This represents element-wise multiplication. It involves using attention weights. With low-level feature maps Multiply them and simultaneously apply their complementary weights. Compared with the upsampled intermediate feature map By multiplying and then adding, an adaptive fusion of the two features is achieved.

[0060] It's important to note that the 'complementary weights' here refer to a weight distribution that forms a mutually exclusive and complementary relationship with the attention weight matrix in the numerical space. Specifically, this is achieved by constructing a one-dimensional matrix with the same dimensions as the attention weight matrix and all elements equal to 1. Then, this one-dimensional matrix is ​​used to subtract the current attention weight matrix element by element. The resulting complementary weight matrix has the following characteristics: a position assigned a high weight (close to 1, indicating that lower-level features are more critical) in the attention weight matrix is ​​naturally assigned a low weight (close to 0, suppressing mid-level features) in the complementary weight matrix; and vice versa. This complementary gating mechanism, based on '1 minus the current weight', mathematically forces the system to dynamically and smoothly allocate computational power between extremely fine textures (low-level features) and macroscopic dirt outlines (mid-level features), avoiding information redundancy and conflicts during feature fusion.

[0061] Finally, the fused feature maps are normalized to stabilize the data distribution and generate the final image feature data. This step uses layer normalization technology to standardize the features of all channels of a single sample.

[0062] In this formula, It is the value of the final generated image feature data. It is the value at the corresponding position in the fused feature map. and These are the mean and variance of the features of the sample across all channels, respectively. and These are learnable scaling and offset parameters used to maintain the expressiveness of the model. It is a very small constant, such as 1e-5, used to prevent the denominator from being zero.

[0063] It should be noted that, in further detail, a learnable scaling parameter is introduced during this normalization process. and offset parameters The underlying mechanism and necessity of this approach. If only the mean is subtracted from the fused feature map... The pure standardization operation of dividing by the standard deviation can force the feature data distribution to be stable in a standard normal distribution with a mean of 0 and a variance of 1, thereby accelerating the convergence speed during model training. However, this forced linear transformation will change the activation distribution of the original features of the network, which can easily destroy or lose the non-linear feature information that the previous convolutional layer has worked hard to extract. For example, the weak edge features of some light water stains themselves need to be represented by extreme values ​​that deviate from the mean.

[0064] To solve this problem, the following was introduced: and These two parameters, like the weights of the convolutional kernel, are parameters that are continuously optimized and updated through the backpropagation algorithm during the training of the neural network. After the features are forcibly standardized, they are multiplied by... And add This essentially gives the neural network a 'repentance' mechanism. That is, if the neural network determines during the learning process that the original unnormalized feature distribution is optimal for identifying a specific stain, it can learn to adjust the scaling parameters. Standard deviation approaching the original feature And offset parameter Approaching the mean of the original features This effectively restores the original features before normalization; conversely, if the standardized features are better, the network will learn other features. and This mechanism, while enjoying the training stability brought by normalization, preserves to the greatest extent the nonlinear expressive and generalization capabilities of deep learning models for extremely complex tableware backgrounds and multi-form residual stains.

[0065] S4. Input the image feature data into the pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results.

[0066] In a specific embodiment of the present invention, image feature data is input into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results, including: obtaining a deep learning model trained on a historical tableware image annotation dataset using a binary cross-entropy loss function and a backpropagation algorithm.

[0067] Image feature data is input into a deep learning model for forward propagation calculation, and the output is the initial detection output representing the probability that a pixel belongs to a stain.

[0068] Pixel values ​​with a probability value greater than a preset judgment threshold in the initial detection output are set as the first identifier value, and pixel values ​​with a probability value less than or equal to the preset judgment threshold are set as the second identifier value, thus generating a binarized stain detection result.

[0069] Specifically, after obtaining image feature data containing rich stain information through step S3, stain recognition needs to be performed using a deep learning model based on the image feature data to generate the final stain detection result. In this embodiment, the deep learning model is essentially a feature decoding and upsampling network. Since the image feature data generated in step S3 has been reduced in spatial resolution by pooling operations, in order to generate pixel-level prediction outputs that completely correspond to the original input image size, the deep learning model adopts the following specific network structure and forward propagation method: First, the image feature data is input into consecutive deconvolution layers. This embodiment uses two layers of deconvolution operations with a stride of 2 and a kernel size of 4x4. Each deconvolution operation is followed by batch normalization and ReLU activation functions. This operation can not only gradually enlarge the spatial resolution of the feature map to restore it to the size of the original preprocessed image data, but also adaptively learn the spatial mapping relationship during the enlargement process to repair the edge details lost due to pooling. Subsequently, the upsampled and resized feature map is input into a classification mapping layer. This layer consists of a standard 1x1 convolutional layer with its output channel set to 1 (corresponding to single-channel probability prediction). Finally, a sigmoid activation function is applied to the output of this 1x1 convolutional layer, compressing all feature values ​​to the range of 0 to 1. The resulting two-dimensional matrix represents the initial detection output indicating the probability that a pixel belongs to a stain. Each element in the matrix corresponds precisely to a pixel in the image, and its value directly reflects the confidence probability that the pixel belongs to the stain region.

[0070] The first step in this process is to acquire a trained deep learning model. A trained deep learning model is one whose internal parameters have been optimized through a learning process, enabling it to make accurate predictions for a specific task (in this case, stain recognition). The model acquisition process itself involves several sub-steps, beginning with collecting historical tableware image samples to generate a training dataset. These historical tableware image samples contain images of clean and stained tableware under various lighting conditions. These samples are then aggregated to form a large-scale, structured training dataset of 50,000 images—an industry-standard size designed to ensure the model learns sufficient generalization ability.

[0071] Subsequently, the training dataset is annotated to generate labeled data. Annotation involves precisely marking the actual location and outline of the stain in each image of the training dataset, forming a corresponding "true answer" mask. The generated labeled data has the same size as the original images, where each pixel is assigned a value of 1 (representing a stain) or 0 (representing the background), providing a supervisory signal for model training.

[0072] Once labeled data is available, it can be used to train an initial neural network model. Parameters are optimized through backpropagation to generate a trained deep learning model. The initial neural network model is a model with randomly initialized parameters, and its predictive ability is essentially zero. The training process is an iterative optimization loop. In each iteration, the model processes images from the training dataset and outputs a prediction. This prediction is compared to the labeled data, and the difference between the two is calculated; this difference is quantified by a loss function. For pixel-level binary classification tasks, a binary cross-entropy loss function is used. In this formula, This represents the calculated loss value; the smaller the value, the more accurate the model prediction. It represents the total number of pixels in the image. It is the first The true value (0 or 1) of each pixel in the labeled data. The model is for the first Each pixel is predicted to be a stain.

[0073] After calculating the loss value, the backpropagation algorithm is initiated. Backpropagation is a method that calculates the gradient (i.e., the degree of influence) of the loss value on each parameter in the model, and then fine-tunes these parameters in the opposite direction of the gradient in order to reduce the loss in the next iteration. The parameter update rule can be expressed as: in, and These represent the model parameters before and after the update, respectively. It is the learning rate, which controls the step size of each parameter update. Its value is set to 0.001 based on experimental tuning to ensure stable convergence. It is a loss function Relative to parameters The gradient is calculated. By repeatedly performing this process on the entire training dataset, the model parameters are continuously optimized until the loss value converges to a very low level, at which point a well-trained deep learning model is obtained.

[0074] It should be noted that a further detailed explanation is needed regarding the core driving force of the backpropagation algorithm—the gradient of the loss function. The mathematical essence of this concept and its physical significance in model optimization. During the training phase of a deep learning model, there will inevitably be an error between the model's predicted values ​​of stain pixels in a tableware image and the manually labeled true values. This total error is determined by the binary cross-entropy loss function. To quantify. Essentially, it is a set of vectors containing multidimensional partial derivatives. Using the chain rule in calculus, it works backward from the output layer of the model to calculate every single parameter among the millions within the entire model. (Including the weights and biases of all convolutional kernels) for the final total loss The degree of marginal influence. In a multidimensional parametric space geometric model, the gradient vector Always point in the direction where the loss function value increases the fastest (i.e., the steepest). Therefore, in the parameter update formula... In this context, the operation of 'subtracting' the gradient (i.e., gradient descent) is used, and its physical meaning lies in driving each parameter of the model. Make tiny movements in the opposite direction to the current direction that causes the most drastic increase in error. Through continuous input and iterative calculation of thousands of batches of tableware image data, Like a compass, it precisely guides the parameter set of the entire massive neural network toward the loss function. The evolution of global or local minima ultimately enables the model to have a high-level discrimination ability to accurately separate and identify fine dirt pixels from extremely similar background reflections.

[0075] Once the trained deep learning model is obtained, real-time stain recognition can be performed. The image feature data generated in step S3 is input into the trained deep learning model for forward propagation calculation to generate the initial detection output. Forward propagation calculation means that the data starts from the input layer of the model and flows through all neural network layers in sequence. Each layer calculates and transforms the input data, and finally the output layer produces the result. This result is the initial detection output, which is a probability map of the same size as the input image feature data. The value of each pixel in the map is between 0 and 1, representing the probability that the point belongs to a stain.

[0076] In this formula, This represents the initial detection output. This represents the entire trained deep learning model. It is the input image feature data. This represents all parameters in the model that have been optimized and fixed.

[0077] Finally, since the initial detection output is a probability map, it needs to be converted into a definitive detection conclusion. Therefore, threshold segmentation is performed on the initial detection output to generate stain detection results. Threshold segmentation sets a threshold, and the probability value of each pixel in the probability map is compared with this threshold. If the probability is greater than the threshold, the pixel is determined to be a stain; otherwise, it is determined to be background.

[0078] in, The final generated stain detection result is in coordinates The value at this location indicates that a stain was detected (1) or not (0). It is the probability value of the initial detection output at this coordinate point. This is a preset threshold, set based on tests conducted on an independent validation set containing 5000 images. The threshold value was chosen to optimize the combined precision and recall, and was set to 0.5. The final stain detection result is a binary image that clearly marks all identified stain areas.

[0079] S5. Based on the stain detection results and preprocessed image data, perform transparency blending rendering to generate a visual report.

[0080] In a specific embodiment of the present invention, a visualization report is generated by performing transparency blending rendering based on the stain detection results and preprocessed image data, including: obtaining the preset stain area transparency coefficient and the overlay color value.

[0081] When the pixel value of the stain detection result is the first identifier value indicating the stain, the color value of the overlay layer and the corresponding pixel color value of the preprocessed image data are mixed by weighted average according to the stain area transparency coefficient.

[0082] When the pixel value of the stain detection result is a second identifier value indicating the background, the original pixel color value of the preprocessed image data is retained.

[0083] Iterate through all pixels to complete color blending and generate a visual report.

[0084] Specifically, after generating the stain detection results indicating the location of the stains in step S4, in order to present the detection information to the user in an intuitive and easy-to-understand way, the stain detection results need to be visualized and rendered to generate a visually appealing report with both text and images. Visualization and rendering is a process of converting abstract binary data into images that are easily understood by the human visual system. This process uses the preprocessed image data generated in step S2 as the visual base and the stain detection results generated in step S4 as the information layer. The two are superimposed using color mixing technology, thereby highlighting the detected stain areas on the original tableware image.

[0085] The core of this rendering process is to selectively overlay colors onto each pixel of the preprocessed image data based on the stain detection results. This is achieved through a technique called transparency blending, which works by overlaying a semi-transparent color (highlight color) onto the background image. The color of each pixel in the final visualization report is the result of a weighted average of the highlight color and the original background color, calculated using a transparency factor. This transparency factor is directly determined by the stain detection results.

[0086] The specific visualization rendering calculation formula is as follows: In this formula, The final generated visualization report is located at coordinates. The pixel color value at that location. This is the value of the stain detection result at this coordinate point. A value of 1 indicates that a stain has been detected, while a value of 0 indicates that the background has been detected. This is a preset transparency factor for the stained area, ranging from 0 to 1, controlling the opacity of the brighter areas. Based on visual acuity tests conducted on 100 different users, [the following is likely a separate, unrelated sentence:] This will... The value is set to 0.6, which allows users to observe the original texture of the covered area while clearly marking the stain. This is the overlay color used for highlighting. To serve as a warning, this color is set to red, and its value in the 8-bit RGB color space is (255,0,0). This represents the pixel color value at the corresponding coordinate point in the preprocessed image data, used as the background for rendering. This formula is a dimensionless color space calculation. When the value is 1, the final color of this pixel is a mixture of 60% red and 40% of the original image color. When When it is 0, the transparency item When the value becomes 0, the formula simplifies to This means that the pixel retains the original image color.

[0087] By performing this calculation on every pixel in the image, the resulting visualization is a color image of the same size as the original image, with the background of the tableware remaining unchanged, while all areas detected as stains are covered with a semi-transparent red layer, thus clearly presenting the detection results.

[0088] S6. Encode the data in the visualization report and transmit it to the display device for decoding and output.

[0089] In a specific embodiment of the present invention, the visualization report is encoded and transmitted to a display device for decoding and output, including: packaging the pixel matrix and metadata of the visualization report, serializing them into a continuous data stream, and then encoding the data.

[0090] The encoded data stream is transmitted to the display device via a communication link.

[0091] The display device's graphics controller decodes the data stream, restores it to the original pixel color matrix, and writes it into the frame buffer.

[0092] The system reads data from the frame buffer at a preset refresh rate and drives the display panel to present the image of the visualization report.

[0093] Specifically, after generating a visually intuitive report in step S5, in order to effectively convey the detection information to operators or higher-level systems, the visual report needs to be transmitted to a display device for output. This process involves sending the pixel data matrix representing the visual report in memory to a display device capable of converting electrical signals into optical images via a specific data communication link.

[0094] First, the system encodes the visualization report generated in step S5. The visualization report is essentially a two-dimensional matrix of pixel color values. To ensure data integrity and efficiency during transmission, it needs to be serialized into a continuous data stream. This encoding process follows standard image file formats, such as lossless PNG, packaging the pixel matrix and metadata into a single data packet.

[0095] Next, the encoded data stream is transmitted to the display device via a physical or wireless channel. The display device is a human-machine interface terminal, such as an industrial touchscreen or monitoring monitor, whose built-in graphics processing unit is responsible for receiving and parsing the incoming data. The performance of data transmission directly affects the real-time nature of the report display; its transmission time can be determined by the following formula: In this formula, This represents the theoretical time required to transfer a complete visualization report from the processing unit to the display device. It is the total amount of data in the encoded visualization report, and its size depends on the image resolution and color depth. This refers to the bandwidth of the communication link between the processing unit and the display device, i.e., the amount of data that can be transmitted per unit of time. For example, in an industrial environment using a gigabit Ethernet connection, the theoretical bandwidth is set at 100 kilobytes per second. Bits, this setting is the current standard for industrial network deployments.

[0096] After receiving the complete data stream, the display device's internal graphics controller performs a decoding operation, restoring the data stream to the original pixel color matrix. This matrix is ​​then written into the display device's frame buffer. The frame buffer is a dedicated area in video memory used to store single-frame image information that will be displayed on the screen.

[0097] Finally, the display controller of the display device periodically reads data from the frame buffer at a preset refresh rate, and drives the pixels on the display panel to emit light of corresponding color and brightness based on this data, thereby presenting the image of the visual report on the screen. This process completes the final conversion from digital information to a visual image, realizing the output.

[0098] S7. Calculate the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, trigger an alarm signal and generate an alarm notification.

[0099] In a specific embodiment of the present invention, the stain area ratio in the stain detection result is calculated. When the stain area ratio is greater than the preset stain area ratio threshold, an alarm signal is triggered and an alarm notification is generated, including: counting the total number of pixels identified as stains in the stain detection result and the total number of pixels of the tableware body in the preprocessed image data.

[0100] The ratio of the total number of pixels in the stain to the total number of pixels in the main body of the tableware is calculated to obtain the stain area percentage.

[0101] When the percentage of the stain area is greater than the stain area percentage threshold, the alarm status is considered valid.

[0102] In response to a valid alarm status, the voltage of the digital output port is switched to a high level to trigger a physical alarm signal and generate a structured alarm notification that includes a timestamp, tableware identifier, stain area percentage, and a visual report storage path.

[0103] Specifically, after obtaining the stain detection results through step S4 and completing visualization and output through steps S5 and S6, the system needs to trigger an alarm signal based on the quantitative analysis of the stain detection results and generate a corresponding alarm notification to achieve automated quality control. This process aims to determine whether the detected stains exceed acceptable standards and to initiate a physical or digital response when the standards are exceeded.

[0104] The process begins with a quantitative assessment of the stain detection results. The stain detection result is a binary image, where each pixel has a value of either 1 or 0. To determine the severity of the stain, the system calculates the total number of pixels identified as stains and compares this number to the total pixel area of ​​the tableware itself, resulting in a stain area percentage. This percentage is the key basis for triggering an alarm. The alarm trigger condition can be defined by the following logical expression: In this expression, This represents the alarm status. A value of 1 indicates that the alarm has been triggered, while a value of 0 indicates that it has not been triggered. The stain detection results are on the coordinates The value at that location. Therefore, the molecule This calculates the total number of pixels in the stained area. The denominator... It is the preprocessed image data generated in step S2, function It is a binarization function, when the pixel value Returns 1 if the value is greater than the background threshold (e.g., 0), otherwise returns 0. Therefore, the denominator... It calculates the total number of pixels of the main tableware in the image. This is a preset threshold for the percentage of stained area. The threshold is set based on food safety and hygiene standards and a consumer acceptance survey of 500 sets of tableware with varying degrees of stains. It is set at 0.005, meaning that when the stained area exceeds 0.5% of the total area of ​​the tableware, it is considered unqualified.

[0105] Once alarm status When the value is set to 1, the system will immediately generate and issue an alarm signal. An alarm signal is a physical-level electrical signal, typically generated by the system's digital output port. This signal is sent to external actuators, such as audible and visual alarms or robotic arms integrated into the production line. For example, the system can switch the voltage of a general-purpose input / output (GPIO) pin from low (0 volts) to high (24 volts DC); this voltage change serves as the alarm signal, sufficient to drive a relay or PLC (Programmable Logic Controller) module, thereby controlling the alarm device.

[0106] Simultaneously, the system generates a structured alarm notification. This alarm notification is a digital record detailing the alarm event. It includes the timestamp of the event, the unique identifier of the inspected tableware, the calculated percentage of the actual stained area, and a link to the storage path of the visualization report generated in step S5. This alarm notification is sent to the central monitoring system or written to a local database via a network protocol (such as TCP / IP) for subsequent quality traceability and statistical analysis.

[0107] Reference Figure 2 The second aspect of the present invention provides a deep learning-based system for detecting residual stains after tableware disinfection, comprising: a raw image data generation module, a preprocessed image data generation module, an image feature data generation module, a stain detection result generation module, a visualization report generation module, a visualization report output module, and an alarm signal triggering module.

[0108] The raw image data generation module is connected to the preprocessed image data generation module, the preprocessed image data generation module is connected to the image feature data generation module, the image feature data generation module is connected to the stain detection result generation module, both the preprocessed image data generation module and the stain detection result generation module are connected to the visualization report generation module, the visualization report generation module is connected to the visualization report output module, and the stain detection result generation module is connected to the alarm signal triggering module.

[0109] The raw image data generation module acquires the initial image of the tableware after disinfection, performs format conversion and size normalization on the initial image, and generates raw image data.

[0110] The preprocessed image data generation module sequentially performs illumination intensity analysis, illumination correction, and filtering and denoising on the original image data to generate preprocessed image data.

[0111] The image feature data generation module performs convolution and pooling operations on preprocessed image data to extract low-level and mid-level features, and then uses an attention mechanism to weightedly fuse them to generate image feature data.

[0112] The stain detection result generation module inputs image feature data into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results.

[0113] The visualization report generation module generates a visualization report by performing transparency blending rendering based on stain detection results and preprocessed image data.

[0114] The visualization report output module encodes the visualization report data and transmits it to the display device for decoding and output.

[0115] The alarm signal triggering module calculates the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, an alarm signal is triggered and an alarm notification is generated.

[0116] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A method for detecting residual stains on tableware after disinfection based on deep learning, characterized in that, include: S1. Obtain the initial image of the tableware after disinfection, perform format conversion and size normalization on the initial image, and generate the original image data; S2. Perform illumination intensity analysis, illumination correction and filtering and noise reduction on the original image data in sequence to generate preprocessed image data; S3. Perform convolution and pooling operations on the preprocessed image data to extract low-level and mid-level features, and then use an attention mechanism to weighted fuse them to generate image feature data. S4. Input the image feature data into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results; S5. Based on the stain detection results and preprocessed image data, perform transparency blending rendering to generate a visual report; S6. Encode the visualization report and transmit it to the display device for decoding and output; S7. Calculate the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, trigger an alarm signal and generate an alarm notification.

2. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The process of format conversion and size normalization of the initial image to generate original image data includes: Control the image acquisition device to photograph the surface of the tableware after disinfection and obtain initial images; The initial image is decoded and re-encoded into a standard format image using a format conversion function; The scaling factor is calculated based on the preset target width and target height. The pixel coordinates of the standard format image are mapped to the target coordinate system using a resampling interpolation algorithm to generate original image data of fixed size.

3. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The process of sequentially performing illumination intensity analysis, illumination correction, and filtering / denoising on the original image data to generate preprocessed image data includes: The background illumination generation illumination distribution parameters are estimated by performing a convolution operation on the original image data using a two-dimensional Gaussian smoothing kernel function. The illumination equalization coefficient is calculated based on the ratio of the global average value of the illumination distribution parameters to the local illumination intensity. The illumination equalization coefficient is then multiplied pixel by pixel with the original image data to generate the adjusted image. The pixel values ​​of the adjusted image are linearly stretched to the target grayscale display range to generate the corrected image; The local neighborhood window of the corrected image is denoised using a median filter to generate preprocessed image data.

4. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The process of extracting low-level and mid-level features by performing convolution and pooling operations on the preprocessed image data, and then weighting and fusing them through an attention mechanism to generate image feature data, includes: Multiple convolution kernels are used to perform convolution operations on the preprocessed image data and a nonlinear activation function is applied to generate low-level feature maps; Max pooling is performed on the low-level feature map to generate a mid-level feature map. The low-level feature map is concatenated with the upsampled mid-level feature map along the channel dimension, input into the convolutional layer, and the sigmoid function is applied to generate the attention weight matrix. The low-level feature map is weighted using the attention weight matrix, and the upsampled mid-level feature map is weighted using the complementary weights of the attention weight matrix. The two are then added element by element to generate a fused feature map. The fused feature map is subjected to layer normalization in the channel dimension to generate image feature data.

5. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The step of inputting image feature data into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results includes: Obtain a deep learning model trained on a historical tableware image annotation dataset using a binary cross-entropy loss function and a backpropagation algorithm; Image feature data is input into a deep learning model for forward propagation calculation, and the output is the initial detection output representing the probability that a pixel belongs to a stain; Pixel values ​​with a probability value greater than a preset judgment threshold in the initial detection output are set as the first identifier value, and pixel values ​​with a probability value less than or equal to the preset judgment threshold are set as the second identifier value, thus generating a binarized stain detection result.

6. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The process of generating a visualization report by performing transparency blending rendering based on stain detection results and preprocessed image data includes: Obtain the preset transparency coefficient of the stained area and the color value of the overlay; When the pixel value of the stain detection result is the first identifier value indicating the stain, the color value of the overlay layer and the corresponding pixel color value of the preprocessed image data are mixed by weighted average according to the stain area transparency coefficient. When the pixel value of the stain detection result is the second identifier value indicating the background, the original pixel color value of the preprocessed image data is retained; Iterate through all pixels to complete color blending and generate a visual report.

7. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The process of encoding the visualization report and transmitting it to a display device for decoding and output includes: The pixel matrix and metadata of the visualization report are packaged and serialized into a continuous data stream for data encoding; The encoded data stream is transmitted to the display device via a communication link; The display device's graphics controller decodes the data stream, restores it to the original pixel color matrix, and writes it into the frame buffer. The system reads data from the frame buffer at a preset refresh rate and drives the display panel to present the image of the visualization report.

8. The method for detecting residual stains on tableware after disinfection based on deep learning according to claim 1, characterized in that, The calculation of the stain area percentage in the stain detection results, when the stain area percentage is greater than a preset stain area percentage threshold, triggers an alarm signal and generates an alarm notification, including: The total number of pixels identified as stains in the stain detection results and the total number of pixels of the tableware body in the preprocessed image data were counted. Calculate the ratio of the total number of pixels in the stain to the total number of pixels in the main body of the tableware to obtain the stain area percentage; When the percentage of the stain area is greater than the stain area percentage threshold, the alarm status is considered valid. In response to a valid alarm status, the voltage of the digital output port is switched to a high level to trigger a physical alarm signal and generate a structured alarm notification that includes a timestamp, tableware identifier, stain area percentage, and a visual report storage path.

9. A deep learning-based system for detecting residual stains on tableware after disinfection, characterized in that, include: The raw image data generation module acquires the initial image of the tableware after disinfection, performs format conversion and size normalization on the initial image, and generates raw image data. The preprocessed image data generation module performs illumination intensity analysis, illumination correction, and filtering and noise reduction on the original image data in sequence to generate preprocessed image data. The image feature data generation module performs convolution and pooling operations on the preprocessed image data to extract low-level and mid-level features, and then weights and fuses them through an attention mechanism to generate image feature data. The stain detection result generation module inputs image feature data into a pre-trained deep learning model for forward propagation calculation and threshold segmentation to generate stain detection results. The visualization report generation module generates a visualization report by performing transparency blending rendering based on stain detection results and preprocessed image data. The visualization report output module encodes the visualization report data and transmits it to the display device for decoding and output. The alarm signal triggering module calculates the percentage of stain area in the stain detection results. When the percentage of stain area is greater than the preset threshold for stain area percentage, an alarm signal is triggered and an alarm notification is generated.

Images