A machine vision-based fine product packaging dirt classification method and system

Abstract

The application relates to the technical field of image recognition, and discloses a fine product packaging dirt classification method and system based on machine vision, which comprises the following steps: acquiring a multi-modal image sequence of fine product packaging to be detected, and performing dynamic light evaluation and adaptive enhancement on image frames in the multi-modal image sequence to generate a standard light multi-modal image sequence; separating a background feature layer from the standard light multi-modal image sequence and identifying a real dirt area based on spectral response differences to generate a calibrated dirt distribution map; extracting multi-dimensional composite classification features from the standard light multi-modal image sequence for the real dirt area in the calibrated dirt distribution map to generate a composite classification feature tuple; and performing logical comparison between the composite classification feature tuple and a pre-defined dirt type feature rule library to obtain a dirt classification result map of the fine product packaging to be detected; and the application can improve the efficiency of fine product packaging dirt classification.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for classifying dirt and grime on premium packaging based on machine vision. Background Technology

[0002] Current technologies for detecting surface dirt on premium packaging generally employ single-modal visual imaging, which struggles to adapt to the complex and variable lighting environments of production sites. The acquired packaging images are prone to issues such as overexposed and underexposed areas, and uneven lighting distribution directly masks details of dirt on the packaging surface, interfering with the initial localization and feature extraction of dirty areas. Existing detection methods lack a dynamic lighting evaluation system for image frames, making it impossible to accurately identify overexposed and underexposed areas in the image. Furthermore, they cannot implement targeted adaptive enhancement compensation based on lighting defects in different areas, failing to generate standard inspection images with uniform lighting and complete details. This fundamentally reduces the basic accuracy and stability of dirt detection results.

[0003] Existing dirt detection methods lack the technical capability to efficiently separate the background feature layer on the packaging surface. They cannot accurately distinguish between genuine dirt and non-dirty interference areas such as packaging substrate texture, printing defects, and surface scratches through multispectral response differences. During the detection process, false dirt detection and missed detection of genuine dirt frequently occur. Current technologies extract dirt features from a single dimension, only acquiring basic morphological features. They lack support from multi-dimensional composite features such as geometric parameters, edge characteristics, spectral response, and texture distribution. Dirt classification lacks comprehensive feature basis, has low matching degree with preset classification rules, and cannot achieve efficient and accurate dirt type classification, making it difficult to adapt to the practical application requirements of high-precision and high-reliability detection for premium packaging. Summary of the Invention

[0004] This invention provides a machine vision-based method and system for classifying dirt and grime in premium packaging, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a machine vision-based method for classifying dirt and grime in premium packaging, comprising:

[0006] Pt.1. Obtain a multimodal image sequence of the premium packaging to be inspected, and perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence;

[0007] Pt.2. Separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirt areas based on spectral response differences to generate a labeled dirt distribution map;

[0008] Pt.3. For the actual dirty areas in the labeled dirty distribution map, extract multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples;

[0009] Pt.4. Logically compare the composite classification feature tuple with the predefined dirt type feature rule library to obtain the dirt classification result map of the premium packaging to be detected.

[0010] In a preferred embodiment, acquiring a multimodal image sequence of the premium packaging to be detected and performing dynamic illumination evaluation on the image frames in the multimodal image sequence includes:

[0011] The image frame of the premium packaging to be inspected is discretized into non-overlapping sub-regions according to the pixel space coordinates;

[0012] The maximum and minimum pixel brightness values ​​in each sub-region are analyzed, and the analyzed maximum and minimum values ​​are mapped according to the spatial arrangement of the sub-regions to construct a brightness distribution map representing the illumination fluctuation of the entire frame image.

[0013] Based on the brightness distribution map, the sub-regions where the pixel brightness maximum value exceeds the high brightness threshold are identified as overexposed region feature points;

[0014] Simultaneously, the sub-regions where the minimum pixel brightness value is lower than the low brightness threshold are identified as underexposed region feature points.

[0015] The overexposed area feature points and the underexposed area feature points are spatially aggregated to generate the illumination evaluation feature map of the image frame, wherein the illumination evaluation feature map carries overexposure intensity indication and underexposure intensity indication.

[0016] In a preferred embodiment, the step of performing dynamic illumination evaluation and adaptive enhancement on image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence further includes:

[0017] The overexposure intensity value is determined based on the amount by which the maximum pixel brightness value exceeds the maximum value within the overexposure distribution area in the illumination evaluation feature map;

[0018] The underexposure depth value is determined based on the insufficiency of the minimum pixel brightness value within the underexposure distribution area in the illumination evaluation feature map.

[0019] A nonlinear illumination compensation curve is constructed, wherein the compensation factor of the nonlinear illumination compensation curve is jointly determined by the overexposure intensity value and the underexposure depth value, and the compensation factor is expressed in the following form:

[0020] ;

[0021] In the formula, Located at pixel coordinates The compensation factor at the location, In pixel coordinates The average brightness value within a local neighborhood centered on the value. This is the median value of the brightness of all pixels in the image frame. The overexposure suppression coefficient is positively correlated with the overexposure intensity value. The underexposure stretching factor is positively correlated with the underexposure depth value. It is a natural exponential function;

[0022] Based on the nonlinear illumination compensation curve, the compensation factor is applied to the original gray values ​​of the pixels in the image frame to perform gray value remapping, thereby obtaining the compensated gray values ​​of the premium packaging to be detected.

[0023] All pixels after grayscale value mapping transformation are aggregated according to their original spatial coordinates to generate a standard illumination image frame of the premium packaging to be detected.

[0024] The standard illumination image frames are compiled according to the original acquisition time sequence and spectral conditions to generate the standard illumination multimodal image sequence of the premium packaging to be tested.

[0025] In a preferred embodiment, separating the background feature layer from the standard illumination multimodal image sequence includes:

[0026] Retrieve a standard template image that matches the model of the premium packaging to be tested, wherein the standard template image records the standard surface texture layout of the premium packaging to be tested under the same spectral illumination conditions in a clean state;

[0027] Align the image frames in the standard illumination multimodal image sequence with the standard template image using pixel coordinates;

[0028] When there is a deviation between the grayscale value of a pixel in the image frame and the grayscale value of the corresponding pixel in the standard template image, the pixel in the image frame is marked as a difference pixel; when there is no deviation, the pixel in the image frame is marked as a background matching pixel.

[0029] Based on the labeling results, the background matching pixels are stripped from the image frame, the difference pixels are retained, and the retained difference pixels are connected to form a difference foreground layer;

[0030] For image frames corresponding to different spectral conditions in the standard illumination multimodal image sequence, the logical difference discrimination and difference foreground layer extraction operations are repeatedly performed to obtain a set of difference foreground layers corresponding to different spectral conditions;

[0031] The differential foreground layers in the differential foreground layer set are fused at the pixel level according to spectral condition weights;

[0032] If any spectral difference foreground layer in the difference foreground layer set has a difference pixel at the same pixel coordinate position, then the same pixel coordinate position is retained as a difference pixel in the fusion result.

[0033] Connect all the different pixels after fusion to generate a composite foreground mask.

[0034] In a preferred embodiment, the step of identifying real dirty areas based on spectral response differences includes:

[0035] Traverse the connected pixel blocks in the composite foreground mask and define each traversed connected pixel block as a candidate dirty region;

[0036] For the candidate contaminated area, locate the corresponding image patch of the candidate contaminated area under different spectral conditions in the standard illumination multimodal image sequence;

[0037] Pixel brightness response values ​​are extracted from the image block, and all pixel brightness response values ​​extracted under the same spectral conditions are statistically merged to obtain the representative spectral response values ​​of the candidate dirty region.

[0038] The representative values ​​of the spectral response are arranged in order of spectral wavelength to form the spectral response feature vector of the candidate dirty area;

[0039] Extract the standard spectral response vector of the packaging substrate that matches the model of the premium packaging to be tested from the preset spectral feature library of packaging substrates;

[0040] The spectral response feature vector is compared band by band with the standard spectral response vector of the packaging substrate to obtain the difference description value of the premium packaging to be tested.

[0041] In a preferred embodiment, the step of identifying actual dirty areas based on spectral response differences and generating a labeled dirty distribution map includes:

[0042] The difference description value is compared with the authenticity judgment threshold of the premium packaging to be tested;

[0043] When the difference description value is lower than the true / false discrimination threshold, the candidate dirty area is determined to be a false dirty area, and the false dirty area is erased from the comprehensive foreground mask, wherein the erasure is to reset the pixel position corresponding to the false dirty area to a non-dirty background.

[0044] When the difference description value is not lower than the true / false discrimination threshold, the candidate dirty area is determined to be a real dirty area, and the pixel position corresponding to the real dirty area is retained in the comprehensive foreground mask, while a classification label is added to the real dirty area.

[0045] The composite foreground mask after all erasure and retention operations is determined as the labeled dirt distribution map, wherein the labeled dirt distribution map only contains the actual dirt areas and their spatial location information.

[0046] In a preferred embodiment, the step of extracting multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples for the actual dirty areas in the calibrated dirt distribution map includes:

[0047] Based on the spatial location boundary of the actual dirty area, image blocks of the actual dirty area are cropped from the spectral image frames of the standard illumination multimodal image sequence;

[0048] The boundary closure curve is obtained by tracing the contour boundary of the real dirty area image block. The pixel range occupied by the region in the boundary closure curve is determined as the area parameter, and the total extension length of the boundary closure curve is determined as the perimeter parameter. The circularity description value is derived based on the ratio of the area parameter to the perimeter parameter.

[0049] The gray-level transition amplitude is sampled along the boundary normal direction of the image block of the real dirty area, and the maximum sampled gray-level transition amplitude is used as the edge sharpness value.

[0050] The frequency of occurrence of different gray levels within the image block of the real dirty area is statistically analyzed, and the internal texture uniformity value is quantified based on the dispersion of the frequency of occurrence.

[0051] The area parameter, perimeter parameter, circularity description value, edge sharpness value, and internal texture uniformity value are sequentially grouped to form the geometric and edge feature subset in the composite classification feature tuple.

[0052] In a preferred embodiment, the step of extracting multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples for the actual dirty areas in the calibrated dirty distribution map further includes:

[0053] Locate the spectral image blocks of the real dirty area under different spectral conditions in the standard illumination multimodal image sequence;

[0054] Take any two gray value pairs of corresponding pixel positions in the spectral image blocks, map the gray value pairs to the angle in the vector space, sum all the angle values ​​and divide by the total number of pixel positions to obtain the spectral angle change value.

[0055] Obtain the color vector of the pixel in the real dirty area image block, retrieve the reference color vector of the same spatial position in the state without dirt, calculate the chromaticity angle between the pixel's color vector and the reference color vector, accumulate the chromaticity angles of the pixels and divide by the total number of pixel positions to obtain the average accumulated chromaticity angle.

[0056] Morphological filtering is performed on the image blocks of the real dirty area. The gray-level distribution entropy value of the image block before filtering and the gray-level distribution entropy value of the image block after filtering are measured respectively. The reduction ratio is derived based on the ratio of the entropy value after filtering to the entropy value before filtering, and the texture entropy value change rate is obtained.

[0057] The spectral angle change value, the cumulative average value of the chromaticity angle, and the rate of change of the texture entropy value are sequentially grouped to form the spectral and texture feature subsets in the composite classification feature tuple.

[0058] In a preferred embodiment, the step of logically comparing the composite classification feature tuple with a predefined dirt type feature rule base to obtain a dirt classification result image of the premium packaging to be tested includes:

[0059] Retrieve a predefined dirt type feature rule library, wherein the dirt type feature rule library records standard dirt category labels and standard feature intervals bound to the labels, and the standard feature intervals are defined by a lower feature threshold and an upper feature threshold.

[0060] Read the parameter values ​​from the composite classification feature tuple, and perform an inclusion test between the parameter values ​​and the standard feature interval of the standard dirt category label in sequence;

[0061] When the parameter value is greater than or equal to the lower limit threshold of the feature and less than or equal to the upper limit threshold of the feature, it is determined that the parameter value falls into the standard feature range;

[0062] When all parameter values ​​in the composite classification feature tuple fall within the standard feature range, the standard dirt category label is assigned to the actual dirt area of ​​the composite classification feature tuple.

[0063] When a parameter value does not fall within the standard feature range of any standard dirt category label, an unclassified label is added to the actual dirty area;

[0064] The actual dirty areas in the calibrated dirt distribution map are mapped back to the original image coordinate system according to their spatial location, and the assigned dirt type label or unclassified mark is spatially associated with the corresponding actual dirty area to generate the dirt classification result map of the premium packaging to be tested.

[0065] To address the aforementioned problems, the present invention also provides a machine vision-based system for classifying dirt and grime in premium packaging, the system comprising:

[0066] An imaging illumination calibration module is used to acquire a multimodal image sequence of the premium packaging to be inspected, and to perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence.

[0067] The dirty area identification module is used to separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirty areas based on the spectral response differences, generating a labeled dirty distribution map;

[0068] The composite feature extraction module is used to extract multi-dimensional composite classification features from the standard illumination multimodal image sequence for the real dirty areas in the labeled dirty distribution map, and generate composite classification feature tuples.

[0069] The stain type determination module is used to logically compare the composite classification feature tuple with the predefined stain type feature rule library to obtain the stain classification result map of the premium packaging to be tested.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. This invention conducts a full-process dynamic illumination assessment of multimodal image sequences of premium packaging to be inspected, accurately locates overexposed and underexposed areas in the image, implements targeted adaptive enhancement compensation based on the actual degree of illumination defects in each area, and corrects abnormal illumination distribution problems in the image through standardized brightness remapping processing, generating a standard illumination multimodal image sequence with uniform illumination, complete details, and clear texture, fully preserving all original feature information of dirt on the packaging surface, providing stable and accurate basic image support for subsequent identification and classification of dirt areas, and continuously ensuring the reliability of the detection process and the accuracy of the results.

[0072] 2. This invention can efficiently strip the background feature layer from standard illumination multimodal image sequences, accurately identify real dirt areas based on spectral response differences, and eliminate various interference factors, generating a clearly labeled and labeled dirt distribution map. It extracts multi-dimensional composite classification features such as geometric shape, edge sharpness, spectral response, and texture uniformity from real dirt areas, constructing a comprehensive and complete feature tuple. This feature tuple is then precisely logically matched with a dirt type feature rule library to quickly complete the determination and labeling of dirt types, generating a dirt classification result map with accurate spatial location and clear type division. This comprehensively improves the processing efficiency and accuracy of dirt classification, stably meeting the high-precision and high-standardization testing application requirements of premium packaging. Attached Figure Description

[0073] Figure 1 This is a flowchart illustrating a machine vision-based method for classifying dirt and grime in premium packaging, as provided in an embodiment of the present invention.

[0074] Figure 2 A functional module diagram of a machine vision-based dirt sorting system for premium packaging provided in an embodiment of the present invention;

[0075] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0076] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0077] This application provides a machine vision-based method for classifying contamination in premium packaging. The executing entity of this machine vision-based method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the machine vision-based method for classifying contamination in premium packaging can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0078] Reference Figure 1 The diagram shown is a flowchart illustrating a machine vision-based method for classifying dirt and grime on premium packaging according to an embodiment of the present invention. In this embodiment, the machine vision-based method for classifying dirt and grime on premium packaging includes:

[0079] Pt.1. Obtain a multimodal image sequence of the premium packaging to be inspected, and perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence;

[0080] In this embodiment of the invention, acquiring a multimodal image sequence of the premium packaging to be detected and performing dynamic illumination evaluation on the image frames in the multimodal image sequence includes:

[0081] The image frame of the premium packaging to be inspected is discretized into non-overlapping sub-regions according to the pixel space coordinates;

[0082] The maximum and minimum pixel brightness values ​​in each sub-region are analyzed, and the analyzed maximum and minimum values ​​are mapped according to the spatial arrangement of the sub-regions to construct a brightness distribution map representing the illumination fluctuation of the entire frame image.

[0083] Based on the brightness distribution map, the sub-regions where the pixel brightness maximum value exceeds the high brightness threshold are identified as overexposed region feature points;

[0084] Simultaneously, the sub-regions where the minimum pixel brightness value is lower than the low brightness threshold are identified as underexposed region feature points.

[0085] The overexposed area feature points and the underexposed area feature points are spatially aggregated to generate the illumination evaluation feature map of the image frame, wherein the illumination evaluation feature map carries overexposure intensity indication and underexposure intensity indication.

[0086] The step of performing dynamic illumination evaluation and adaptive enhancement on image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence further includes:

[0087] The overexposure intensity value is determined based on the amount by which the maximum pixel brightness value exceeds the maximum value within the overexposure distribution area in the illumination evaluation feature map;

[0088] The underexposure depth value is determined based on the insufficiency of the minimum pixel brightness value within the underexposure distribution area in the illumination evaluation feature map.

[0089] A nonlinear illumination compensation curve is constructed, wherein the compensation factor of the nonlinear illumination compensation curve is jointly determined by the overexposure intensity value and the underexposure depth value, and the compensation factor is expressed in the following form:

[0090] ;

[0091] In the formula, Located at pixel coordinates The compensation factor at the location, In pixel coordinates The average brightness value within a local neighborhood centered on the value. This is the median value of the brightness of all pixels in the image frame. The overexposure suppression coefficient is positively correlated with the overexposure intensity value. The underexposure stretching factor is positively correlated with the underexposure depth value. It is a natural exponential function;

[0092] Based on the nonlinear illumination compensation curve, the compensation factor is applied to the original gray values ​​of the pixels in the image frame to perform gray value remapping, thereby obtaining the compensated gray values ​​of the premium packaging to be detected.

[0093] All pixels after grayscale value mapping transformation are aggregated according to their original spatial coordinates to generate a standard illumination image frame of the premium packaging to be detected.

[0094] The standard illumination image frames are compiled according to the original acquisition time sequence and spectral conditions to generate the standard illumination multimodal image sequence of the premium packaging to be tested.

[0095] Based on the row and column distribution of pixel spatial coordinates of the image frame to be tested, the entire image frame is divided into analysis sub-regions of uniform size that do not overlap or have gaps. Each analysis sub-region corresponds to a fixed pixel spatial range, thus completing the discretization process of the image frame.

[0096] Traverse all pixels within a single analysis sub-region and read the brightness value of each pixel. Select the result with the largest value from all brightness values ​​in the sub-region as the pixel brightness maximum and the result with the smallest value as the pixel brightness minimum. According to the original spatial arrangement of each analysis sub-region in the image frame, map the corresponding pixel brightness maximum and pixel brightness minimum to the matching spatial coordinate position. Combine the brightness extreme value information after all coordinate mappings to form a brightness distribution map that can completely reflect the illumination fluctuation state of the entire frame image.

[0097] The pre-set high brightness threshold is retrieved, and the maximum pixel brightness value corresponding to each analysis sub-region in the brightness distribution map is compared with the high brightness threshold one by one. The analysis sub-regions with the maximum pixel brightness value greater than the high brightness threshold are marked, and the marked analysis sub-regions are the overexposed area feature points.

[0098] The preset low brightness threshold is retrieved, and the minimum pixel brightness value corresponding to each analysis sub-region in the brightness distribution map is compared with the low brightness threshold one by one. The analysis sub-regions with minimum pixel brightness values ​​less than the low brightness threshold are marked, and the marked analysis sub-regions are the underexposed area feature points.

[0099] Based on the original pixel spatial coordinates of the image frame, all overexposed area feature points and all underexposed area feature points are spatially integrated and aggregated. Within the integrated coordinate system, the corresponding attributes of each feature point are labeled, and finally, the illumination evaluation feature map of the image frame is generated. This illumination evaluation feature map fully carries the overexposure intensity indication corresponding to each overexposed area feature point and the underexposure intensity indication corresponding to each underexposed area feature point.

[0100] Read the maximum pixel brightness value corresponding to the overexposed distribution area in the illumination evaluation feature map, calculate the difference between the maximum pixel brightness value and the preset high brightness threshold, and obtain the calculation result as the excess of the maximum pixel brightness value in the overexposed distribution area. Based on this excess, directly determine the overexposed intensity value of the corresponding overexposed distribution area.

[0101] Read the minimum pixel brightness value corresponding to the underexposed distribution area in the illumination evaluation feature map, calculate the difference between the preset low brightness threshold and the minimum pixel brightness value, and obtain the calculation result as the deficiency of the minimum pixel brightness value in the underexposed distribution area. Based on this deficiency, directly determine the underexposed depth value of the corresponding underexposed distribution area.

[0102] Traverse the coordinates of each pixel within the image frame, extract a fixed range of local neighboring pixels centered on the current pixel coordinates, count the brightness of all pixels within this local neighboring region and calculate the average brightness value, count the brightness values ​​of all pixels within the image frame and determine the median value of all brightness values, determine the overexposure suppression coefficient positively correlated with the overexposure intensity value, determine the underexposure stretching coefficient positively correlated with the underexposure depth value, and construct a nonlinear illumination compensation curve by combining the local neighboring average brightness value, the pixel brightness median value, the overexposure suppression coefficient, and the underexposure stretching coefficient. This curve generates a compensation factor for each pixel coordinate position that is jointly determined by the overexposure intensity value and the underexposure depth value.

[0103] The compensation factor corresponding to the coordinate position of each pixel in the image frame is obtained according to the nonlinear illumination compensation curve. The original gray value of each pixel is combined with the corresponding compensation factor to adjust the brightness. The gray value remapping process of all pixels is completed. After remapping, the compensated gray value of each pixel of the premium packaging to be detected is obtained.

[0104] All pixels that have completed grayscale value mapping transformation are repositioned and integrated according to their pixel spatial coordinates in the original image frame. After all pixels are repositioned and integrated, a standard illumination image frame of the premium packaging to be inspected is generated.

[0105] All generated standard illumination image frames are classified and cataloged according to the acquisition sequence of the original multimodal image sequence and the corresponding spectral acquisition conditions. After all standard illumination image frames are cataloged, a standard illumination multimodal image sequence of the premium packaging to be tested is generated.

[0106] The location information corresponding to the compensation factor is taken from the spatial coordinates of each pixel in the image frame of the premium packaging to be detected.

[0107] The average brightness value of a local neighborhood is obtained by defining a fixed range of local neighborhoods centered on a single pixel coordinate, counting the brightness values ​​of all pixels within that neighborhood, and calculating the average value.

[0108] The global luminance median value is obtained by statistically analyzing the luminance values ​​of all pixels within an image frame, sorting all luminance values ​​by size, and then taking the value at the median position.

[0109] The overexposure suppression coefficient is set based on the overexposure intensity value determined by the amount by which the maximum pixel brightness of the overexposure distribution area in the illumination evaluation feature map exceeds the preset high brightness threshold. The two maintain a positive correlation.

[0110] The underexposure stretching factor is set based on the underexposure depth value determined by the insufficient amount of pixel brightness in the underexposure distribution area of ​​the illumination evaluation feature map, which is lower than the preset low brightness threshold. The two maintain a positive correlation.

[0111] The natural index conversion follows the mathematical rules of the natural index, performing exponential operations on the difference between the local neighborhood average brightness value and the global brightness median value.

[0112] Nonlinear illumination compensation curves are used to generate a unique compensation factor for each pixel coordinate position within an image frame, providing an adjustment basis for pixel grayscale value remapping.

[0113] The compensation factor can dynamically determine the brightness adjustment range of each pixel by combining the difference between local and global brightness of the pixel, the overexposure suppression requirement and the underexposure stretching requirement.

[0114] By remapping the original grayscale values ​​using a compensation factor, overexposure and underexposure issues in image frames can be accurately corrected, ensuring that the brightness distribution of the entire image reaches a standard uniform state.

[0115] The standard illumination image frame generated after processing with this compensation curve can completely preserve the texture and dirt details of the packaging surface, providing stable and reliable image support for subsequent identification and classification of dirty areas.

[0116] Pt.2. Separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirt areas based on spectral response differences to generate a labeled dirt distribution map;

[0117] In this embodiment of the invention, separating the background feature layer from the standard illumination multimodal image sequence includes:

[0118] Retrieve a standard template image that matches the model of the premium packaging to be tested, wherein the standard template image records the standard surface texture layout of the premium packaging to be tested under the same spectral illumination conditions in a clean state;

[0119] Align the image frames in the standard illumination multimodal image sequence with the standard template image using pixel coordinates;

[0120] When there is a deviation between the grayscale value of a pixel in the image frame and the grayscale value of the corresponding pixel in the standard template image, the pixel in the image frame is marked as a difference pixel; when there is no deviation, the pixel in the image frame is marked as a background matching pixel.

[0121] Based on the labeling results, the background matching pixels are stripped from the image frame, the difference pixels are retained, and the retained difference pixels are connected to form a difference foreground layer;

[0122] For image frames corresponding to different spectral conditions in the standard illumination multimodal image sequence, the logical difference discrimination and difference foreground layer extraction operations are repeatedly performed to obtain a set of difference foreground layers corresponding to different spectral conditions;

[0123] The differential foreground layers in the differential foreground layer set are fused at the pixel level according to spectral condition weights;

[0124] If any spectral difference foreground layer in the difference foreground layer set has a difference pixel at the same pixel coordinate position, then the same pixel coordinate position is retained as a difference pixel in the fusion result.

[0125] Connect all the different pixels after fusion to generate a composite foreground mask.

[0126] The method of identifying real dirty areas based on spectral response differences includes:

[0127] Traverse the connected pixel blocks in the composite foreground mask and define each traversed connected pixel block as a candidate dirty region;

[0128] For the candidate contaminated area, locate the corresponding image patch of the candidate contaminated area under different spectral conditions in the standard illumination multimodal image sequence;

[0129] Pixel brightness response values ​​are extracted from the image block, and all pixel brightness response values ​​extracted under the same spectral conditions are statistically merged to obtain the representative spectral response values ​​of the candidate dirty region.

[0130] The representative values ​​of the spectral response are arranged in order of spectral wavelength to form the spectral response feature vector of the candidate dirty area;

[0131] Extract the standard spectral response vector of the packaging substrate that matches the model of the premium packaging to be tested from the preset spectral feature library of packaging substrates;

[0132] The spectral response feature vector is compared band by band with the standard spectral response vector of the packaging substrate to obtain the difference description value of the premium packaging to be tested.

[0133] The process of identifying actual contaminated areas based on spectral response differences and generating a labeled contamination distribution map includes:

[0134] The difference description value is compared with the authenticity judgment threshold of the premium packaging to be tested;

[0135] When the difference description value is lower than the true / false discrimination threshold, the candidate dirty area is determined to be a false dirty area, and the false dirty area is erased from the comprehensive foreground mask, wherein the erasure is to reset the pixel position corresponding to the false dirty area to a non-dirty background.

[0136] When the difference description value is not lower than the true / false discrimination threshold, the candidate dirty area is determined to be a real dirty area, and the pixel position corresponding to the real dirty area is retained in the comprehensive foreground mask, while a classification label is added to the real dirty area.

[0137] The composite foreground mask after all erasure and retention operations is determined as the labeled dirt distribution map, wherein the labeled dirt distribution map only contains the actual dirt areas and their spatial location information.

[0138] Retrieve a standard template image that perfectly matches the model of the premium packaging to be tested. This standard template image fully records the standard surface texture layout information of the premium packaging of this model under the same spectral illumination conditions in a clean state.

[0139] Each frame of the standard illumination multimodal image sequence is aligned point by point with the standard template image according to pixel space coordinates to ensure that the corresponding pixels of the two are in completely consistent spatial coordinate positions.

[0140] A preset grayscale deviation threshold is retrieved, and the grayscale value of each pixel in the image frame is compared with the grayscale value of the pixel at the same coordinate position in the standard template image. If the difference between the grayscale value of the image frame pixel and the corresponding pixel grayscale value in the standard template image is greater than the preset grayscale deviation threshold, the pixel is marked as a difference pixel. If the difference between the grayscale value of the image frame pixel and the corresponding pixel grayscale value in the standard template image is less than or equal to the preset grayscale deviation threshold, the pixel is marked as a background matching pixel.

[0141] Based on the pixel labeling results, all parts of the image frame marked as background matching pixels are completely peeled off and removed from the image, leaving only all parts marked as difference pixels. The retained difference pixels are then connected according to their spatial adjacency to form the difference foreground layer corresponding to the image frame.

[0142] For each image frame corresponding to a different spectral condition in the standard illumination multimodal image sequence, grayscale value comparison, pixel marking, background matching pixel stripping, and difference pixel connectivity are performed sequentially. After processing all image frames under different spectral conditions, a set of difference foreground layers corresponding to different spectral conditions is obtained.

[0143] According to the preset spectral condition weights, pixel-by-pixel fusion processing is performed on all different foreground layers corresponding to different spectral conditions in the difference foreground layer set.

[0144] Traverse all pixel coordinate positions. At the same pixel coordinate position, as long as there is a difference pixel in any spectral condition of the difference foreground layer in the difference foreground layer set, retain it as a difference pixel in the fusion result at that pixel coordinate position.

[0145] After fusion, all retained difference pixels are fully connected according to their spatial adjacency to generate a comprehensive foreground mask corresponding to the standard illumination multimodal image sequence.

[0146] By traversing all interconnected pixel blocks within the foreground mask according to their spatial location, each independent connected pixel block is completely defined as the corresponding candidate dirty region.

[0147] For each candidate contaminated region, the image spatial range corresponding to the candidate contaminated region under different spectral conditions is accurately located in the standard illumination multimodal image sequence, and the image patch matching the candidate contaminated region under each spectral condition is extracted.

[0148] The brightness response values ​​of all pixels are extracted one by one from the image blocks corresponding to different spectral conditions. The brightness response values ​​of all pixels extracted under the same spectral condition are summarized and the average value is calculated. This average value is used as the representative value of the spectral response of the candidate dirty area under the current spectral condition.

[0149] The representative values ​​of the spectral response of the candidate contaminated area under various spectral conditions are arranged in a fixed order from shortest to longest spectral wavelength, and combined to form the spectral response feature vector of the candidate contaminated area.

[0150] From the pre-built spectral feature library of packaging substrates, retrieve and extract the standard spectral response vector of the packaging substrate that perfectly matches the model of the premium packaging to be tested.

[0151] The spectral response feature vectors of the candidate soiled areas are compared with the standard spectral response vectors of the packaging substrate one by one according to the same spectral bands. The numerical difference results of each band are summarized to obtain the difference description value of the candidate soiled areas corresponding to the premium packaging to be tested.

[0152] The difference description value corresponding to each candidate dirty area is compared with the pre-set authenticity judgment threshold of the premium packaging to be tested.

[0153] When the difference description value is less than the true / false discrimination threshold, the candidate dirty area is determined to be a false dirty area. In the comprehensive foreground mask, all pixel positions corresponding to the false dirty area are reset to non-dirty background to complete the erasure process of the false dirty area.

[0154] When the difference description value is greater than or equal to the true / false discrimination threshold, the candidate dirty area is determined as a real dirty area. All pixel positions corresponding to the real dirty area are retained in the comprehensive foreground mask, and a classification label is added to the real dirty area.

[0155] After completing the erasure and retention operations on all candidate dirty areas within the composite foreground mask, the processed composite foreground mask is determined as a labeled dirty distribution map. This labeled dirty distribution map only contains the actual dirty areas and their corresponding spatial location information.

[0156] Pt.3. For the actual dirty areas in the labeled dirty distribution map, extract multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples;

[0157] In this embodiment of the invention, the step of extracting multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples for the actual dirty areas in the calibrated dirty distribution map includes:

[0158] Based on the spatial location boundary of the actual dirty area, image blocks of the actual dirty area are cropped from the spectral image frames of the standard illumination multimodal image sequence;

[0159] The boundary closure curve is obtained by tracing the contour boundary of the real dirty area image block. The pixel range occupied by the region in the boundary closure curve is determined as the area parameter, and the total extension length of the boundary closure curve is determined as the perimeter parameter. The circularity description value is derived based on the ratio of the area parameter to the perimeter parameter.

[0160] The gray-level transition amplitude is sampled along the boundary normal direction of the image block of the real dirty area, and the maximum sampled gray-level transition amplitude is used as the edge sharpness value.

[0161] The frequency of occurrence of different gray levels within the image block of the real dirty area is statistically analyzed, and the internal texture uniformity value is quantified based on the dispersion of the frequency of occurrence.

[0162] The area parameter, perimeter parameter, circularity description value, edge sharpness value, and internal texture uniformity value are sequentially grouped to form the geometric and edge feature subset in the composite classification feature tuple.

[0163] The step of extracting multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples for the actual dirty areas in the labeled dirty distribution map also includes:

[0164] Locate the spectral image blocks of the real dirty area under different spectral conditions in the standard illumination multimodal image sequence;

[0165] Take any two gray value pairs of corresponding pixel positions in the spectral image blocks, map the gray value pairs to the angle in the vector space, sum all the angle values ​​and divide by the total number of pixel positions to obtain the spectral angle change value.

[0166] Obtain the color vector of the pixel in the real dirty area image block, retrieve the reference color vector of the same spatial position in the state without dirt, calculate the chromaticity angle between the pixel's color vector and the reference color vector, accumulate the chromaticity angles of the pixels and divide by the total number of pixel positions to obtain the average accumulated chromaticity angle.

[0167] Morphological filtering is performed on the image blocks of the real dirty area. The gray-level distribution entropy value of the image block before filtering and the gray-level distribution entropy value of the image block after filtering are measured respectively. The reduction ratio is derived based on the ratio of the entropy value after filtering to the entropy value before filtering, and the texture entropy value change rate is obtained.

[0168] The spectral angle change value, the cumulative average value of the chromaticity angle, and the rate of change of the texture entropy value are sequentially grouped to form the spectral and texture feature subsets in the composite classification feature tuple.

[0169] Based on the spatial location boundary of the actual dirty area in the calibrated dirty distribution map, the actual dirty area image block that perfectly matches the actual dirty area is completely cropped out on the spectral image frame corresponding to the standard illumination multimodal image sequence according to the coordinate range of the boundary.

[0170] Continuous and uninterrupted tracking is performed along the outer contour boundary of the real dirty area image block to form a complete and unbroken boundary closed curve. The total number of pixels covered by the boundary closed curve is used as the area parameter of the pixel range occupied by the region. The total number of pixels traversed by the extension of the boundary closed curve itself is used as the perimeter parameter of the total extension length of the curve. The area parameter and the perimeter parameter are correlated and calculated to obtain a circularity description value that can characterize the regularity of the region contour.

[0171] Based on the boundary closed curve of the real dirty area image block, the gray values ​​of pixels are sampled along the normal direction of each point on the boundary to the inside and outside of the region respectively. The gray value difference between adjacent sampled pixels is calculated to obtain the gray value transition amplitude. The result with the largest value among all the gray value transition amplitudes obtained by sampling is selected as the edge sharpness value of the region.

[0172] The gray values ​​of all pixels in the image block of the real dirty area are divided into fixed levels. The number of pixels contained in each gray level is counted to obtain the corresponding occurrence frequency. The distribution dispersion of the occurrence frequency of each gray level is calculated, and the dispersion is quantified into an internal texture uniformity value that can characterize the texture regularity of the region.

[0173] The measured area parameters, perimeter parameters, roundness description values, edge sharpness values, and internal texture uniformity values ​​are combined and arranged in a fixed order to form a subset of geometric and edge features in the composite classification feature tuple used to describe the geometric shape and edge characteristics of dirt.

[0174] In a standard illumination multimodal image sequence, the spectral image blocks corresponding to the real dirt area under each spectral condition are located according to the spatial coordinate range of the real dirt area, ensuring that the spatial range of each spectral image block completely overlaps with the real dirt area.

[0175] Select any two sets of spectral image blocks corresponding to different spectral conditions, pair the gray values ​​of the same pixel coordinate positions in the two sets of image blocks to form gray value pairs, map each gray value pair to the vector space to calculate the corresponding angle, sum all the angle values ​​corresponding to all pixel positions, divide the total summed angle value by the total number of pixel positions involved in the calculation to obtain the spectral angle change value of the real dirty area.

[0176] Read the color vector corresponding to each pixel in the image block of the real dirty area, retrieve the reference color vector at the same spatial coordinate position in the state without dirt from the standard template data, calculate the chromaticity angle between the color vector and the reference color vector for each pixel, and sum the chromaticity angle values ​​of all pixels and divide by the total number of pixel positions to obtain the average chromaticity angle of the real dirty area.

[0177] Morphological filtering is performed on the image block of the real dirty area. First, the entropy value corresponding to the gray-level distribution of all pixels in the image block before filtering is measured. Then, the entropy value corresponding to the gray-level distribution of all pixels in the image block after filtering is measured. The reduction ratio is obtained by dividing the entropy value after filtering by the entropy value before filtering. This reduction ratio is the rate of change of texture entropy value of the real dirty area.

[0178] The measured spectral angle change values, cumulative average values ​​of chromaticity angles, and texture entropy change rates are combined and arranged in a fixed order to form a subset of spectral and texture features in the composite classification feature tuple used to describe the spectral and texture characteristics of dirt.

[0179] Pt.4. Logically compare the composite classification feature tuple with the predefined dirt type feature rule library to obtain the dirt classification result map of the premium packaging to be detected.

[0180] In this embodiment of the invention, the step of logically comparing the composite classification feature tuple with a predefined dirt type feature rule library to obtain a dirt classification result image of the premium packaging to be detected includes:

[0181] Retrieve a predefined dirt type feature rule library, wherein the dirt type feature rule library records standard dirt category labels and standard feature intervals bound to the labels, and the standard feature intervals are defined by a lower feature threshold and an upper feature threshold.

[0182] Read the parameter values ​​from the composite classification feature tuple, and perform an inclusion test between the parameter values ​​and the standard feature interval of the standard dirt category label in sequence;

[0183] When the parameter value is greater than or equal to the lower limit threshold of the feature and less than or equal to the upper limit threshold of the feature, it is determined that the parameter value falls into the standard feature range;

[0184] When all parameter values ​​in the composite classification feature tuple fall within the standard feature range, the standard dirt category label is assigned to the actual dirt area of ​​the composite classification feature tuple.

[0185] When a parameter value does not fall within the standard feature range of any standard dirt category label, an unclassified label is added to the actual dirty area;

[0186] The actual dirty areas in the calibrated dirt distribution map are mapped back to the original image coordinate system according to their spatial location, and the assigned dirt type label or unclassified mark is spatially associated with the corresponding actual dirty area to generate the dirt classification result map of the premium packaging to be tested.

[0187] Retrieve a predefined dirt type feature rule base. This rule base fully stores various standard dirt category labels and the standard feature range bound to each label. Each standard feature range is defined by a pre-set lower feature threshold and an upper feature threshold.

[0188] Read the parameter values ​​corresponding to all features in the composite classification feature tuple, and perform an inclusion check on each parameter value with the standard feature interval corresponding to each standard dirt category label in the dirt type feature rule library, and determine whether the parameter value is within the corresponding interval range one by one.

[0189] For each parameter value, an interval comparison is performed. If the parameter value is greater than or equal to the lower limit threshold of the corresponding standard feature interval, and less than or equal to the upper limit threshold of the interval, the parameter value is determined to fall within the current standard feature interval.

[0190] When all parameter values ​​within a composite classification feature tuple fall within the standard feature range corresponding to the same standard dirt category label, the standard dirt category label is directly assigned to the actual dirt area corresponding to the current composite classification feature tuple.

[0191] If any parameter value in the composite classification feature tuple does not fall within the standard feature range corresponding to any standard dirt category label in the dirt type feature rule library, an unclassified label is added to the real dirt area corresponding to that parameter value.

[0192] All real dirt areas within the calibrated dirt distribution map are mapped back to the original image coordinate system of the premium packaging to be inspected according to their original spatial coordinate positions. The dirt type label or unclassified mark corresponding to each real dirt area is associated and bound one-to-one with the spatial coordinate position of that area. After the association is completed, a dirt classification result map of the premium packaging to be inspected is generated.

[0193] like Figure 2 The diagram shown is a functional block diagram of a machine vision-based dirt classification system for premium packaging provided in an embodiment of the present invention.

[0194] The machine vision-based high-end packaging soiling classification system described in this invention can be installed in an electronic device. Depending on the functions implemented, the machine vision-based high-end packaging soiling classification system may include an imaging illumination calibration module, a soiled area identification module, a composite feature extraction module, and a soiling type determination module. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and is stored in the memory of the electronic device.

[0195] In this embodiment, the functions of each module / unit are as follows:

[0196] The imaging illumination calibration module is used to acquire a multimodal image sequence of the premium packaging to be inspected, and to perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence.

[0197] The dirty area identification module is used to separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirty areas based on the spectral response differences, generating a labeled dirty distribution map;

[0198] The composite feature extraction module is used to extract multi-dimensional composite classification features from the standard illumination multimodal image sequence for the real dirty areas in the labeled dirty distribution map, and generate composite classification feature tuples.

[0199] The stain type determination module is used to logically compare the composite classification feature tuple with the predefined stain type feature rule library to obtain the stain classification result map of the premium packaging to be tested.

[0200] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0201] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0202] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0203] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0204] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A machine vision-based method for classifying dirt and grime in premium packaging, characterized in that, The method includes: Pt.

1. Obtain a multimodal image sequence of the premium packaging to be inspected, and perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence. Pt.

2. Separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirt areas based on spectral response differences to generate a labeled dirt distribution map; Pt.

3. For the actual dirty areas in the labeled dirty distribution map, extract multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples; Pt.

4. Logically compare the composite classification feature tuple with the predefined dirt type feature rule library to obtain the dirt classification result map of the premium packaging to be detected.

2. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 1, characterized in that, The process of acquiring a multimodal image sequence of the premium packaging to be inspected and performing dynamic illumination evaluation on the image frames in the multimodal image sequence includes: The image frame of the premium packaging to be inspected is discretized into non-overlapping sub-regions according to the pixel space coordinates; The maximum and minimum pixel brightness values ​​in each sub-region are analyzed, and the analyzed maximum and minimum values ​​are mapped according to the spatial arrangement of the sub-regions to construct a brightness distribution map representing the illumination fluctuation of the entire frame image. Based on the brightness distribution map, the sub-regions where the pixel brightness maximum value exceeds the high brightness threshold are identified as overexposed region feature points; Simultaneously, the sub-regions whose pixel brightness minimum values ​​are lower than the low brightness threshold are identified as underexposed region feature points. The overexposed area feature points and the underexposed area feature points are spatially aggregated to generate the illumination evaluation feature map of the image frame, wherein the illumination evaluation feature map carries overexposure intensity indication and underexposure intensity indication.

3. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 2, characterized in that, The step of performing dynamic illumination evaluation and adaptive enhancement on image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence further includes: The overexposure intensity value is determined based on the amount by which the maximum pixel brightness value exceeds the maximum value within the overexposure distribution area in the illumination evaluation feature map; The underexposure depth value is determined based on the insufficiency of the minimum pixel brightness value within the underexposure distribution area in the illumination evaluation feature map. A nonlinear illumination compensation curve is constructed, wherein the compensation factor of the nonlinear illumination compensation curve is jointly determined by the overexposure intensity value and the underexposure depth value, and the compensation factor is expressed in the following form: ； In the formula, Located at pixel coordinates The compensation factor at the location, In pixel coordinates The average brightness value within a local neighborhood centered on the value. This is the median value of the brightness of all pixels in the image frame. The overexposure suppression coefficient is positively correlated with the overexposure intensity value. The underexposure stretching factor is positively correlated with the underexposure depth value. It is a natural exponential function; Based on the nonlinear illumination compensation curve, the compensation factor is applied to the original gray values ​​of the pixels in the image frame to perform gray value remapping, thereby obtaining the compensated gray values ​​of the premium packaging to be detected. All pixels after grayscale value mapping transformation are aggregated according to their original spatial coordinates to generate a standard illumination image frame of the premium packaging to be detected. The standard illumination image frames are compiled according to the original acquisition time sequence and spectral conditions to generate the standard illumination multimodal image sequence of the premium packaging to be tested.

4. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 1, characterized in that, The step of separating the background feature layer from the standard illumination multimodal image sequence includes: Retrieve a standard template image that matches the model of the premium packaging to be tested, wherein the standard template image records the standard surface texture layout of the premium packaging to be tested under the same spectral illumination conditions in a clean state; Align the image frames in the standard illumination multimodal image sequence with the standard template image using pixel coordinates; When there is a deviation between the grayscale value of a pixel in the image frame and the grayscale value of the corresponding pixel in the standard template image, the pixel in the image frame is marked as a difference pixel; when there is no deviation, the pixel in the image frame is marked as a background matching pixel. Based on the labeling results, the background matching pixels are stripped from the image frame, the difference pixels are retained, and the retained difference pixels are connected to form a difference foreground layer; For image frames corresponding to different spectral conditions in the standard illumination multimodal image sequence, the logical difference discrimination and difference foreground layer extraction operations are repeatedly performed to obtain a set of difference foreground layers corresponding to different spectral conditions; The differential foreground layers in the differential foreground layer set are fused at the pixel level according to spectral condition weights; If any spectral difference foreground layer in the difference foreground layer set has a difference pixel at the same pixel coordinate position, then the same pixel coordinate position is retained as a difference pixel in the fusion result. Connect all the different pixels after fusion to generate a composite foreground mask.

5. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 4, characterized in that, The method of identifying real dirty areas based on spectral response differences includes: Traverse the connected pixel blocks in the composite foreground mask and define each traversed connected pixel block as a candidate dirty region; For the candidate contaminated area, locate the corresponding image patch of the candidate contaminated area under different spectral conditions in the standard illumination multimodal image sequence; Pixel brightness response values ​​are extracted from the image block, and all pixel brightness response values ​​extracted under the same spectral conditions are statistically merged to obtain the representative spectral response values ​​of the candidate dirty region. The representative values ​​of the spectral response are arranged in order of spectral wavelength to form the spectral response feature vector of the candidate dirty area; Extract the standard spectral response vector of the packaging substrate that matches the model of the premium packaging to be tested from the preset spectral feature library of packaging substrates; The spectral response feature vector is compared band by band with the standard spectral response vector of the packaging substrate to obtain the difference description value of the premium packaging to be tested.

6. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 5, characterized in that, The process of identifying actual contaminated areas based on spectral response differences and generating a labeled contamination distribution map includes: The difference description value is compared with the authenticity judgment threshold of the premium packaging to be tested; When the difference description value is lower than the true / false discrimination threshold, the candidate dirty area is determined to be a false dirty area, and the false dirty area is erased from the comprehensive foreground mask, wherein the erasure is to reset the pixel position corresponding to the false dirty area to a non-dirty background. When the difference description value is not lower than the true / false discrimination threshold, the candidate dirty area is determined to be a real dirty area, and the pixel position corresponding to the real dirty area is retained in the comprehensive foreground mask, while a classification label is added to the real dirty area. The composite foreground mask after all erasure and retention operations is determined as the labeled dirt distribution map, wherein the labeled dirt distribution map only contains the actual dirt areas and their spatial location information.

7. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 1, characterized in that, For the actual dirty areas in the labeled dirt distribution map, multi-dimensional composite classification features are extracted from the standard illumination multimodal image sequence to generate composite classification feature tuples, including: Based on the spatial location boundary of the actual dirty area, image blocks of the actual dirty area are cropped from the spectral image frames of the standard illumination multimodal image sequence; The boundary closure curve is obtained by tracing the contour boundary of the real dirty area image block. The pixel range occupied by the region in the boundary closure curve is determined as the area parameter, and the total extension length of the boundary closure curve is determined as the perimeter parameter. The circularity description value is derived based on the ratio of the area parameter to the perimeter parameter. The gray-level transition amplitude is sampled along the boundary normal direction of the image block of the real dirty area, and the maximum sampled gray-level transition amplitude is used as the edge sharpness value. The frequency of occurrence of different gray levels within the image block of the real dirty area is statistically analyzed, and the internal texture uniformity value is quantified based on the dispersion of the frequency of occurrence. The area parameter, perimeter parameter, circularity description value, edge sharpness value, and internal texture uniformity value are sequentially grouped to form the geometric and edge feature subset in the composite classification feature tuple.

8. The method for classifying dirt and grime in premium packaging based on machine vision as described in claim 7, characterized in that, The step of extracting multi-dimensional composite classification features from the standard illumination multimodal image sequence to generate composite classification feature tuples for the actual dirty areas in the labeled dirty distribution map also includes: Locate the spectral image blocks of the real dirty area under different spectral conditions in the standard illumination multimodal image sequence; Take any two gray value pairs of corresponding pixel positions in the spectral image blocks, map the gray value pairs to the angle in the vector space, sum all the angle values ​​and divide by the total number of pixel positions to obtain the spectral angle change value. Obtain the color vector of the pixel in the real dirty area image block, retrieve the reference color vector of the same spatial position in the state without dirt, calculate the chromaticity angle between the pixel's color vector and the reference color vector, accumulate the chromaticity angles of the pixels and divide by the total number of pixel positions to obtain the average accumulated chromaticity angle. Morphological filtering is performed on the image blocks of the real dirty area. The gray-level distribution entropy value of the image block before filtering and the gray-level distribution entropy value of the image block after filtering are measured respectively. The reduction ratio is derived based on the ratio of the entropy value after filtering to the entropy value before filtering, and the texture entropy value change rate is obtained. The spectral angle change value, the cumulative average value of the chromaticity angle, and the rate of change of the texture entropy value are sequentially grouped to form the spectral and texture feature subsets in the composite classification feature tuple.

9. A machine vision-based method for classifying dirt and grime in premium packaging as described in claim 1, characterized in that, The step of logically comparing the composite classification feature tuples with a predefined dirt type feature rule library to obtain a dirt classification result image of the premium packaging to be detected includes: Retrieve a predefined dirt type feature rule library, wherein the dirt type feature rule library records standard dirt category labels and standard feature intervals bound to the labels, and the standard feature intervals are defined by a lower feature threshold and an upper feature threshold. Read the parameter values ​​from the composite classification feature tuple, and perform an inclusion test between the parameter values ​​and the standard feature interval of the standard dirt category label in sequence; When the parameter value is greater than or equal to the lower limit threshold of the feature and less than or equal to the upper limit threshold of the feature, it is determined that the parameter value falls into the standard feature range; When all parameter values ​​in the composite classification feature tuple fall within the standard feature range, the standard dirt category label is assigned to the actual dirt area of ​​the composite classification feature tuple. When a parameter value does not fall within the standard feature range of any standard dirt category label, an unclassified label is added to the actual dirty area; The actual dirty areas in the calibrated dirt distribution map are mapped back to the original image coordinate system according to their spatial location, and the assigned dirt type label or unclassified mark is spatially associated with the corresponding actual dirty area to generate the dirt classification result map of the premium packaging to be tested.

10. A machine vision-based system for classifying dirt and grime in premium packaging, characterized in that, The system for implementing the machine vision-based method for classifying dirt and grime in premium packaging as described in claim 1 includes: An imaging illumination calibration module is used to acquire a multimodal image sequence of the premium packaging to be inspected, and to perform dynamic illumination evaluation and adaptive enhancement on the image frames in the multimodal image sequence to generate a standard illumination multimodal image sequence. The dirty area identification module is used to separate the background feature layer from the standard illumination multimodal image sequence and identify the real dirty areas based on the spectral response differences, generating a labeled dirty distribution map; The composite feature extraction module is used to extract multi-dimensional composite classification features from the standard illumination multimodal image sequence for the real dirty areas in the labeled dirty distribution map, and generate composite classification feature tuples. The stain type determination module is used to logically compare the composite classification feature tuple with the predefined stain type feature rule library to obtain the stain classification result map of the premium packaging to be tested.

Images