Image recognition-based atomization effect cosmetic product detection method, system and device

Abstract

The application discloses an atomization effect cosmetic detection method based on image recognition and belongs to the intelligent detection field, and the method comprises the following steps: collecting a standard mist drop deposition image of a standard cosmetic emulsion under standard process parameters by using a standard spray gun; collecting a to-be-detected mist drop deposition image of a to-be-detected emulsion under the same conditions; extracting a global coarse-grained vector for global similarity calculation, and if the similarity is lower than a first threshold value, it is determined that the product is unqualified, otherwise, subsequent steps are entered; performing pose alignment based on positioning mark points, dividing detection partitions and verification partitions according to a preset grid, extracting fine-grained features of each partition based on an attention mechanism, and calculating the deviation of each partition; and constructing the fine-grained feature values and the deviation information of each partition into a spatial grid feature tensor, and outputting a quality type label by a pre-trained quality classification neural network. The application uses a constant temperature and humidity production line environment, replaces manual subjective interpretation with objective quantitative image analysis, and realizes online full detection and intelligent root cause diagnosis.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection. More specifically, this invention relates to a method, system, and device for detecting atomized cosmetic effects based on image recognition. Background Technology

[0002] In the cosmetics manufacturing industry, the fineness and uniformity of lotion-type products such as foundation, primer, moisturizing spray, and setting spray are core quality indicators that determine the product's skin feel, makeup effect, and batch stability. With the deepening of Industry 4.0 and intelligent manufacturing concepts, cosmetics production lines are accelerating their upgrade towards unmanned and fully automated operations, and cleanroom environments with constant temperature and humidity have become standard configurations for high-end production lines.

[0003] For a long time, the evaluation of the fineness and dispersion uniformity of cosmetic emulsions has mainly relied on offline manual testing. The most representative method is the fineness plate test: the operator takes a small amount of emulsion sample, spreads it on a metal plate with micron-level gradients, smooths it with a scraper, and observes the stripes or spots formed by particle aggregation on the surface under a specific angle of light, interpreting the particle size index based on personal experience. Although this method has low equipment costs, its limitations have become increasingly apparent with the popularization of automated production lines. First, the test results are highly dependent on the operator's experience and subjective judgment; the interpretation results of different personnel or the same personnel under different conditions may have significant deviations, making it difficult to establish quantitative quality standards. Second, for colored emulsions such as foundation with a certain degree of coverage, the light transmittance after scraping the film on the fineness plate is poor, further increasing the difficulty of manual interpretation. Third, this method is an offline sampling inspection, and the frequency of testing and the number of samples are limited by manpower, making it difficult to meet the needs of unmanned production lines for online full inspection.

[0004] Studies have shown a close physical correlation between the atomization quality of cosmetic emulsions and their microscopic particle size characteristics. Under standardized process conditions, the droplet deposition image formed by spraying emulsion onto the surface of a collection plate through a spray gun of fixed specifications at preset pressure and time is a stable characterization of the emulsion's intrinsic quality properties. The rheological properties of the emulsion, such as viscosity, surface tension, viscoelastic modulus, and degree of homogenization, collectively determine the droplet size distribution, spatial dispersion uniformity, and deposition pattern morphology. By quantitatively analyzing various indicators of the spray deposition image, the atomization performance and batch consistency of the emulsion can be inferred.

[0005] Currently, there is no mature solution in the quality inspection process of the cosmetics industry that systematically applies spray image analysis technology to the automated evaluation of emulsions. It is necessary to propose an image recognition-based atomization effect detection method that can fully utilize the existing constant temperature and humidity environment of cosmetic workshops and be seamlessly integrated into unmanned production lines. Through techniques such as coarse-fine granular grading, this method can achieve automated, quantitative, and intelligent evaluation of the atomization quality of cosmetic emulsions, filling a technological gap in the industry. Summary of the Invention

[0006] One objective of this invention is to provide a method, system, and device for detecting atomization effects in cosmetics based on image recognition, which can make full use of the existing constant temperature and humidity environment in cosmetic workshops and can be seamlessly integrated into unmanned production lines.

[0007] To achieve these objectives and other advantages according to the present invention, according to one aspect of the present invention, a method for detecting atomization effects in cosmetics based on image recognition is provided, comprising the following steps: S1. Using a standard spray gun and under standard process parameters, standard cosmetic emulsion is sprayed onto the image acquisition plate to acquire standard droplet deposition images; S2. Using the standard spray gun under the standard process parameters, the cosmetic emulsion to be tested is sprayed onto the image acquisition plate to acquire the droplet deposition image of the test. S3. Extract a global coarse-grained vector from the droplet deposition image to be tested to characterize the shape features of the deposition image. Calculate the global similarity between the vector and the global coarse-grained vector extracted from the standard droplet deposition image. If the similarity between the two is lower than the first threshold, determine that the spray effect of the cosmetic lotion to be tested is unqualified and terminate the test; otherwise, proceed to step S4. S4. Align the standard droplet deposition image with the droplet deposition image to be tested, and divide the droplet deposition image to be tested and the standard droplet deposition image into a one-to-one corresponding detection zone and verification zone according to a preset grid. S5. Based on the attention mechanism, extract the fine-grained feature information of each detection partition and the verification partition, compare the fine-grained feature information between the detection partition and the verification partition one by one and calculate the deviation, and output the fine-grained feature information and the calculated deviation information of each detection partition. S6. Construct the spatial grid feature tensor of the droplet deposition image to be tested by constructing the fine-grained feature information and calculation deviation information of each detection zone, input it into the pre-trained quality classification neural network, and output the quality type label of the cosmetic emulsion to be tested.

[0008] Preferably, the method for extracting the global coarse-grained vector is as follows: The standard droplet deposition image or the droplet deposition image to be tested is sequentially subjected to grayscale conversion, filtering and binarization to obtain the corresponding binary image; Multiple shape features with rotation and translation invariance are extracted from the binary image. The shape features include at least one or more of the following: area of ​​the droplet deposition region, perimeter of the outline, length of the major and minor axes of the fitted ellipse, rectangularity, circularity, eccentricity, and Hu moment. The extracted shape features are concatenated in a preset order to form the global coarse-grained vector.

[0009] Preferably, the similarity is calculated as follows: in, n Let be the dimension of the global coarse-grained vector. The global coarse-grained vector of the cosmetic emulsion to be tested. j Quantity, Let j be the j-component of the global coarse-grained vector for a standard cosmetic emulsion. ε This is a preset, extremely small positive number, used to prevent the denominator from being zero. D max This is the normalization constant determined based on the statistics of qualified samples. D This is a relative Euclidean distance. S c For similarity.

[0010] Preferably, the fine-grained feature information includes droplet coverage, average droplet size, and droplet size uniformity. The extraction of the fine-grained feature information includes the following process: A1. Input the partition image of the detection partition or verification partition into the attention map extractor. The attention map extractor includes a convolutional neural network, a feature map reshaping module and a correlation attention module. The convolutional neural network extracts the convolutional feature map of the partition image. The feature map reshaping module reshapes the convolutional feature map into a two-dimensional matrix. Then, the correlation attention module performs matrix multiplication on the two-dimensional matrix to generate an attention map for identifying the feature-prominent regions within the detection partition or verification partition. A2. Based on the attention response value, a stride vector generator is used to generate a wide stride vector along the horizontal direction of the image and a high stride vector along the vertical direction of the image based on the attention map. A3. Based on the wide stride vector and the high stride vector, perform a dynamic stride convolution operation on the convolution feature map to obtain a fine-grained feature map that reflects the details of the droplet morphology. A4. Input the fine-grained feature map into the pre-trained multi-task regression network, and output the fine-grained feature information of the detection partition or the verification partition respectively.

[0011] Preferably, the method by which the stride vector generator generates the wide stride vector and the high stride vector is as follows: For each column of the attention map along the horizontal direction, the maximum value of the attention response value of all pixels in that column is taken, and the maximum value is accumulated and integrated along the horizontal direction to obtain the horizontal cumulative distribution function. Then, the horizontal cumulative distribution function is inversely mapped to obtain the wide stride vector. For each row of the attention map along the vertical direction, the maximum value of the attention response value of all pixels in that row is taken, and the maximum value is accumulated and integrated along the vertical direction to obtain the vertical cumulative distribution function. Then, the vertical cumulative distribution function is inversely mapped to obtain the high stride vector. The wide stride vector and the high stride vector are both increasing sequences, and the difference between their adjacent elements is inversely proportional to the response intensity of the attention map at the corresponding position.

[0012] Preferably, the dynamic stride convolution operation includes: Preset the kernel size and dilation rate, and define the regular sampling grid. , k The kernel size; Based on the wide stride vector S w and high stride vector S h Determine each position on the fine-grained feature map on the convolutional feature map. (p, q) Corresponding sampling center point C p,q ,in C p,q The x-coordinate is determined by S w The q The vertical axis is determined by a single element, and is composed of... S h The p One element determines; For each sampling center point C p,q The grid is sampled on the convolutional feature map according to the rules. G Sampling is performed, and when the coordinates of the sampling point are not integers, bilinear interpolation is used to calculate the feature value of the sampling point. The sampled feature values ​​are weighted and summed with the convolution kernel weights to obtain the fine-grained feature map at the corresponding position. (p, q) The value of .

[0013] Preferably, constructing the spatial grid feature tensor includes the following steps: Each detection zone is arranged into a two-dimensional grid according to its original spatial position. Each grid cell corresponds to a multi-dimensional feature vector. The multi-dimensional feature vector is constructed based on the fine-grained feature information and the calculated deviation information, and includes at least the coverage deviation, average particle size deviation, particle size uniformity deviation, measured coverage value, and measured average particle size value of the zone. Standardize all grid cells of the same type of feature value in the two-dimensional grid so that the values ​​of each type of feature value are in a similar range. The standardized feature vectors are arranged according to the spatial position of the two-dimensional grid to form a spatial grid feature tensor.

[0014] Preferably, the quality classification neural network outputs the confidence level of each preset quality category through the spatial grid feature tensor of the droplet deposition image to be tested, and takes the quality category with the highest confidence level as the output quality type label; The quality categories include at least one or more of the following: qualified, excessive droplet accumulation, missing droplets, poor edge transition, and abnormally large particle size.

[0015] A cosmetic atomization effect detection system is provided for implementing the above-mentioned image recognition-based cosmetic atomization effect detection method, including: The image acquisition board has a hydrophilic surface and is equipped with positioning markers. A standard spray gun and spray gun control unit are used to spray cosmetic emulsion onto an image acquisition board according to a preset trajectory under standard process parameters. Image acquisition module, including at least one industrial camera; The image processing and control module includes: The standard template library unit stores the standard binary image, standard global coarse-grained vector, standard fine-grained feature information, and first threshold obtained based on the standard droplet deposition image; The coarse-grained decision unit is used to perform coarse-grained feature extraction and similarity calculation. The image alignment unit is used to perform marker extraction and spatial transformation; Fine-grained comparison and judgment unit, with embedded attention network, is used to perform feature extraction and bias judgment; The defect classification and diagnosis unit embeds a quality classification neural network, constructs a spatial grid feature tensor, and labels the quality type.

[0016] An electronic device is provided, including a memory and a processor, wherein the memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned image recognition-based method for detecting atomized cosmetic effects.

[0017] The present invention has at least the following beneficial effects: First, this invention is adaptable to the constant temperature and humidity environment of existing cosmetic workshops and unmanned automated production lines, enabling online full inspection. The inspection process requires no manual intervention and can be seamlessly integrated into the online inspection station at the front end of the filling line, achieving full inspection coverage of each batch of emulsion and providing key technical support for closed-loop quality control of unmanned production lines.

[0018] Secondly, this invention replaces traditional subjective manual interpretation with objective and quantitative image analysis, improving the consistency and traceability of the detection results. All judgments are based on repeatable mathematical calculations, and the detection results are not affected by subjective factors such as personnel, time, and fatigue. Furthermore, the fine-grained feature information and deviation information of each detection can be stored in the database, realizing full traceability of quality data.

[0019] Third, this invention, based on image recognition and intelligent diagnostic technology, can assist process engineers in quickly locating the root cause of problems and guiding process adjustments. Through the collaborative work of a coarse-fine granular two-level detection architecture and a quality classification neural network, it can determine whether the current batch of emulsion is qualified. In the case of non-compliance, it can output specific quality type labels—for example, "excessive droplet accumulation" indicates that the emulsion viscosity may be too high or the homogenization may be insufficient; "droplet loss" indicates that there may be partial nozzle blockage or formulation stratification; "poor edge transition" indicates that the spray fan formation is unstable; and "abnormally large particle size" indicates that the emulsion viscoelastic modulus is abnormal. The diagnostic conclusions point to potential problems in the emulsion formulation, homogenization process, or filling process. Process engineers do not need to rely on personal experience to check each step one by one; they can quickly pinpoint the problem direction based on the quality type labels and take targeted adjustment measures, shortening downtime and improving production line efficiency.

[0020] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a method flow diagram of one technical solution of the present invention; Figure 2 This is a system schematic diagram of one technical solution of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0023] It should be noted that, unless otherwise specified, the experimental methods described in the following embodiments are all conventional methods, and the reagents and materials described are all commercially available unless otherwise specified. In the description of this invention, the orientation or positional relationship indicated by the terms is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. It does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0024] like Figure 1 As shown, this invention provides a method for detecting the atomization effect of cosmetics based on image recognition, including the following steps: S1. Using a standard spray gun under standard process parameters, a standard cosmetic emulsion is sprayed onto an image acquisition plate to capture standard droplet deposition images. Specifically, the standard spray gun refers to a designated model spray gun that has been calibrated, has stable atomization performance, and is in good condition. It serves as a fixed spraying tool throughout the entire testing cycle and is not replaced regardless of changes in the sample being tested. Standard process parameters refer to a set of pre-set and fixed spraying control parameters, including at least temperature, humidity, spraying pressure, spraying time, vertical distance between the spray gun nozzle and the image acquisition plate, and the spray gun movement path. During the standard template establishment phase, these parameters remain strictly unchanged once determined to ensure that the acquisition conditions for all standard droplet deposition images are completely consistent. The standard cosmetic emulsion refers to the same qualified emulsion that has undergone rigorous quality inspection and confirmed that all rheological properties (such as viscosity, surface tension, and viscoelastic modulus) and atomization performance meet the design requirements. The image acquisition plate is a transparent or semi-transparent flat plate used to collect spray droplets. Its surface may be hydrophilic, and its back is equipped with a surface light source to provide uniform backlighting, making the droplets appear as clearly distinguishable dark or light spots in the image. In step S1, a standard cosmetic emulsion is loaded into a standard spray gun, and a complete spraying operation is performed according to standard process parameters. After the droplets have stably adhered to the acquisition plate, an industrial camera is triggered to capture a raw image, which is the standard droplet deposition image. In actual operation, to improve the robustness of the baseline data, the above acquisition process is usually repeated several times (e.g., ten times), and the statistical average of the multiple measurement results is taken as the final baseline value stored in the standard template library.

[0025] S2. Using the standard spray gun under the standard process parameters, the cosmetic emulsion to be tested is sprayed onto the image acquisition plate, and the droplet deposition image is acquired. Specifically, the droplet deposition pattern of the emulsion to be tested on the image acquisition plate is obtained under the exact same spraying conditions as in step S1, so as to compare it item by item with the standard droplet deposition images in the standard template library as reference images. The cosmetic emulsion to be tested refers to the same batch of emulsion in the production line waiting for atomization quality judgment. During the execution of step S2, except that the spraying object is replaced by the cosmetic emulsion to be tested instead of the standard cosmetic emulsion, all other conditions—including spray gun model, spraying pressure, spraying time, spraying distance, acquisition plate specifications, and lighting method—are strictly consistent with step S1. This ensures that any observable difference between the image to be tested and the standard image is directly due to the intrinsic quality difference between the cosmetic emulsion to be tested and the standard cosmetic emulsion, rather than systematic errors introduced by fluctuations in spraying process parameters or changes in hardware environment. The acquired image is called the droplet deposition image to be tested. This image, along with the standard droplet deposition images stored in the standard template library, will be used in subsequent analysis processes.

[0026] S3. Extract a global coarse-grained vector from the droplet deposition image to be tested to characterize the shape features of the deposition image. Calculate the global similarity between the vector and the global coarse-grained vector extracted from the standard droplet deposition image. If the similarity between the two is lower than the first threshold, determine that the spray effect of the cosmetic lotion to be tested is unqualified and terminate the test; otherwise, proceed to step S4.

[0027] Specifically, step S3 is the first screening stage in the entire testing process. Its function is to quickly identify and eliminate cosmetic lotions with serious macroscopic defects by utilizing shape features with low computational cost, thus avoiding further computationally intensive fine-grained analysis on obviously unqualified samples. The global coarse-grained vector refers to a set of numerical sequences that can describe the overall shape characteristics of the droplet deposition pattern at a macroscopic level. The global coarse-grained vector is composed of multiple shape descriptors with rotation, translation, and scaling invariance, such as the area, perimeter, fitted ellipse parameters, rectangularity, circularity, and Hu invariant moments of the droplet deposition region. These features focus on the overall contour shape of the droplet deposition region. Global similarity calculation calculates the distance between two high-dimensional vectors in the feature space; the smaller the distance, the higher the similarity. In this embodiment, Euclidean distance based on relative error weighting is used as the distance metric, and the distance value is mapped to the zero-to-one interval as the similarity score. The first threshold is a preset similarity threshold value, determined through statistical analysis of a large number of qualified and unqualified samples. If the similarity score between the droplet deposition image to be tested and the standard droplet deposition image is lower than the first threshold, it indicates a significant deviation in their shapes. Possible causes include excessively high or low emulsion viscosity leading to poor spraying, a sudden reduction in spray area, or formula stratification causing intermittent spraying. In this case, the system directly determines that the batch of cosmetic emulsion to be tested is unqualified and terminates the subsequent testing process, while also triggering an alarm signal.

[0028] S4. Align the standard droplet deposition image with the droplet deposition image to be tested, and divide the droplet deposition image to be tested and the standard droplet deposition image into a one-to-one corresponding detection zone and verification zone according to a preset grid.

[0029] Specifically, positioning markers are set at specific locations on the image acquisition board. These markers can be solid circles or crosshairs, facilitating automatic identification and precise positioning by image processing algorithms. When establishing the standard template library, the coordinates of the markers in the standard droplet deposition image are extracted and stored. A spatial mapping function is established by calculating the translation, rotation, and scaling relationships between the coordinates of the positioning markers in the test droplet deposition image and their corresponding coordinates in the standard droplet deposition image. This spatial mapping function is used to perform geometric correction on the entire test droplet deposition image, ensuring that the corrected test droplet deposition image corresponds point-by-point with the standard droplet deposition image in pixel space. After alignment, the system divides the standard droplet deposition image and the test droplet deposition image into regions using the same preset grid template. The number of rows and columns and the boundary positions of the grid are fixed during the template establishment stage. The grid cells divided on the standard droplet deposition image are called verification partitions. Each verification partition stores fine-grained feature information corresponding to that physical location as a baseline value. The grid cells divided on the droplet deposition image under test are called detection partitions. In step S5, each detection partition will independently extract fine-grained feature information and compare it with the baseline value of the corresponding verification partition. This one-to-one partition design ensures that each detection partition and verification partition physically correspond to the same area on the image acquisition board, thus giving the subsequent item-by-item deviation calculations a clear physical meaning.

[0030] S5. Based on the attention mechanism, extract the fine-grained feature information of each detection partition and the verification partition, compare the fine-grained feature information between the detection partition and the verification partition one by one and calculate the deviation, and output the fine-grained feature information and the calculated deviation information of each detection partition.

[0031] Specifically, under the unified spatial coordinate system established in step S4, the microscopic morphology of droplets within each detection zone and its corresponding verification zone is independently quantitatively characterized and precisely compared. Compared to the global coarse-grained vector, the focus of fine-grained feature information shifts from the macroscopic outline of the droplet deposition pattern to the microscopic quality within a single zone. Fine-grained feature information includes at least one or more of the following: droplet coverage, average droplet size, and uniformity of droplet size distribution. Droplet coverage reflects the percentage of the projected area of ​​droplets within a zone to the total area of ​​the zone; average droplet size is obtained by statistically analyzing the equivalent diameter of droplet clusters within the zone or by regression prediction based on texture features using a deep learning model, reflecting the overall fineness of the atomization; uniformity of droplet size is obtained by calculating indicators such as the coefficient of variation of the droplet size distribution, used to measure the consistency of droplet size.

[0032] Attention mechanisms mimic selective focusing in the human visual system, their core principle being to adaptively allocate computational resources to regions with higher information density within a partitioned image. In practice, the system processes local images of each partition to generate an attention map that identifies the importance of spatial locations. There are several ways to generate attention maps: for example, a lightweight pre-trained convolutional neural network can be used to extract multi-channel feature maps of the image, and the response intensity at each spatial location can be calculated using inter-channel correlation or gradient information; alternatively, classic computer vision algorithms can be employed, such as constructing saliency maps by analyzing the gray-level co-occurrence matrix, edge density, or speckle response intensity of local regions. Regardless of the specific method used, the fundamental function of the attention map is to distinguish between critical regions (such as regions with highly dense, severely overlapping, or abnormally large droplets) and secondary regions (such as regions with sparse droplets, regular shapes, or simply background). Guided by this attention map, the feature extraction process allocates higher computational resolution to critical regions, while using sparser computational resolution to quickly skim through secondary regions.

[0033] After feature extraction, the system will compare the fine-grained feature information between each detection partition and its corresponding validation partition based on a unified spatial index. Bias calculation can employ various mathematical forms, such as directly calculating the absolute value of the differences between various indicators, or calculating the percentage relative deviation between the difference and the benchmark value, providing a decision-making basis for the quality classification neural network in subsequent steps.

[0034] S6. Construct the spatial grid feature tensor of the droplet deposition image to be tested by constructing the fine-grained feature information and calculation deviation information of each detection zone, input it into the pre-trained quality classification neural network, and output the quality type label of the cosmetic emulsion to be tested.

[0035] The spatial grid feature tensor is a three-dimensional data structure that arranges the fine-grained feature information and deviation information of all detection partitions according to the original row and column positions of each partition on the image. Its first and second dimensions correspond to the row and column indices of the partitions, respectively, preserving the spatial distribution information of defects or anomalies; the third dimension corresponds to various feature indicators within each partition, such as coverage deviation, average particle size deviation, particle size uniformity deviation, measured coverage value, and measured average particle size value.

[0036] Before inputting into the quality classification neural network, all grid cells of the same type of feature value in the two-dimensional grid are standardized to ensure that features with different dimensions and numerical ranges are in similar value intervals. The quality classification neural network is a pre-trained three-dimensional convolutional neural network model that can simultaneously perform convolution operations along the spatial dimension (rows and columns) and the feature index dimension to capture the joint spatial and index patterns of quality anomalies. The network output layer corresponds to several preset quality categories, with each category outputting a confidence score representing the probability that the sample belongs to that category. Quality categories include at least one or more of the following: "Qualified," "Excessive Droplet Accumulation," "Droplet Missing," "Poor Edge Transition," and "Abnormally Large Particle Size." The system selects the category with the highest confidence score as the final output quality type label. For example, when the network outputs "Excessive Droplet Accumulation" with a high confidence score, it suggests that the batch of emulsion may have excessive viscosity or insufficient homogenization; when it outputs "Qualified," it indicates that the batch of emulsion maintains good consistency with the standard template in all partitions and indicators. This completes a full atomization effect detection process.

[0037] The standard template library establishment process described in step S1 is part of the offline preparation phase before the deployment of the testing system. It only needs to be performed once during system initialization, significant changes to standard process parameters, or periodic calibration and maintenance. In the actual online testing process, the system does not necessarily need to re-execute the physical spraying and image acquisition operations of step S1 for every batch of cosmetic emulsions to be tested. Specifically, before the testing system is officially put into production line operation, a standardized template database covering all in-production models can be pre-built and stored for each standard cosmetic emulsion model that needs to be tested, including the corresponding standard droplet deposition image, baseline contour information, global coarse-grained baseline vector, fine-grained baseline feature values ​​of each verification partition, and pass / fail threshold set. When actually performing the online testing tasks of steps S2 to S6, the system only needs to index and retrieve the corresponding pre-stored baseline data from this standardized template database according to the product model or batch code of the cosmetic emulsion to be tested. This completes the subsequent global similarity calculation, fine-grained feature deviation calculation, and quality classification judgment without having to repeat the standard emulsion spraying and standard image acquisition in each testing process.

[0038] In another technical solution, the method for extracting the global coarse-grained vector is as follows: The standard droplet deposition image or the droplet deposition image to be tested is sequentially subjected to grayscale conversion, filtering and binarization to obtain the corresponding binary image. Specifically, the processing procedure is completely applicable to both the standard image and the image to be tested. The following description uses a single image as an example.

[0039] Images captured by raw industrial cameras are usually multi-channel color images, which contain interference from environmental noise, uneven lighting, and other factors. Therefore, it is necessary to first convert the raw images into high-quality binary images to clearly separate the fog droplet deposition area from the background area.

[0040] Grayscale conversion is the process of converting a multi-channel color image into a single-channel grayscale image. Each pixel in the original color image is typically described by the brightness values ​​of three color channels: red (R), green (G), and blue (B), with each channel having a value ranging from 0 to 255. The mathematical expression for grayscale conversion is: In the above formula, x, y This represents the coordinate position of a pixel in an image. I gray (x, y) This is the grayscale value of the pixel. R(x, y) y) , G(x, y) , B(x, y) , representing the brightness values ​​of the pixel in the red, green, and blue channels, respectively; , representing the grayscale value of the pixel after conversion, also ranging from 0 to 255. The weighting coefficients are selected based on the differences in human eye sensitivity to different wavelengths of visible light—the human eye is most sensitive to green light, followed by red light, and least sensitive to blue light. Therefore, the green channel is assigned the highest weight, and the blue channel is assigned the lowest weight. After grayscale conversion, the image is compressed from three-dimensional data to one-dimensional data, significantly reducing the amount of subsequent computation while preserving the original texture structure information.

[0041] Filtering is used to suppress or eliminate noise interference in images. During spray image acquisition, noise sources mainly include thermal noise from the industrial camera sensor, slight brightness fluctuations from the surface light source, and tiny dust particles that may adhere to the image acquisition board surface. This solution employs a median filtering algorithm for noise reduction. The mathematical principle of this algorithm can be expressed as: for any position in the image... (x, y) Take a dimension centered at that position. w x h A rectangular window is used to collect the grayscale values ​​of all pixels within that window. These grayscale values ​​are then sorted by numerical value, and the median value after sorting is taken as the filtered grayscale value for that location. Mathematically, this can be expressed as: In the above formula, I med (x, y) For position (x, y) The grayscale value after filtering Med{•} This means taking the median value of all elements in the set. In the above formula... i and j Relative position within the rectangular window(x, y) The horizontal and vertical offsets; a and b These represent the half-width and half-height of the rectangular window, respectively. When using a common 3x3 filter window... a = b=1 The rectangular window covers a total of nine pixels. Median filtering is a nonlinear filtering method. Its advantage lies in its ability to effectively eliminate isolated salt-and-pepper noise and impulse noise while better preserving the sharpness of droplet edges in the image, thus avoiding blurring of droplet boundaries due to smoothing operations and affecting the accuracy of subsequent particle size measurements.

[0042] Binarization transforms the filtered grayscale image into a binary image containing only black (background) and white (foreground) pixels, thus clearly segmenting the blank areas on the image acquisition board in the fog deposition region. This scheme uses Otsu's method to determine the binarization threshold, which uses a certain grayscale value... T As a candidate segmentation threshold, image pixels are divided into foreground class (grayscale values ​​greater than 100%). T The pixels (corresponding to the fog droplet deposition area) and the background class (grayscale value less than or equal to) T The pixels (corresponding to the background of the image acquisition board) are used to calculate the pixel count ratio and average gray value of the two classes respectively, and then the inter-class variance of the two classes is calculated.

[0043] The threshold that maximizes the inter-class variance T This is the optimal binarization threshold. Its mathematical expression is as follows: in, For inter-class variance, T Iterate through all integer values ​​between 0 and 255; Indicates T When the segmentation threshold is set, it represents the proportion of background pixels to the total number of pixels in the image. Indicates T When the segmentation threshold is set, the ratio of foreground pixels to the total number of pixels in the image satisfies the following condition: ; The average grayscale value of all pixels in the background class; This represents the average grayscale value of all pixels in the foreground class.

[0044] Multiple shape features with rotation and translation invariance are extracted from the binary image. These shape features include at least: the area of ​​the droplet deposition region, the perimeter of the outline, the lengths of the major and minor axes of the fitted ellipse, rectangularity, circularity, and eccentricity. Hu One or more of the following: a rectangle; Total area SThe total number of white pixels in a binary image multiplied by the square of the pixel equivalent directly reflects the total physical area covered by the spray on the image acquisition panel. This is a fundamental indicator for measuring whether the spray volume is sufficient or excessive. The larger the area value, the wider the deposition range.

[0045] Outline perimeter P The perimeter is the sum of the Euclidean distances between adjacent points in the contour point sequence multiplied by the pixel equivalent. It reflects the total length of the boundary of the deposition pattern and, in conjunction with the area, can further deduce the compactness or expansiveness of the pattern.

[0046] Fitted ellipse major axis length L major With minor axis length L minor The least-squares ellipse fitting is performed on the contour point set to solve the generalized eigenvalue problem. The basic idea of ​​least-squares ellipse fitting is to find an ellipse that minimizes the sum of the squared algebraic distances from all points on the contour to the boundary of the ellipse. This problem can be transformed into solving the generalized eigenvalue problem. The major axis length represents the span of the deposition pattern along the main extension direction, and the minor axis length represents the span perpendicular to the main extension direction. Together, they describe the directional extension characteristics of the spray pattern.

[0047] Rectangularity This feature reflects the degree to which the shape of the droplet deposition pattern is close to a rectangle. When the deposition pattern is a full rectangle, this ratio approaches one; when there are large voids inside the pattern or the boundaries are extremely irregular, this ratio decreases.

[0048] Circularity This feature measures how close a deposition pattern is to a circle. When the pattern is perfectly circular, the circularity value is one; the more the shape deviates from a circle and the more intricate and complex the boundaries, the smaller the value.

[0049] Eccentricity The value ranges from zero to one. The closer the eccentricity is to zero, the closer the deposition pattern is to a circle; the closer the eccentricity is to one, the flatter and more elongated the pattern is.

[0050] Calculated using the standard formula based on second- and third-order normalized central moments. Hu Rectangle Hu 1~ Hu 7. The calculation process first calculates the geometric moments and central moments of each order from the binary image or contour region, then eliminates the scale effect through normalization, and finally obtains seven moment features with rotation, translation and scaling invariance through algebraic combination. Hu The rectangle characterizes the shape information of the deposition pattern as a whole, is not sensitive to the subtle deformation of the pattern, but has a good ability to distinguish the differences in the overall outline.

[0051] The above eigenvalues ​​are concatenated into a global coarse-grained vector. The dimension of this vector depends on the number of features actually selected. When all the above features are used, the vector dimension is fourteen. The vector extracted from the standard droplet deposition image using this process is the standard global coarse-grained vector, which is stored in the standard template library for comparison. The vector extracted from the droplet deposition image to be tested using the same process is used as the global coarse-grained vector to be tested, which is used for global similarity calculation in subsequent steps.

[0052] In another technical solution, the similarity is calculated as follows: in, n Let be the dimension of the global coarse-grained vector. The global coarse-grained vector of the cosmetic emulsion to be tested. j Quantity, For the global coarse-grained vector of standard cosmetic emulsions j Quantity, ε This is a preset, extremely small positive number, used to prevent the denominator from being zero. D max This is the normalization constant determined based on the statistics of qualified samples. D This is a relative Euclidean distance. S c For similarity.

[0053] Normalization constant D max The method for determining the distance is as follows: Collect standard droplet deposition images of several batches of known qualified standard cosmetic emulsions, extract their global coarse-grained vectors respectively, calculate the relative Euclidean distance with the standard global coarse-grained vectors, and obtain a sample distribution of a set of distance values. Take the upper quantile of this distribution (e.g., the 95th percentile) or directly take the upper limit empirical value of all D values ​​as the distance distribution. D max The value of is determined in this way. The physical meaning of this setting is that the relative Euclidean distance of the vast majority of qualified samples is less than 1. D max Therefore, the similarity score of qualified samples will fall within a relatively high range close to one; while the relative Euclidean distance of unqualified samples will be significantly greater than that of standard fog deposition images due to their obvious macroscopic morphology. D max This caused the similarity score to plummet.

[0054] The calculated similarity score S c With the preset first threshold θ cA comparison is then made. The typical range for the first threshold is between 0.7 and 0.9, with the specific value determined experimentally based on the stringent testing requirements of the actual production line. If... S c If the spray pattern is below the first threshold, it indicates a significant overall deviation between the tested droplet deposition image and the standard droplet deposition image. Therefore, the spray effect of this batch of cosmetic emulsion is deemed unqualified, and subsequent testing procedures are terminated. S c If the result is not lower than the first threshold, it indicates that the macroscopic shape is basically consistent, and the detection process can continue to the next step.

[0055] In another technical solution, the fine-grained feature information includes droplet coverage, average droplet size, and droplet size uniformity. The extraction of the fine-grained feature information includes the following process: A1. Input the partition image of the detection partition or verification partition into the attention map extractor. The attention map extractor includes a convolutional neural network, a feature map reshaping module, and a correlation attention module. The convolutional neural network extracts the convolutional feature map of the partition image. The feature map reshaping module reshapes the convolutional feature map into a two-dimensional matrix. Then, the correlation attention module performs matrix multiplication on the two-dimensional matrix to generate an attention map for identifying the feature-prominent regions within the detection partition or verification partition.

[0056] The partition image of the detection partition or verification partition to be tested is input into the attention map extractor, which includes a convolutional neural network, a feature map reshaping module and a correlation attention module, to generate an attention map for identifying the feature-prominent regions within the partition.

[0057] Convolutional neural networks (CNNs) are used to perform multi-layer nonlinear feature extraction on input partitioned images, transforming raw pixels into a high-dimensional feature table with semantic discriminative capabilities. This CNN can employ a pre-trained lightweight backbone architecture, such as a ResNet-18 network with some downsampling layers removed, followed by dilated convolutional layers. Dilated convolutional layers can expand the receptive field of neurons without increasing the number of parameters, enabling them to simultaneously capture texture, edge, and aggregation morphology information at different scales in the fog-droplet deposition pattern. After forward propagation of the partitioned image through the CNN, a high-dimensional convolutional feature map is output. ,in C This represents the number of channels in the convolutional feature map. H and W These represent the spatial height and width of the convolutional feature map, respectively.

[0058] The feature map reshaping module converts the convolutional feature maps of the above three-dimensional structure into a two-dimensional matrix form. This module reshapes the feature maps to a size of... C x H x W Convolutional feature maps FThe operation involves flattening the surface in the spatial dimension; specifically, flattening each channel... H× W The two-dimensional response graph is straightened into a graph of length . H W A one-dimensional vector is then formed, and the vectors of all C channels are stacked along the row direction to obtain a vector of size C. C x (H W) Two-dimensional matrix M .

[0059] Reshaped two-dimensional matrix M The physical meaning is as follows: each row of the matrix corresponds to a sequence of response values ​​for a specific convolutional channel at all spatial locations, describing the distribution of the visual pattern detected by that channel across the entire image; each column of the matrix corresponds to a vector of response values ​​for a specific spatial location at all convolutional channels, i.e., the multi-channel feature descriptor for that location. This reshaping operation unifies the spatial and channel information, which were originally decoupled in the data structure, into a single matrix framework, laying the data structure foundation for subsequent cross-channel global correlation modeling.

[0060] The relevance attention module is used for two-dimensional matrix-based... M Attention maps are generated. When characterizing droplet deposition quality, the visual pattern is that multiple related channels simultaneously produce high responses in key regions. Therefore, by analyzing the response correlations between different channels, we can determine which spatial locations are simultaneously attended by numerous interconnected channels, thus highlighting areas with collective consensus.

[0061] Matrix multiplication specifically includes the following steps: B1: Calculate the correlation between channels, and convert the two-dimensional matrix. M Its transpose M T Multiplying them together gives a size of C× C Symmetric matrix MM T . MM T The Middle i Line number j The element of the column is equal to the first element. i The response values ​​of the first channel at all spatial locations are compared with those of the second channel. j The inner product of the response values ​​of each channel. A larger value indicates a higher level of integration. i The convolutional channel and the first j Each convolutional channel exhibits a more consistent co-occurrence or co-suppression relationship across the image.

[0062] B2. Symmetric matrix MM T As the weight matrix, multiply by a two-dimensional matrix on the left. M The weighted feature matrix is ​​obtained. (MMT )M The feature matrix size remains the same. C x (H W) Specifically, for each channel, its correlation with all other channels is used as a weight to linearly combine the original features of all channels, thereby generating the weighted modulated features of that channel. If a channel has a strong correlation with many other high-response channels, its features will be further enhanced; conversely, if a channel's response pattern is generally inconsistent with other channels, its features will be relatively suppressed.

[0063] B3. The weighted modulated feature matrix (MM T )M Summation is performed along the channel dimension. Because... (MM T )M Each column in the matrix corresponds to the spatial response distribution of a modulated channel. Summing each column along the channel dimension yields a matrix of length . H W A one-dimensional vector. Restore this one-dimensional vector to its original spatial order. H x W The two-dimensional spatial arrangement yields the attention map. The attention map A up Spatial dimensions and the convolutional feature map F The spatial dimensions are the same.

[0064] In actual deployment, the generated attention map can be... A up conduct Softmax Normalization maps all element values ​​to the [0,1] interval and sums to 1, transforming it into a standard probability distribution form, which helps with the numerical stability of the subsequent stride vector generation step.

[0065] Attention Map A up The response value at each location reflects the discriminative importance of that spatial location after comprehensively considering all convolutional channels and their interrelationships. Regions with higher response values ​​indicate that the droplet texture, edge, or aggregation morphology features contained within that region are simultaneously deemed significant by most relevant channels, making them more critical for characterizing spray quality. These correspond to areas of dense droplet accumulation, areas with significant anomalies in droplet size, or areas with discontinuous edge transitions. Conversely, regions with low response values ​​are often background areas with regular droplet morphology and normal distribution.

[0066] A2. Based on the attention response value, a stride vector generator is used to generate a wide stride vector along the horizontal direction of the image and a high stride vector along the vertical direction of the image, based on the attention map. The method by which the stride vector generator generates the wide stride vector and the high stride vector is as follows: For each column of the attention map along the horizontal direction, the maximum value of the attention response value of all pixels in that column is taken, and the maximum value is accumulated and integrated along the horizontal direction to obtain the horizontal cumulative distribution function. Then, the horizontal cumulative distribution function is inversely mapped to obtain the wide stride vector. Specifically, the attention map along the horizontal direction A up For each column, the maximum value of the attention response is taken, and the maximum value is cumulatively integrated along the horizontal direction to obtain the horizontal cumulative distribution function. F x Then, the horizontal cumulative distribution function is mapped inversely to obtain the wide stride vector.

[0067] In the above formula, c(j) Indicates the first j The maximum attention response value of the column, i For row index, j For column indexes, H For the height of the attention map, W To determine the width of the attention map, the maximum attention values ​​of the above columns are accumulated and summed horizontally from left to right to obtain the horizontal cumulative distribution function. F x .

[0068] because c(j) ≥0, F x (n) It is a monotonically non-decreasing function, and its growth rate is directly proportional to the intensity of the attention response at the corresponding column position—in regions with strong attention responses, c(j) The value is relatively large. F x (n) Rapid rise; in areas of weak attentional response, c(j) The value is small. F x (n) A gradual rise.

[0069] Set the desired output width to W out (This value is a preset parameter), will F x The range of values ​​is [0, F x (W) Divided into W out Divide the data into several equal intervals, and take the right endpoint of each interval as the target cumulative value. For each target cumulative value, in...F x Find the column index that first reaches or exceeds this value to obtain the stride vector. S x The expression is: in, S w It is a wide stride vector. n The number of columns accumulated horizontally. q For the column index of the output fine-grained feature map, F x (W) It is the sum of the horizontal cumulative distribution functions. For the first q The target cumulative value corresponding to each equivalence point.

[0070] For each row of the attention map along the vertical direction, the maximum value of the attention response of all pixels in that row is taken, and the maximum value is accumulated and integrated along the vertical direction to obtain the vertical cumulative distribution function. Then, the vertical cumulative distribution function is inversely mapped to obtain the high stride vector. A completely symmetrical operation is used along the vertical direction, and the expression is: in, S h For high-strike-rate vectors, p This refers to the row index of the output fine-grained feature map. F y (H) It is the sum of the vertical cumulative distribution functions. m This represents the number of rows accumulated vertically. H out The desired output height.

[0071] The wide stride vector and the high stride vector are both increasing sequences, and the difference between their adjacent elements is inversely proportional to the response intensity of the attention map at the corresponding position.

[0072] A3. Based on the wide stride vector and high stride vector, perform dynamic stride convolution operation on the convolution feature map to obtain a fine-grained feature map that reflects the details of the droplet morphology. The core purpose of this step is to use the wide stride vector and high stride vector generated in step A2 to perform non-uniform sampling and convolution operation on the convolution feature map corresponding to the partitioned image, so as to extract the fine droplet morphology features with high density in key areas and scan with low density in non-key areas, thereby achieving an adaptive balance between computing resources and detection accuracy.

[0073] The dynamic stride convolution operation includes: The kernel size and dilation rate are preset. The kernel size is usually an odd value (e.g., 3 or 5), which determines the spatial neighborhood covered in each sampling. The dilation rate determines the spacing between sampling points in the sampling grid. Based on these parameters, a regular sampling grid with the center of the kernel as the origin can be defined. G It contains k x k The coordinate offset of each sampling point relative to the center position. This sampling grid can be represented as: in, k The kernel size is the convolution kernel size. d x 、d y These represent the offsets of the sampling points relative to the center of the convolution kernel in the horizontal and vertical directions, respectively. This indicates rounding down. In actual sampling, the actual position of each sampling point needs to be multiplied by the porosity to control the sampling step size.

[0074] Based on the wide stride vector S w and high stride vector S h Determine each position on the fine-grained feature map on the convolutional feature map. (p, q) Corresponding sampling center point C p,q ,in C p,q The x-coordinate is determined by S w The q The vertical axis is determined by a single element, and is composed of... S h The p Each element determines the height of the fine-grained feature map; the height of the fine-grained feature map is set to... H out Width is W out Then for the first [characteristic] in the fine-grained feature map p line, number q The column position, and the coordinates of its corresponding sampling center point are: The x-axis y-axis Due to the stride vector S w and S h These are increasing sequences along the horizontal and vertical directions, respectively. The difference between adjacent elements is smaller in regions with strong attention response and larger in regions with weak attention response. Therefore, the sampling center point... C p,qThe spatial distribution of the convolutional feature map exhibits a clear non-uniformity—the sampling center points are densely distributed in regions with high attention response and sparsely distributed in regions with low attention response. This mechanism ensures that computational resources are automatically tilted towards discriminative regions.

[0075] For each sampling center point C p,q The grid is sampled on the convolutional feature map according to the rules. G Sampling is performed, and when the coordinates of the sampling point are non-integer, bilinear interpolation is used to calculate the feature value of the sampling point; for regular sampling grids... G Each offset in (d x ,d y ) The actual sampling position on the convolutional feature map has the following coordinates: .

[0076] The sampled feature values ​​are weighted and summed with the convolution kernel weights to obtain the fine-grained feature map at the corresponding position. (p, q) The value of the convolution kernel weight is related to the sampling grid. G A one-to-one learnable parameter matrix, whose size is the same as the sampling grid, is denoted as . For each offset in the sampling grid (d x ,d y ) There exists a unique weighting coefficient. w(dx, dy) Correspondingly, the specific operation of weighted summation can be expressed as: in, Y(p, q) This indicates that the output fine-grained feature map is in the first... p line, number q The response value for the column position; w(dx, dy) For the convolution kernel at the offset w(dx, dy) Weight parameters at the location; This represents the feature value calculated by bilinear interpolation at specified floating-point coordinates in the convolutional feature map. and These are the elements at corresponding positions in the width stride vector and the height stride vector, respectively; r The preset porosity is used to adjust the spacing between sampling points.

[0077] The physical meaning of the above weighted summation process is as follows: For each spatial location on the fine-grained feature map, the system determines a non-uniform sampling center on the corresponding convolutional feature map based on the stride vector, and collects a set of local features in the regular neighborhood based on this center. Then, the system uses the trained convolutional kernel weights to match and encode the visual pattern of this local region. Since the spatial distribution of the sampling centers is related to the response intensity of the attention map, in high attention response regions, the spacing between sampling centers corresponding to adjacent positions on the fine-grained feature map is small, and the output response points generated in the same physical region are more dense, thus preserving richer details of the droplet morphology; in low attention response regions, the spacing between sampling centers is large, the output response points are relatively sparse, and only macroscopic structural information is preserved.

[0078] By traversing all row and column indices and repeating the above operations of determining the sampling center, regular grid sampling, bilinear interpolation, and weighted summation, a complete fine-grained feature map can be obtained. This fine-grained feature map is the direct input to the multi-task regression network in subsequent step A4. The response value at each spatial location encodes visual semantic information closely related to fogging quality, such as texture coarseness, edge sharpness, and overlap of the fog deposition pattern within the corresponding local receptive field. This provides a highly discriminative feature representation for accurately predicting the fog coverage, average particle size, and particle size uniformity of the region.

[0079] A4. Input the fine-grained feature map into the pre-trained multi-task regression network, and output the fine-grained feature information of the detection partition or the verification partition respectively.

[0080] Multi-task regression networks can simultaneously predict multiple related but independent quality metrics during a single forward propagation, rather than training a separate network model for each metric. This design has two advantages: First, multiple tasks share the same feature extraction backbone, which can effectively reduce the number of model parameters and computational overhead, facilitating real-time inference at online detection stations; second, coverage, average particle size, and particle size uniformity are all closely related to the atomization behavior of emulsions in terms of physical causes, and joint training enables the network to learn the potential coupling relationships between these metrics, thereby improving the prediction accuracy of each individual metric.

[0081] In one implementation, the multi-task regression network structure consists of a global average pooling layer and multiple fully connected layers connected in series. The function of the global average pooling layer is to pool layers with a size of [missing information]. H out ×W outThe fine-grained feature map is compressed in the spatial dimension—the arithmetic mean of the response values ​​at all spatial locations is calculated for each channel of the fine-grained feature map, thus transforming a three-dimensional feature tensor into a one-dimensional feature vector. The advantages of this operation are: it significantly reduces the number of parameters in the fully connected layers, effectively preventing overfitting; and it makes the network somewhat invariant to small perturbations in the spatial location of the fine-grained feature map, enhancing the model's generalization ability to different batches of samples.

[0082] The one-dimensional feature vector after global pooling is then fed into several fully connected layers for non-linear mapping. The fully connected layers can be designed with a gradually decreasing "funnel-shaped" structure, for example, with 256, 128, and 64 neurons respectively. Each layer is followed by a ReLU activation function and Dropout regularization to prevent overfitting. Finally, the output layer generates multiple predictions simultaneously. The number of nodes in the output layer corresponds to the number of fine-grained feature metrics to be predicted: if coverage, average particle size, and particle size uniformity need to be output simultaneously, the output layer contains three nodes.

[0083] The training of the multi-task regression network was completed offline before the detection system was deployed. The training data was constructed as follows: a large number of partitioned images of standard emulsions sprayed under different batches and quality conditions were collected. For each partitioned image, fine-grained feature maps were extracted according to steps A1 to A3 as network input. Simultaneously, the actual droplet coverage, average particle size, and particle size uniformity of the partition were obtained using a high-precision particle size analysis instrument (e.g., a laser particle size analyzer, such as the Malvern Mastersizer series) as supervision labels. During training, the multi-task regression network continuously adjusted the connection weights between neurons in each layer through the backpropagation algorithm, gradually reducing the error between the network's predicted values ​​and the true label values.

[0084] After training, the weight parameters of the multi-task regression network are fixed. When deployed to the detection system, it only performs forward inference calculations and does not update the parameters further. For each input fine-grained feature map, the network outputs three floating-point values, corresponding to the predicted droplet coverage, average droplet size, and uniformity of droplet size distribution for that region. These predicted values ​​constitute the fine-grained feature information for that region and will be fed into the subsequent bias calculation stage.

[0085] In another technical solution, constructing a spatial grid feature tensor includes the following steps: The detection partitions are arranged into a two-dimensional grid according to their original spatial positions. Each grid cell corresponds to a multi-dimensional feature vector, which is constructed based on the fine-grained feature information and calculated deviation information. This multi-dimensional feature vector includes at least the partition's coverage deviation, average particle size deviation, particle size uniformity deviation, measured coverage value, and measured average particle size value. The system has obtained a set of fine-grained feature information and deviation information for each detection partition. All detection partitions are arranged sequentially according to their original row and column indices on the image, forming a regular two-dimensional grid structure. The number of rows and columns of the two-dimensional grid is determined by the grid division scheme in step S4. Each detection partition corresponds to a multi-dimensional feature vector. This multi-dimensional feature vector includes at least the following five components: the coverage deviation, average particle size deviation, and particle size uniformity deviation between the detection partition and the corresponding verification partition, as well as the measured coverage value and measured average particle size value of the partition. The first three deviation information directly reflect the degree and direction of deviation of the partition's various indicators relative to the standard template; the latter two measured values ​​retain the original physical magnitude information of the partition.

[0086] Standardize all grid cells of the same type of feature value in the two-dimensional grid to make the values ​​of each type of feature value fall within a similar range. To avoid channels with larger value ranges dominating in subsequent classification, Z-score standardization is performed on all grid cells of the same type of feature value in the two-dimensional grid to make their mean zero and standard deviation 1. This makes the influence weight of each channel on the subsequent neural network more balanced, while preserving the relative difference information between different partitions.

[0087] The standardized feature vectors are arranged according to their spatial positions within the two-dimensional grid to form a spatial grid feature tensor. The standardized feature vectors of all detection partitions are then placed sequentially according to their original row and column positions. The overall data can be considered as a tensor of size [size missing]. G r × G c × K A three-dimensional array, in which G r and G c These represent the number of rows and columns of the detection partition grid, respectively. K The number of feature channels corresponding to each partition. This three-dimensional array is the spatial grid feature tensor, whose first two dimensions preserve the spatial topology of each detection partition, and the third dimension encodes the collaborative variation relationship of multiple quality indicators.

[0088] In another technical solution, the quality classification neural network outputs the confidence level of each preset quality category through the spatial grid feature tensor of the droplet deposition image to be tested, and takes the quality category with the highest confidence level as the output quality type label. The quality category includes at least one or more of the following: qualified, excessive droplet accumulation, missing droplets, poor edge transition, and abnormally large particle size.

[0089] The quality classification neural network can be directly implemented using mature image classification models in the industry, without the need for customized modifications. In one embodiment, the quality classification neural network adopts a three-dimensional convolutional neural network, which can be custom-built based on open-source frameworks such as PyTorch / TensorFlow, and designed with reference to open-source three-dimensional anomaly detection frameworks such as C3D-AD. The network includes multiple three-dimensional convolutional layers, three-dimensional pooling layers, global average pooling layers, and fully connected classification layers. The input is a spatial grid feature tensor, which extracts spatial-channel joint features through three-dimensional convolution, and compresses it into a one-dimensional feature vector through global pooling. The confidence scores of each quality category are output by the fully connected layer and the Softmax function. The one with the highest confidence score is taken as the quality type label output. The quality categories include acceptable, excessive droplet accumulation, missing droplets, poor edge transition, and abnormally large particle size.

[0090] The training of the quality classification neural network was completed offline before system deployment. Training data came from a large number of historical detection samples already labeled with quality categories. The spatial grid feature tensors of the samples were extracted according to the aforementioned steps, and the quality category labels were determined through manual verification or offline high-precision detection instruments. The training process employed the standard cross-entropy loss function and the Adam optimizer. After the trained model weights were solidified, they were deployed to the online detection station. During inference, only forward propagation calculations were performed, and no further parameter updates were performed.

[0091] The spatial grid feature tensor is constructed from the fine-grained feature values ​​and bias information of each detection partition output in step S5. Specifically, all detection partitions are arranged into a two-dimensional grid according to their original row and column positions. Each two-dimensional grid cell is filled with a multi-dimensional feature vector, which is then standardized to form a tensor of size [size missing]. G r × G c × K The spatial grid feature tensor is used as the input to the quality classification neural network. After forward propagation, the output layer provides the confidence score for each quality category, and the highest confidence score is taken as the quality type label.

[0092] A cosmetic atomization effect testing system is provided for the aforementioned testing method. This system utilizes the constant temperature, humidity, and clean environment commonly found in cosmetic workshops and can be seamlessly integrated into the online testing station at the front end of the filling line, achieving automated, full-coverage atomization quality assessment for each batch of cosmetic emulsions. The entire testing system consists of two main parts: hardware execution components and software processing modules. The following provides a detailed description of each component.

[0093] The hardware components mainly include an image acquisition board, a standard spray gun and spray gun control unit, and an image acquisition module.

[0094] The image acquisition plate is used to capture spray droplets and form a deposition pattern. The acquisition plate is made of a light-transmitting material (such as optical-grade acrylic or ultra-clear glass) with a hydrophilic surface treatment, allowing the droplets to spread into clearly defined circular patterns. The acquisition plate has at least four high-contrast positioning markers (such as solid circles or cross lines) at its four corners, serving as a spatial reference for subsequent image alignment.

[0095] The standard spray gun uses a metered and calibrated specified model. Optional options include Spraying Systems Co.'s AutoJet 2150+ modular automatic spray system or Graco's G40 automatic air-assisted spray gun. The spray gun control unit includes a pressure regulation module, a time control module, and a motion execution module, which are used to precisely maintain spray pressure, control spray duration, and drive the spray gun along a preset trajectory, ensuring consistent spray conditions for each application.

[0096] The image acquisition module includes at least one industrial camera and a matching backlight source. The industrial camera can be a Basler Ace Classic series camera (e.g., one equipped with a Sony IMX250 CMOS chip) or a Hikvision CA series GigE industrial area scan camera, with a resolution of at least 5 megapixels. The illumination source uses a high-uniformity LED area light source, placed on the back side of the acquisition board to form backlight illumination, creating high contrast between the fog droplet area and the background, facilitating subsequent image segmentation and feature extraction.

[0097] The software is integrated into the image processing and control module, which is typically deployed on an industrial control computer or embedded vision controller. This module includes a standard template library unit, a coarse-grained judgment unit, an image alignment unit, a fine-grained comparison and judgment unit, and a defect classification and diagnosis unit. These units are called sequentially according to the detection process, forming a complete detection data stream.

[0098] The system stores a standard binary image obtained from a standard droplet deposition image, a standard global coarse-grained vector, standard fine-grained feature information for each verification partition, and a first threshold. This data is created offline before system deployment and is available for use by each decision unit.

[0099] The coarse-grained determination unit performs grayscale, filtering, and binarization processing on the droplet deposition image to be tested, and extracts shape features such as area, perimeter, fitted ellipse parameters, rectangularity, circularity, eccentricity, and Hu moment to construct the global coarse-grained vector to be tested. It calculates the similarity between this vector and the standard global coarse-grained vector. If the similarity is lower than a first threshold, the image is deemed unqualified and the detection process terminates; otherwise, it proceeds to the next unit.

[0100] The image alignment unit extracts the sub-pixel-level center coordinates of the positioning markers in the standard image and the image under test. Based on the least squares method, it solves the affine transformation matrix and performs spatial transformation and bilinear interpolation resampling on the image under test to obtain a corrected image that is pixel-level aligned with the standard image. The corrected image and the standard image are then divided into corresponding detection and verification partitions according to a preset grid.

[0101] The fine-grained comparison and judgment unit embeds an attention network to perform feature extraction on each partition image: First, a convolutional feature map is extracted through a convolutional neural network; then, an attention map is generated through a feature map reshaping module and a correlation attention module; based on the attention map, a non-uniform stride vector is generated, and dynamic stride convolution is performed to obtain a fine-grained feature map; finally, a multi-task regression network outputs the fine-grained feature information (droplet coverage, average droplet size, and size uniformity) of each partition. The feature deviation between the detection partition and the corresponding verification partition is calculated partition by partition, compared with the allowable deviation threshold, and a pass / fail determination is made.

[0102] The defect classification and diagnosis unit embeds a quality classification neural network, which is activated when a fine-grained comparison determines non-compliance. The measured values ​​and deviation information of the fine-grained features of each detection zone are arranged into a two-dimensional grid according to their original row and column positions. Each grid cell corresponds to a five-dimensional feature vector (coverage deviation, average particle size deviation, particle size uniformity deviation, measured coverage value, and measured average particle size value). Features of the same type are standardized separately and stacked to form a spatial grid feature tensor.

[0103] The quality classification neural network employs a 3D convolutional neural network, which can be custom-built based on open-source frameworks such as PyTorch / TensorFlow, and its design references open-source 3D anomaly detection frameworks such as C3D-AD. The network includes multiple 3D convolutional layers, 3D pooling layers, global average pooling layers, and fully connected classification layers. The input is a spatial grid feature tensor, which extracts spatial-channel joint features through 3D convolution, and compresses them into a one-dimensional feature vector through global pooling. The fully connected layer and the Softmax function output the confidence score for each quality category. The label with the highest confidence score is output as the quality type label, including acceptable, excessive droplet accumulation, missing droplets, poor edge transition, and abnormally large particle size.

[0104] An electronic device includes a memory and a processor. The memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned image recognition-based method for detecting atomized cosmetic effects. The computer device can be any terminal device, including mobile phones, laptops, desktop computers, tablets, PDAs (Personal Digital Assistants), POS (Point of Sales), and in-vehicle computers.

[0105] Computer instructions are stored in a readable storage medium, such as a computer floppy disk, USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0106] It should be noted that although the steps are described in a specific order above, this does not mean that they must be performed in that order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required functionality is achieved. The number of devices and processing scale described herein are for simplification of the invention; applications, modifications, and variations of this invention will be readily apparent to those skilled in the art.

[0107] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for detecting the atomization effect of cosmetics based on image recognition, characterized in that, Includes the following steps: S1. Using a standard spray gun and under standard process parameters, standard cosmetic emulsion is sprayed onto the image acquisition plate to acquire standard droplet deposition images; S2. Using the standard spray gun under the standard process parameters, the cosmetic emulsion to be tested is sprayed onto the image acquisition plate to acquire the droplet deposition image of the test. S3. Extract a global coarse-grained vector from the droplet deposition image to be tested to characterize the shape features of the deposition image. Calculate the global similarity between the vector and the global coarse-grained vector extracted from the standard droplet deposition image. If the similarity between the two is lower than the first threshold, determine that the spray effect of the cosmetic lotion to be tested is unqualified and terminate the test; otherwise, proceed to step S4. S4. Align the standard droplet deposition image with the droplet deposition image to be tested, and divide the droplet deposition image to be tested and the standard droplet deposition image into a one-to-one corresponding detection zone and verification zone according to a preset grid. S5. Based on the attention mechanism, extract the fine-grained feature information of each detection partition and the verification partition, compare the fine-grained feature information between the detection partition and the verification partition one by one and calculate the deviation, and output the fine-grained feature information and the calculated deviation information of each detection partition. S6. Construct the spatial grid feature tensor of the droplet deposition image to be tested by constructing the fine-grained feature information and calculation deviation information of each detection zone, input it into the pre-trained quality classification neural network, and output the quality type label of the cosmetic emulsion to be tested.

2. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 1, characterized in that, The method for extracting the global coarse-grained vector is as follows: The standard droplet deposition image or the droplet deposition image to be tested is sequentially subjected to grayscale conversion, filtering and binarization to obtain the corresponding binary image; Multiple shape features with rotation and translation invariance are extracted from the binary image. The shape features include at least one or more of the following: area of ​​the droplet deposition region, perimeter of the outline, length of the major and minor axes of the fitted ellipse, rectangularity, circularity, eccentricity, and Hu moment. The extracted shape features are concatenated in a preset order to form the global coarse-grained vector.

3. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 2, characterized in that, The similarity is calculated as follows: in, n Let be the dimension of the global coarse-grained vector. The global coarse-grained vector of the cosmetic emulsion to be tested. j Quantity, For the global coarse-grained vector of standard cosmetic emulsions j Quantity, ε This is a preset, extremely small positive number, used to prevent the denominator from being zero. D max This is the normalization constant determined based on the statistics of qualified samples. D This is a relative Euclidean distance. S c For similarity.

4. The method for detecting atomization effects in cosmetics based on image recognition according to claim 1, characterized in that, The fine-grained feature information includes droplet coverage, average droplet size, and droplet size uniformity. The extraction of the fine-grained feature information includes the following process: A1. Input the partition image of the detection partition or verification partition into the attention map extractor. The attention map extractor includes a convolutional neural network, a feature map reshaping module and a correlation attention module. The convolutional neural network extracts the convolutional feature map of the partition image. The feature map reshaping module reshapes the convolutional feature map into a two-dimensional matrix. Then, the correlation attention module performs matrix multiplication on the two-dimensional matrix to generate an attention map for identifying the feature-prominent regions within the detection partition or verification partition. A2. Based on the attention response value, a stride vector generator is used to generate a wide stride vector along the horizontal direction of the image and a high stride vector along the vertical direction of the image based on the attention map. A3. Based on the wide stride vector and the high stride vector, perform a dynamic stride convolution operation on the convolution feature map to obtain a fine-grained feature map that reflects the details of the droplet morphology. A4. Input the fine-grained feature map into the pre-trained multi-task regression network, and output the fine-grained feature information of the detection partition or the verification partition respectively.

5. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 4, characterized in that, The method by which the stride vector generator generates the wide stride vector and the high stride vector is as follows: For each column of the attention map along the horizontal direction, the maximum value of the attention response value of all pixels in that column is taken, and the maximum value is accumulated and integrated along the horizontal direction to obtain the horizontal cumulative distribution function. Then, the horizontal cumulative distribution function is inversely mapped to obtain the wide stride vector. For each row of the attention map along the vertical direction, the maximum value of the attention response value of all pixels in that row is taken, and the maximum value is accumulated and integrated along the vertical direction to obtain the vertical cumulative distribution function. Then, the vertical cumulative distribution function is inversely mapped to obtain the high stride vector. The wide stride vector and the high stride vector are both increasing sequences, and the difference between their adjacent elements is inversely proportional to the response intensity of the attention map at the corresponding position.

6. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 4, characterized in that, The dynamic stride convolution operation includes: Preset the kernel size and dilation rate, and define the regular sampling grid. , k The kernel size; Based on the wide stride vector S w and high stride vector S h Determine each position on the fine-grained feature map on the convolutional feature map. (p,q) Corresponding sampling center point C p,q ,in C p,q The x-coordinate is determined by S w The q The ordinate is determined by a single element, and is composed of... S h The p Each element determines, dx , dy These represent the offsets of the sampling points relative to the center of the convolution kernel in the horizontal and vertical directions, respectively. For each sampling center point C p,q The grid is sampled on the convolutional feature map according to the rules. G Sampling is performed, and when the coordinates of the sampling point are not integers, bilinear interpolation is used to calculate the feature value of the sampling point. The sampled feature values ​​are weighted and summed with the convolution kernel weights to obtain the fine-grained feature map at the corresponding position. (p,q) The value of .

7. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 1, characterized in that, Constructing a spatial grid feature tensor involves the following steps: Each detection zone is arranged into a two-dimensional grid according to its original spatial position. Each grid cell corresponds to a multi-dimensional feature vector. The multi-dimensional feature vector is constructed based on the fine-grained feature information and the calculated deviation information, and includes at least the coverage deviation, average particle size deviation, particle size uniformity deviation, measured coverage value, and measured average particle size value of the zone. Standardize all grid cells of the same type of feature value in the two-dimensional grid so that the values ​​of each type of feature value are in a similar range. The standardized feature vectors are arranged according to the spatial position of the two-dimensional grid to form a spatial grid feature tensor.

8. The method for detecting atomization effects in cosmetics based on image recognition as described in claim 7, characterized in that, The quality classification neural network outputs the confidence level of each preset quality category through the spatial grid feature tensor of the droplet deposition image to be tested, and takes the quality category with the highest confidence level as the output quality type label. The quality categories include at least one or more of the following: qualified, excessive droplet accumulation, missing droplets, poor edge transition, and abnormally large particle size.

9. A cosmetic atomization effect detection system, used to implement the image recognition-based cosmetic atomization effect detection method according to any one of claims 1 to 8, characterized in that, include: The image acquisition board has a hydrophilic surface and is equipped with positioning markers. A standard spray gun and spray gun control unit are used to spray cosmetic emulsion onto an image acquisition board according to a preset trajectory under standard process parameters. Image acquisition module, including at least one industrial camera; The image processing and control module includes: The standard template library unit stores the standard binary image, standard global coarse-grained vector, standard fine-grained feature information, and first threshold obtained based on the standard droplet deposition image; The coarse-grained decision unit is used to perform coarse-grained feature extraction and similarity calculation. The image alignment unit is used to perform marker extraction and spatial transformation; Fine-grained comparison and judgment unit, with embedded attention network, is used to perform feature extraction and bias judgment; The defect classification and diagnosis unit embeds a quality classification neural network, constructs a spatial grid feature tensor, and labels the quality type.

10. An electronic device, performing the image recognition-based cosmetic atomization effect detection method according to any one of claims 1 to 8, characterized in that, It includes a memory and a processor, wherein the memory stores computer instructions, and the processor executes the computer instructions.

Images