Method for detecting egg oiling quality based on machine vision

Abstract

The present application belongs to the technical field of image analysis, and particularly relates to a kind of egg oiling quality detection method based on machine vision, to solve the technical problems of low contrast, high light interference and single information dimension faced by traditional machine vision in prior art, comprising the following steps: S1, obtaining the multispectral polarized image of the egg to be detected;S2, obtaining the contour mask and highlight area mask of the egg;S3, obtaining the effective analysis area;S4, generating a linear polarization degree feature map representing oil film uniformity;S5, generating a spectral angle feature map representing oiling thickness;S6, constructing a region graph with each superpixel as a node and an adjacency relationship as an edge;S7, fusing the initial feature vector of the superpixel node;S8, identifying and outputting the superpixel node determined as an oiling defect;S9, comprehensively evaluating the oiling quality grade of the egg. The automatic, high-precision comprehensive grading of egg oiling quality is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image analysis, and particularly relates to a method for detecting egg oiling quality based on machine vision. BACKGROUND

[0002] As an important agricultural product, the shelf life of eggs is significantly affected by the storage environment and its own quality. In order to prolong the shelf life of eggs and reduce water evaporation and microbial invasion, a preservation technology of coating a layer of edible mineral oil on the surface of eggs is often used in industrial production. This oiling treatment can effectively seal the pores on the surface of the eggshell and maintain the internal quality of the egg. However, the effect of oiling preservation directly depends on the quality of the oil film, that is, the uniformity and thickness of the oil film. If the oil film has defects such as missing coating, too thin, too thick or uneven distribution (such as stripes, patches), it will not achieve the expected preservation effect, and may even affect the appearance of the product. At present, the detection of egg oiling quality mainly relies on manual visual inspection, which is not only low in efficiency and labor-intensive, but also the detection results are easily affected by subjective factors, lack of consistency and objectivity, and it is difficult to meet the high-speed detection demand on modern large-scale production lines. Therefore, it has become an urgent need in the industry to develop an automatic, high-precision and high-efficiency oiling quality detection technology.

[0003] In order to realize automatic detection, some machine vision-based solutions have appeared in the prior art. These solutions usually use a conventional industrial camera to obtain an egg image, and then analyze the oil film defects through image processing algorithms. However, these traditional methods face many challenges in practical applications: since the oil film itself is transparent and extremely thin, its contrast in ordinary visible light images is very low, and the defect features are weak, which makes it difficult to effectively distinguish from the eggshell background. Secondly, the surface of the egg is smooth and curved, and under external light source illumination, it is easy to produce highlight (specular reflection) areas. These highlight areas will seriously interfere with the effective extraction of the underlying oil film information, and even be misjudged as defects, thereby significantly reducing the accuracy and robustness of the detection. In addition, a single imaging mode (such as RGB imaging) cannot simultaneously obtain and distinguish the information representing the physical uniformity of the oil film (such as the coating morphology) and the chemical information representing the oiling amount (such as the coating thickness), resulting in a single detection dimension, which cannot comprehensively and finely evaluate the oiling quality. Therefore, how to effectively suppress the highlight interference and fuse multi-dimensional information to realize the synchronous and accurate evaluation of the uniformity and thickness of the oil film is a key problem to be solved in the current egg oiling quality detection technology. SUMMARY

[0004] The present application provides a method for detecting egg oiling quality based on machine vision to solve the technical problems of low contrast, highlight interference and single information dimension faced by traditional machine vision in the prior art.

[0005] To solve the above problems, the method for detecting the oiling quality of eggs based on machine vision provided by the present application adopts the following technical scheme: the method for detecting the oiling quality of eggs based on machine vision comprises the following steps:

[0006] S1, obtaining a multi-spectral polarization image of the egg to be detected by imaging at multiple spectral bands and at least three different polarization angles;

[0007] S2, synchronously segmenting the contour mask and the highlight area mask of the egg according to the multi-spectral polarization image by using a multi-task convolutional neural network;

[0008] S3, removing the pixels corresponding to the highlight area mask within the area defined by the contour mask to obtain an effective analysis area;

[0009] S4, calculating the linear polarization degree of each pixel point in the effective analysis area based on the image data at different polarization angles, and generating a linear polarization degree feature map representing the uniformity of the oil film;

[0010] S5, calculating the spectral angle based on the multi-spectral data of each pixel point in the effective analysis area and a pre-constructed standard oiling spectral library, and generating a spectral angle feature map representing the oiling thickness;

[0011] S6, performing superpixel segmentation on the effective analysis area to construct a region graph with each superpixel as a node and an adjacency relationship as an edge;

[0012] S7, for each superpixel node, extracting the statistical moments of all pixels within the superpixel node on the linear polarization degree feature map and the mean and variance on the spectral angle feature map, and fusing them into an initial feature vector of the superpixel node;

[0013] S8, inputting the region graph with the initial feature vector into a pre-trained graph neural network model to identify and output the superpixel nodes determined as oiling defects;

[0014] S9, comprehensively evaluating the oiling quality grade of the egg according to the position, number and total area of the superpixel nodes determined as oiling defects.

[0015] Further, when obtaining the multi-spectral polarization image, the following steps are included:

[0016] A multi-spectral imaging system is used to image at multiple bands covering the visible light to near-infrared spectral range;

[0017] At each band, at least three images at different polarization angles are collected by rotating a linear polarizer in front of the camera.

[0018] Further, when collecting images at three different polarization angles, the three polarization angles are 0°, 45° and 90°, respectively.

[0019] Further, when calculating the linear polarization degree of each pixel point in the effective analysis region based on the image data under different polarization angles, the following steps are included: using the image intensity collected under three polarization angles of 0°, 45° and 90° , , , the Stokes component of each pixel point is calculated according to the following formula , , , and then the linear polarization degree of each pixel point is calculated:

[0020] ;

[0021] ;

[0022] ;

[0023] .

[0024] Further, in S5, when making the standard sample, a standard oil film with a known thickness is coated on the eggshell, and the reflectivity thereof under different wavebands is measured to form a standard oil coating spectrum library, wherein the standard oil coating spectrum library includes standard spectrum vectors corresponding to different thicknesses.

[0025] Further, for each effective pixel in the effective analysis region, the gray value thereof under different wavebands is extracted to form a spectrum vector of the pixel;

[0026] The spectrum angle of the pixel is obtained by calculating the inverse cosine of the included angle between the spectrum vector of the pixel and the standard spectrum vector.

[0027] Further, when performing superpixel segmentation on the effective analysis region and constructing a region graph, the following steps are included:

[0028] The effective analysis region is segmented into a plurality of superpixel regions with uniform sizes by using a simple linear iterative clustering algorithm or a variant thereof;

[0029] All superpixels are traversed, and if two superpixels have a shared boundary, an edge is added between the nodes representing them, thereby constructing a region graph.

[0030] Further, the shared boundary includes a contact or overlapping part of the boundaries of the two superpixels.

[0031] Further, when extracting an initial feature vector for each superpixel node, the following steps are included:

[0032] The mean, variance, skewness and kurtosis of all pixel points in the superpixel node on the linear polarization degree feature map are calculated as the linear polarization degree statistical moment features;

[0033] The mean and variance of all pixel points in the superpixel node on the spectral angle feature map are calculated as spectral angle statistical features;

[0034] The linear polarization degree statistical moment features and the spectral angle statistical features are concatenated to form an initial feature vector of the superpixel node.

[0035] Further, according to the position, number and total area of the superpixel nodes of the oiling defects, the oiling quality grade of the eggs is comprehensively evaluated, including the following steps:

[0036] A defect number threshold and a defect area percentage threshold are preset;

[0037] According to whether the total number of the superpixel nodes of the oiling defects exceeds the defect number threshold and whether the percentage of the total defect area to the total area of the egg exceeds the defect area percentage threshold, the oiling quality is evaluated as a premium, qualified or unqualified grade.

[0038] Beneficial effects are: the present application obtains the multi-spectral polarization image of the egg, and simultaneously segments the highlight area by using the multi-task convolutional neural network, overcomes the problem of difficult defect feature extraction caused by highlight interference and oil film transparency in traditional visual detection, and improves the robustness of detection. At the same time, the present application combines two physical mechanisms, i.e. using the linear polarization degree calculated by the polarization information to represent the physical uniformity of the oil film, and using the spectral angle calculated by the multi-spectral information to represent the thickness of the oiling, so as to realize the simultaneous quantitative evaluation of the two key dimensions of the oiling quality, so that the detection result is more comprehensive and accurate. By introducing the superpixel segmentation and graph neural network technology, the local features and spatial context information between regions can be fused for comprehensive judgment, the recognition accuracy of complex morphological defects such as stripes and patches is improved, and the automatic and high-precision comprehensive grading of the egg oiling quality is realized. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The flowchart of the egg oiling quality detection method based on machine vision. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be known by those skilled in the art that the embodiments described below are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] Embodiments of the egg oiling quality detection method based on machine vision provided by the present application:

[0042] As Figure 1As shown, the machine vision-based egg oiling quality detection method comprises the following steps:

[0043] S1, obtain the multi-spectral polarized images of the eggs to be detected by imaging under multiple spectral bands and at least three different polarization angles.

[0044] In an optional embodiment, when obtaining the multi-spectral polarized images, the following steps are included:

[0045] A multi-spectral imaging system is used to image at multiple bands covering the visible to near-infrared spectral range;

[0046] At each band, at least three images at different polarization angles are collected by rotating a linear polarizer in front of the camera.

[0047] Specifically, the multi-spectral imaging system can obtain image information of the eggs at multiple pre-set narrowband wavelengths. For example, three filters with center wavelengths of 530 nm, 660 nm and 850 nm can be selected. These bands correspond to the optical response characteristics of different substances on the surface and inside the egg, and can capture subtle differences before and after oiling. The visible light band helps to observe the surface color and texture, while the near-infrared band is more sensitive to the absorption and scattering characteristics of the oil film, thereby enhancing the detection contrast.

[0048] At each selected band, for example at the 660 nm band, the linear polarizer is rotated to three or more pre-set angles in turn for imaging. A typical angle setting is 0°, 45° and 90°. When the linear polarizer is at 0°, the first image is collected, when it is rotated to 45°, the second image is collected, and when it is rotated to 90°, the third image is collected. In this way, not only the intensity information of the egg under different spectra is obtained, but also the polarization state information after the interaction of light waves with the egg surface, providing the necessary data basis for subsequent accurate calculation of the degree of polarization.

[0049] S2, use a multi-task convolutional neural network to simultaneously segment the multi-spectral polarized images to obtain the contour mask and highlight area mask of the eggs.

[0050] In an optional embodiment, the nine images obtained are stacked into a 9-channel input tensor. A multi-task learning model based on the U-Net architecture is used. The multi-task learning model shares an encoder backbone network, but has two independent decoder branches. The first decoder branch outputs the binary contour mask of the egg, and the second decoder branch outputs the binary mask of the highlight area. Through end-to-end training on a dataset with manually annotated contours and highlight areas, the multi-task learning model can learn the mapping relationship from the multi-dimensional input to the two segmentation tasks.

[0051] S3, in the region defined by the contour mask, the pixels corresponding to the highlight region mask are removed to obtain the effective analysis region.

[0052] S4, based on the image data under different polarization angles, the linear polarization degree of each pixel point in the effective analysis region is calculated, and a linear polarization degree feature map representing the uniformity of the oil film is generated.

[0053] In an optional embodiment, when three different polarization angles are selected as 0°, 45° and 90°, the linear polarization degree (DoLP) of each pixel point in the effective analysis region is calculated based on the image data under different polarization angles, including the following steps: using the image intensity collected under the polarization angles of 0°, 45° and 90° , , , the Stokes component of each pixel point is calculated according to the following formula , , , and then the linear polarization degree of each pixel point is calculated:

[0054] ;

[0055] ;

[0056] ;

[0057] .

[0058] Specifically, the light intensity values measured at three polarization angles for each pixel point are converted into physical quantities describing its polarization state. The Stokes component represents the total light intensity, reflects the difference between the horizontal and vertical polarization components, and reflects the difference between the ±45° diagonal polarization components. For example, for a pixel point on the surface of an egg, the image intensity values collected under a certain waveband are =180, =150, =100. Then according to the formula, is 280, is 80, .

[0059] After obtaining the Stokes component, the linear polarization degree (DoLP) is further calculated. The linear polarization degree is a normalized value between 0 and 1, which quantifies the proportion of linearly polarized components in the light wave. Continuing the example, the linear polarization degree of the pixel point is A higher DoLP value of a region indicates that the light reflected by the surface of the region has strong polarization characteristics, which is usually related to uniform oil film coverage; on the contrary, a lower DoLP value means that the surface is rough or not covered by an oil film, thereby realizing quantitative differentiation of the oiling defect region.

[0060] S5, based on the multispectral data of each pixel point in the effective analysis region and the pre-constructed standard oiling spectrum library, a spectral angle is calculated to generate a spectral angle feature map representing the oiling thickness.

[0061] Specifically, standard samples are prepared: different known thickness standard oil films are coated on eggshells, and their reflectivities at 750 nm, 850 nm and 940 nm wavebands are measured to form a standard oiling spectrum library (containing standard spectrum vectors corresponding to different thicknesses). For each effective pixel in the effective analysis region, the gray values at the three wavebands are extracted to form a spectral vector of the pixel. By calculating the inverse cosine of the included angle between the spectral vector of the pixel and the standard spectrum vector, the spectral angle of the pixel is obtained. Perform the above calculation for each pixel in the effective analysis region to generate a single-channel spectral angle (SAM) feature map, in which the size of the pixel value reflects the difference between the oiling thickness and the standard thickness at that point. The smaller the pixel value in the map, the closer the oiling thickness at that point to the corresponding standard thickness; the larger the pixel value in the map, the more significant the difference between the oiling thickness at that point and the corresponding standard thickness.

[0062] S6, superpixel segmentation is performed on the effective analysis region to construct a region graph with each superpixel as a node and an adjacency relationship as an edge.

[0063] In an optional embodiment, when performing superpixel segmentation on the effective analysis region to construct a region graph, the following steps are included:

[0064] The effective analysis region is segmented into multiple uniform-size superpixel regions using the simple linear iterative clustering (SLIC) algorithm or its variants;

[0065] All superpixels are traversed, and if two superpixels have a shared boundary, an edge is added between the nodes representing them, thereby constructing a region graph.

[0066] Specifically, the method only performs superpixel segmentation on the effective analysis region to avoid background interference and improve computational efficiency. Taking an effective analysis region with a total of about 200,000 pixels as an example, the target number of superpixels for the SLIC algorithm can be set to 500. The SLIC algorithm will consider the color similarity and spatial distance between pixels, iteratively clustering the 200,000 pixel points into 500 compact regions with irregular shapes but approximately 400 pixels in size, i.e., superpixels.

[0067] Superpixels can well fit the local texture and color boundary of the egg surface, and serve as the basic unit for subsequent analysis.

[0068] After segmentation, 500 superpixels are abstracted as 500 nodes in the region graph. Each pair of superpixels, for example, superpixel No. 123 and superpixel No. 124, is checked to see if any part of their boundary is in contact or coincides. If so, a connection edge is established between the nodes representing them respectively. After traversing all pairs of superpixels, a non-directed graph is finally formed, which not only contains the internal information of each superpixel, but also encodes their adjacency relationship, laying a foundation for context analysis using advanced models such as graph neural networks.

[0069] S7, for each superpixel node, extract the statistical moments on the DoLP feature map and the mean and variance on the SAM feature map of all pixels inside it, and fuse them into the initial feature vector of the superpixel node.

[0070] In an optional embodiment, when extracting the initial feature vector for each superpixel node, the following steps are included:

[0071] Calculate the mean, variance, skewness and kurtosis of all pixel points inside the superpixel node on the DoLP feature map as DoLP statistical moment features;

[0072] Calculate the mean and variance of all pixel points inside the superpixel node on the SAM feature map as SAM statistical features;

[0073] Concatenate the DoLP statistical moment features and the SAM statistical features to form the initial feature vector of the superpixel node.

[0074] Specifically, this step gives each superpixel node in the region graph a quantitative descriptive information. For a superpixel node representing a uniform oil-coated area, the DoLP values of its internal pixel points may be highly concentrated around 0.8, so the calculated DoLP mean will be close to 0.8, and the variance will be very small, for example, 0.01. The skewness and kurtosis, two high-order statistical quantities, can more finely describe the shape of the data distribution, reflecting the smoothness of the oil film. At the same time, the SAM mean of the superpixel node may be 0.1, and the variance may be 0.005, indicating that its spectral characteristics are very close to the standard oil film and the internal material is uniform.

[0075] The statistics are connected to form the initial feature of the superpixel node. Taking the above data as an example, the initial feature vector of the superpixel node can be represented as a six-dimensional vector, and the specific numerical value is (0.8, 0.01, the corresponding skewness and kurtosis values, 0.1, 0.005). In contrast, a superpixel containing a missed coating area may have a DoLP mean value of 0.3, a variance of 0.2, and a SAM mean value that will also increase accordingly. Its initial feature vector will be significantly different from the aforementioned uniform area node. By generating a unique initial feature vector for each superpixel node, the local physical characteristics of the image are converted into digital signals that can be processed by machine learning models.

[0076] S8, input the region map with the initial feature vector into a pre-trained graph neural network model, identify and output the superpixel nodes determined as oil coating defects.

[0077] In an optional embodiment, a graph convolutional network (GCN) model pre-trained on a large number of labeled defect egg image data is used. The region map previously constructed, each superpixel node with a six-dimensional initial feature vector, is used as the input of the GCN model. Through multi-layer graph convolution operations inside the model, the features of each superpixel node can aggregate the feature information of its neighbor nodes, thereby learning the spatial context patterns of the defect area, such as striped or patchy defects. The last layer of the model outputs a binary classification result for each node, i.e., whether the superpixel node belongs to the oil coating defect area.

[0078] S9, according to the position, number and total area of the superpixel nodes of the oil coating defects, comprehensively evaluate the oil coating quality grade of the egg.

[0079] In an optional embodiment, according to the position, number and total area of the superpixel nodes of the oil coating defects, comprehensively evaluating the oil coating quality grade of the egg includes the following steps:

[0080] Pre-set a defect quantity threshold and a defect area percentage threshold;

[0081] According to whether the total number of superpixel nodes of oil coating defects exceeds the defect quantity threshold, and whether the percentage of the total defect area to the total area of the egg exceeds the defect area percentage threshold, the oil coating quality is evaluated as a premium, qualified or unqualified grade.

[0082] Specifically, the evaluation standard realizes automatic and standardized quality grading by setting clear digital thresholds. For example, two sets of thresholds can be pre-set: the first set is the premium threshold, i.e., the defect quantity does not exceed 3 and the total defect area does not exceed 1% of the total area of the egg; the second set is the qualified baseline threshold, i.e., the defect quantity does not exceed 10 and the total defect area does not exceed 5% of the total area of the egg. The thresholds can be flexibly adjusted according to the production process requirements.

[0083] After the detection is completed, the total number of superpixel nodes identified as oiling defects and their total pixel area are counted. Suppose a total area of an egg is 250,000 pixels, and 2 defect superpixel nodes are detected with a total area of 2,000 pixels. At this time, the defect number 2 is less than the good quality threshold 3, and the defect area percentage 0.8% is also less than the good quality threshold 1%, so the egg is rated as good quality.

[0084] If another egg detects 8 defect superpixel nodes with a total area of 10,000 pixels, neither the defect number nor the area percentage exceeds the good product bottom threshold, but exceeds the good quality threshold, so the egg is rated as qualified.

[0085] If either index of the defect number or the area exceeds the good product bottom threshold, for example, the total defect area reaches 15,000 pixels, which is 6% of the total area, then the egg will be directly rated as unqualified.

[0086] In addition, in the description of the present specification, the meaning of "a plurality of" is at least two, such as two, three or more, etc., unless otherwise expressly specifically limited.

Claims

1. A machine vision-based method for detecting the quality of oil coating on eggs, characterized in that, Includes the following steps: S1, by imaging in multiple spectral bands and at least three different polarization angles, a multispectral polarization image of the egg to be detected is obtained; S2, using a multi-task convolutional neural network, synchronously segment the multispectral polarization image to obtain the outline mask and highlight region mask of the egg; S3, remove the pixels corresponding to the highlight area mask within the area defined by the contour mask to obtain the effective analysis area; S4, calculate the degree of linear polarization of each pixel in the effective analysis area based on image data under different polarization angles, and generate a linear polarization feature map characterizing the uniformity of the oil film; S5 calculates the spectral angle based on the multispectral data of each pixel in the effective analysis area and the pre-built standard oil coating spectral library, and generates a spectral angle feature map characterizing the oil coating thickness; S6, perform superpixel segmentation on the effective analysis region and construct a region graph with each superpixel as a node and the adjacency relationship as an edge; S7. For each superpixel node, extract the statistical moments of all pixels in its internal polarization degree feature map and the mean and variance in the spectral angle feature map, and fuse them into the initial feature vector of the superpixel node. S8: Input the region map with the initial feature vector into the pre-trained graph neural network model to identify and output the superpixel nodes that are determined to be oiling defects; S9 comprehensively evaluates the oiling quality grade of the egg based on the position, number, and total area of ​​the superpixel nodes of the oiling defects.

2. The method for detecting the quality of egg oil coating based on machine vision according to claim 1, characterized in that, Acquiring a multispectral polarization image includes the following steps: A multispectral imaging system is used to perform imaging by utilizing multiple bands covering the visible to near-infrared spectral range; At each wavelength, images with at least three different polarization angles are acquired by rotating the linear polarizer in front of the camera.

3. The method for detecting the quality of egg oil coating based on machine vision according to claim 2, characterized in that, When acquiring images at three different polarization angles, the three polarization angles are 0°, 45° and 90°.

4. The egg oiling quality inspection method based on machine vision according to claim 3, characterized in that, When calculating the linear polarization degree of each pixel within the effective analysis area based on image data at different polarization angles, the following steps are included: using image intensity acquired at three polarization angles of 0°, 45°, and 90°. , , For each pixel, calculate the Stokes component using the following formula. , , Then, the linear polarization degree of each pixel is calculated: ； ； ； 。 5. The method for detecting the quality of egg oil coating based on machine vision according to claim 1, characterized in that, In S5, when making standard samples, standard oil films of different known thicknesses are coated on eggshells, and their reflectance at different wavelengths is measured to form a standard oil-coated spectral library. The standard oil-coated spectral library includes standard spectral vectors corresponding to different thicknesses.

6. The egg oiling quality detection method based on machine vision according to claim 5, characterized in that, For each valid pixel in the effective analysis area, extract its grayscale value in different bands to form the pixel's spectral vector; The spectral angle of a pixel is obtained by calculating the inverse cosine function of the angle between the pixel's spectral vector and the standard spectral vector.

7. The method for detecting the quality of egg oil coating based on machine vision according to claim 1, characterized in that, When constructing a region map by performing superpixel segmentation on the effective analysis region, the following steps are included: A simple linear iterative clustering algorithm or its variants are used to segment the effective analysis region into multiple superpixel regions of uniform size; Iterate through all superpixels. If two superpixels share a boundary, add an edge between the nodes representing them to construct a region graph.

8. The method for detecting the quality of egg oil coating based on machine vision according to claim 7, characterized in that, The shared boundary includes the boundary of two superpixels having contact or overlapping portions.

9. The machine vision-based method for detecting the quality of oiled eggs according to claim 1, characterized in that, When extracting the initial feature vector for each superpixel node, the following steps are included: Calculate the mean, variance, skewness, and kurtosis of all pixels within a superpixel node on the linear polarization feature map, and use them as the statistical moment features of linear polarization. Calculate the mean and variance of all pixels within a superpixel node on the spectral angle feature map, and use them as spectral angle statistical features; The linear polarization degree statistical moment feature is concatenated with the spectral angle statistical feature to form the initial feature vector of the superpixel node.

10. The machine vision-based method for detecting the quality of oiled eggs according to any one of claims 1-9, characterized in that, The quality grade of egg coating is comprehensively evaluated based on the location, number, and total area of ​​superpixel nodes with coating defects, including the following steps: Preset thresholds for the number of defects and the percentage of defect area; The quality of the coating is rated as excellent, qualified, or unqualified based on whether the total number of superpixel nodes of the coating defects exceeds the defect number threshold, and whether the percentage of the total defect area to the total egg area exceeds the defect area percentage threshold.

Images