Egg freshness detection method based on machine vision color classification and hyperspectrum

Abstract

The invention relates to the technical field of agricultural product nondestructive testing, and discloses an egg freshness detection method based on machine vision color classification and hyperspectrum, which comprises the following steps: collecting a visual image and a hyperspectral image of an egg, and carrying out black and white correction processing on the hyperspectral image; processing the visual image to obtain a color classification result of the egg and a surface feature mask pattern for marking flaws; selecting a flawless region of interest on the corrected hyperspectral data by utilizing the guidance of the mask pattern, and extracting an average spectral curve; and performing grouping preprocessing on the spectrum curve according to a color classification result, extracting a characteristic wavelength combination through a cascade characteristic screening algorithm, and constructing a K-nearest neighbor classification model optimized by a particle swarm optimization algorithm to judge freshness. Through visual guidance and color grouping, interference of eggshell surface factors is effectively avoided, and effectiveness of data extraction and accuracy and reliability of a detection model are improved.

Description

Technical Field

[0001] This invention relates to the field of non-destructive testing technology for agricultural products, specifically a method for detecting egg freshness based on machine vision color classification and hyperspectral imaging. Background Technology

[0002] As an important source of nutrition, the freshness of eggs directly affects food safety and consumer quality. Therefore, developing rapid and accurate non-destructive testing technologies for egg freshness is of significant practical importance. Currently, mainstream non-destructive testing technologies mainly include machine vision and hyperspectral imaging.

[0003] Machine vision technology excels at analyzing the external features of products. Existing research has used machine vision to detect cracks or stains on the surface of eggs. However, these methods have failed to assess the internal freshness level of eggs. Although some studies have attempted to correlate visual features with freshness, they lack classification of egg color, appearance, etc., or are susceptible to environmental factors such as mechanical vibration. Furthermore, when faced with internal biochemical changes caused by storage time, it is difficult to make an accurate judgment based solely on external vision.

[0004] Hyperspectral imaging technology, by acquiring the spectral fingerprint information of substances inside matter, has shown potential in detecting the internal quality of eggs. Researchers can classify freshness by collecting spectral images of eggs and combining chemometric methods with machine learning models. However, current hyperspectral detection technologies still face technical bottlenecks in practical applications.

[0005] First, the accuracy of hyperspectral detection is highly dependent on the purity of the acquired spectral data. Existing methods, when selecting regions of interest (ROIs), employ simple strategies such as fixing the size of the bounding box at the center of the egg, failing to proactively avoid imperfections such as stains, spots, or micro-cracks on the eggshell surface. Once these imperfect areas are included in the spectral acquisition range, their abnormal spectral responses contaminate the average spectral data of the entire region, causing the extracted spectral information to fail to accurately reflect the biochemical state inside the egg, thus affecting the accuracy of the final model's judgment.

[0006] Secondly, the inherent differences in eggshell color (such as between white and brown eggs) are a key interference factor that is often overlooked in current spectroscopic detection techniques. Eggshells of different colors have different baseline reflectances, which introduces a systematic shift in the spectral data. Existing technologies typically use a uniform process to process all eggs, failing to specifically correct for these color-related differences. This leads to a decrease in the stability and accuracy of the model when detecting a mix of eggs of different colors.

[0007] Furthermore, existing methods still need improvement in balancing model complexity and detection performance during the construction of detection models. For example, in the feature wavelength screening stage, a single screening algorithm struggles to achieve the optimal balance between effective dimensionality reduction and retention of key information; in the classification model construction stage, the setting of model hyperparameters relies on experience or employs local optimization, failing to fully explore the model's optimal performance potential, which results in the final model failing to reach its theoretically highest accuracy. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method for detecting egg freshness based on machine vision color classification and hyperspectral imaging. The core idea of ​​this method is to leverage the precise analytical capabilities of machine vision for egg surface features to guide hyperspectral imaging for information extraction within the effective area, thereby establishing a deeply coupled detection model. This eliminates interference from surface factors and improves the accuracy and reliability of freshness detection. The specific technical solution of this invention is as follows: First, visual and hyperspectral images of the same egg sample were acquired simultaneously. To eliminate the influence of dark current in the imaging system and uneven light source, black-and-white correction operations were performed on the acquired raw hyperspectral image using pre-acquired dark current images and standard white board images, converting it into standard reflectance data.

[0009] Subsequently, the acquired visual images are processed. This invention employs an improved deep learning object detection model (an improved YOLO model), which can not only classify the color of eggshells (e.g., dark or light) but also simultaneously identify and locate defective areas on the eggshell surface. After processing, two key pieces of information are output: the color classification result of the eggshell and a surface feature mask map indicating the location of defects on the eggshell surface.

[0010] To improve the performance of the above-mentioned target detection model, this invention introduces several technical improvements: First, a mobile multi-query attention module is embedded in the backbone network of the model. This module reweights the channels and spatial dimensions of the input feature map through a multi-head attention mechanism, making the network pay more attention to key features such as the color and defects of the egg.

[0011] Secondly, the neck network of the model is simplified, retaining some key bidirectional feature map fusion paths, and Gs convolution is used to replace standard convolution for feature integration, which reduces the computational complexity of the model while ensuring the feature fusion effect.

[0012] Third, during the model training phase, a weighted intersection-over-union (IoU) loss function is employed. This function assigns differentiated loss weights to anchor boxes of varying quality using a non-monotonic dynamic weighting factor, thereby optimizing the model's bounding box regression accuracy.

[0013] Next, using the surface feature mask image generated in the previous step as spatial guidance information, the region of interest (ROI) is selected on the standard reflectance data. This invention employs an adaptive spectral ROI dynamic programming algorithm. Based on the defect location information provided by the mask image, this algorithm automatically finds and determines the largest continuous defect-free region within the main body of the egg using dynamic programming, serving as the optimal spectral information extraction region. Subsequently, the average spectral reflectance of all pixels within this region is calculated, generating a spectral curve that represents the characteristics of the pure eggshell surface.

[0014] Finally, the generated spectral curves are processed to construct and output an egg freshness detection model.

[0015] In the spectral data preprocessing stage, based on the color classification results, the samples were divided into two sets: dark-colored eggs and light-colored eggs. For these two sets, a series of preprocessing operations, such as normalization, standardization, and multivariate scattering correction, were performed independently to eliminate the systematic spectral shift caused by the difference in eggshell base color.

[0016] In the characteristic wavelength screening stage, a cascaded screening strategy is adopted. The first stage uses Pearson correlation coefficient analysis to calculate and rank the correlation between all wavelengths and preset freshness labels, completing the initial screening and obtaining a preliminary set of characteristic wavelengths. The second stage introduces an extreme gradient boosting classification algorithm to further refine this preliminary set. The importance of each wavelength is evaluated based on the algorithm's internal gain index, ultimately determining a lower-dimensional, more representative combination of characteristic wavelengths.

[0017] In the classification model construction phase, the K-Nearest Neighbors (KNN) classification model was adopted. To obtain the best model performance, the Particle Swarm Optimization (PSO) algorithm was used to globally optimize the key hyperparameters of the KNN model (such as the number of neighbors K), thereby determining its optimal parameter combination. Finally, using the selected feature wavelength combination and the optimal model parameters, the egg freshness detection model was trained and used to determine the freshness level of new samples.

[0018] This invention provides a method for detecting egg freshness based on machine vision color classification and hyperspectral imaging. It has the following beneficial effects: 1. This invention solves the problem of systematic spectral shift caused by the different shades of eggshell color by using machine vision to classify the eggshell color in step S2 and performing differential preprocessing on the spectral data according to the classification results in step S4. This solves the core interference factor from the source of the data, making the final freshness judgment result more reliable and unaffected by the individual appearance differences of the eggs.

[0019] 2. In step S2, the present invention generates a surface feature mask map indicating defects, and in step S3, the mask map is used to guide an adaptive spectral ROI dynamic programming algorithm to automatically select the largest continuous defect-free area on the standard reflectance data as the spectral extraction object. This method ensures that the extracted spectral curve can truly reflect the internal quality of the egg and eliminates the interference of local abnormal points such as stains and cracks, thereby improving the data quality of the input model.

[0020] 3. In step S4 of this invention, a cascaded feature wavelength screening method (combining Pearson correlation coefficient method and extreme gradient boosting algorithm) is adopted, which can reduce data dimensionality and reduce model computation while ensuring accuracy. At the same time, the particle swarm optimization algorithm is combined to globally optimize the hyperparameters of the K-nearest neighbor classification model, ensuring that the classification model can reach its best performance state. Thus, the final detection model is not only accurate in prediction, but also more lightweight and has the potential for rapid detection. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the detection method provided by the present invention; Figure 2 A schematic diagram of the physical environment system architecture of the detection method provided by the present invention; Figure 3 This is a schematic diagram of the spectral data preprocessing process based on color classification in this invention; Figure 4 This is a schematic diagram of the hyperspectral image black-and-white correction process of the present invention; Figure 5 This is a schematic diagram illustrating the process of visual feature-guided spectral information extraction according to the present invention. Figure 6 This is a schematic flowchart of the cascaded characteristic wavelength screening method of the present invention; Figure 7 This is a schematic diagram of the overall architecture of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See attached document Figure 2This invention provides a method for detecting egg freshness based on machine vision color classification and hyperspectral imaging. The method is executed in a physical environment, which includes a machine vision image acquisition unit, a hyperspectral imaging unit, a light source unit, a sample carrying unit, and a control and processing unit.

[0024] The machine vision image acquisition unit includes a machine vision camera for acquiring visual images of an egg to be inspected. The visual images capture the egg's shell color, texture information, and surface features. The machine vision image acquisition unit may also include a machine vision light source for providing uniform illumination for image acquisition.

[0025] The hyperspectral imaging unit includes a hyperspectral imager, such as the SOC710VP hyperspectral imager, which can acquire reflectance spectral images of eggs in a wavelength range of, for example, 373 nm to 1053 nm, to obtain internal physicochemical information of the eggs. The hyperspectral imaging unit may also include a hyperspectral light source to provide stable broadband illumination for spectral acquisition.

[0026] A sample carrier unit is used to carry the egg sample to be tested. The sample carrier unit can realize the positioning or transportation of the egg sample, ensuring that the egg sample can be in a preset stable position during image and spectral acquisition. For example, the sample carrier unit can be a conveyor belt system or a stage.

[0027] The control and processing unit is a computer system or embedded processing platform. It is connected to the machine vision image acquisition unit, hyperspectral imaging unit, and light source unit, and is used to control their operating states. The control and processing unit receives and stores the visual image data and hyperspectral image data acquired by the machine vision image acquisition unit and the hyperspectral imaging unit. Furthermore, the color classification processing function, data fusion analysis function, and result output function of this invention are all implemented by the control and processing unit by executing corresponding algorithms and programs.

[0028] See attached document Figure 1 and attached Figure 7 The core of the detection method provided by this invention lies in the deep coupling of machine vision's surface feature analysis capabilities with hyperspectral imaging's internal quality detection capabilities. Through a progressive processing flow, it solves the problem of interference from eggshell color and surface defects on the accuracy of spectral detection. The method specifically includes the following steps: First, step S1 is performed: multimodal data acquisition and correction. In step S1, for the same egg sample, two different modalities of image data are acquired simultaneously: one is a visual image used to characterize shell color and physical defects, and the other is a hyperspectral image used to reflect internal chemical composition information. To ensure the accuracy of subsequent analysis, after acquiring the hyperspectral image data, black and white correction processing is performed immediately. This correction operation is based on pre-acquired dark current images and standard white board images, and converts the original image brightness values ​​(DN values) into standard reflectance data through formula calculation, thereby eliminating the influence of systematic factors such as dark current noise from the acquisition equipment itself and uneven distribution of external light sources on the data.

[0029] Next, step S2 is performed: color classification and surface feature analysis based on machine vision. Step S2 processes the visual image obtained in step S1 to obtain two key outputs: first, the color classification result of the eggshell, such as determining whether it is a white-shelled egg or a pink-shelled egg; second, generating a surface feature mask map that marks the defective areas (such as spots, stains, and cracks) on the eggshell surface. This step ensures the accurate quantification of the egg's appearance attributes.

[0030] Then, step S3 is performed: spectral information extraction guided by visual features. Step S3 is a key step in achieving information collaboration in this invention. It receives the surface feature mask generated in step S2 and uses this mask as spatial guidance information. On the corrected hyperspectral image obtained in step S1, an adaptive algorithm is used to dynamically plan and select a region of interest (ROI) that completely avoids all defective areas. Subsequently, only the average spectral reflectance within this pure ROI is calculated to generate a spectral curve that truly reflects the internal quality of the egg. This step ensures the purity and representativeness of the spectral data.

[0031] Finally, step S4 is executed: egg freshness detection model construction and determination. Step S4 receives the pure spectral data extracted in step S3 and can combine it with the color classification results obtained in step S2 for grouping processing. First, the spectral data is preprocessed by normalization, multivariate scattering correction, etc.; second, the feature wavelength combination most relevant to freshness is extracted through a feature selection algorithm to reduce the data dimensionality; finally, a classification model (such as the K-nearest neighbor model) is trained using the feature wavelength combination, and its optimal parameters are determined through an optimization algorithm. Ultimately, a detection model that can accurately determine the freshness level of eggs based on the input spectral data is established, and the determination result is output.

[0032] This invention provides a method for detecting egg freshness based on machine vision color classification and hyperspectral imaging, wherein step S1 includes the construction and acquisition of a multimodal dataset.

[0033] In the construction and collection of the multimodal dataset, the first step was to construct a dataset of egg samples containing different shell colors. The egg samples included white-shelled eggs, pink-shelled eggs, brown-shelled eggs, and green-shelled eggs. For each egg sample, data was collected using the following methods: A visual image dataset of egg samples was obtained. The visual image dataset uses high-resolution machine vision cameras to capture images of the appearance of eggs. The visual images are designed to record visual features such as eggshell color, texture, and surface defects, which will be used for subsequent color classification model training and surface feature analysis.

[0034] A hyperspectral image dataset of egg samples was acquired simultaneously. The hyperspectral image dataset was generated using a hyperspectral imaging system equipped with a SOC710VP hyperspectral imager, which acquired full-band hyperspectral images of each egg sample in the 373nm to 1053nm wavelength range. The hyperspectral image dataset aims to obtain spectral information related to the internal quality of the eggs.

[0035] Furthermore, for each egg sample, multiple physicochemical parameters related to egg freshness were obtained as ground truth labels using standard destructive testing methods. These parameters included Hardy units (HU), yolk color (YC), egg weight (EW), egg major diameter (MD), egg minor diameter (md), yolk weight (YW), eggshell weight (SW), and average eggshell thickness (STA). These parameters were used for subsequent calibration and validation of the freshness detection model. A one-to-one correspondence was established between the collected visual image dataset, hyperspectral image dataset, and physicochemical parameter data.

[0036] See attached document Figure 4 In the egg freshness detection method based on machine vision color classification and hyperspectral according to an embodiment of the present invention, step S1 further includes a black-and-white correction process for the hyperspectral image.

[0037] Black-and-white correction aims to eliminate systematic errors generated during data acquisition in hyperspectral imaging systems. Because hyperspectral imagers are inevitably affected by factors such as dark current, thermal noise, and uneven light source intensity distribution during operation, the raw image data (DN values) cannot directly and accurately reflect the true spectral characteristics of the sample. Therefore, converting the raw image data into standard reflectance data through black-and-white correction is a necessary prerequisite for ensuring data consistency between different samples and the accuracy of subsequent model detection.

[0038] The specific execution process of black and white correction is as follows: First, dark current images were acquired. In a completely dark environment with external light completely blocked and the system light source turned off, images were acquired using a hyperspectral imager, and these images were denoted as... Dark current images characterize the background noise level of the imaging sensor under dark conditions.

[0039] Secondly, acquire a standard white board image. Place a standard white board with known high reflectivity within the field of view of the hyperspectral imager. Under preset illumination conditions, use the hyperspectral imager to acquire an image of the standard white board, which is then denoted as [image of the white board]. Standard whiteboards are typically made of materials such as polytetrafluoroethylene or barium sulfate and are used to calibrate the system's response under total internal reflection and the spectral distribution of the light source.

[0040] Next, acquire the original image of the sample. Place the egg sample to be tested within the field of view of the hyperspectral imager, maintaining the same lighting conditions and system parameters as when acquiring the standard white board image. Use the hyperspectral imager to acquire an image of the egg sample, obtaining the original hyperspectral image, which is denoted as [image of the original hyperspectral image]. .

[0041] Finally, based on the acquired dark current image, standard white board image, and original hyperspectral image, the control and processing unit calculates the corrected standard reflectance image according to the following formula. : ; in, This image represents the standard reflectance of an egg sample after black-and-white correction. This represents the original hyperspectral image of an egg sample acquired by a hyperspectral imager. This represents the dark current image acquired by the hyperspectral imager; This represents a standard whiteboard image acquired by a hyperspectral imager.

[0042] Through the above calculations, the system dark current noise was subtracted from the original image data, and the spatial distribution differences of the light source intensity were normalized, thus obtaining spectral reflectance data that is only related to the material properties of the egg sample itself.

[0043] Step S2 includes color and spatial feature extraction.

[0044] Color and spatial feature extraction is achieved by embedding a Mobile Multi-Query Attention (Mobile MQA) module within the A2C2f module of the backbone layer of the improved YOLO v12 model. The Mobile MQA module aims to enhance the cohesion of color features and improve classification accuracy by using an attention mechanism to enable the improved YOLO v12 model to more effectively focus on the color features of the eggshell and suppress background interference from non-eggshell regions in the image.

[0045] The implementation principle of the mobile multi-query attention module is as follows: First, the input feature map is linearly transformed into a query (denoted as Query). ), key (denoted as ) and value (denoted as Transformations are typically performed using lightweight convolution operations to meet the computational efficiency requirements of mobile or embedded devices.

[0046] Secondly, the mobile multi-query attention module employs a multi-head attention mechanism to manage queries. ,key Sum The system is divided into multiple heads. Attention is calculated independently for each head. For each attention head... The calculation process is as follows: ; in, It is the first A query matrix with attention heads; It is the first The key matrix of each attention head; It is the first A matrix of values ​​for each attention head; It is the dimension of the key vector, used to scale the dot product to stabilize the gradient; This represents the dot product of the query and the key, used to calculate the similarity between the query and each key. This similarity represents the relevance or importance between different features. The function is used to normalize the similarity scores into attention weights, ensuring that the sum of all weights is 1.

[0047] The attention weights are then multiplied by the value matrix. This allows for the weighted aggregation of information from different spatial locations and feature channels.

[0048] Then, the weighted values ​​calculated from all attention heads are concatenated and output through a final linear transformation (e.g., a 1x1 convolutional layer) to generate a feature map that integrates enhanced color and spatial information.

[0049] By moving the multi-query attention module, the advanced YOLO v12 model can generate corresponding queries based on the eggshell's color information and search for keys related to these queries throughout the feature map, identifying regions with similar color features. Simultaneously, for background regions unrelated to the eggshell color, their corresponding keys have low similarity to the color queries, thus receiving lower weights in the attention weight allocation, achieving the goal of suppressing background interference. This mechanism allows the feature map processed by the backbone layer to more clearly highlight the eggshell's color boundaries and internal color consistency, improving the accuracy of subsequent color classification.

[0050] Step S2 further includes feature fusion network optimization.

[0051] Feature fusion network optimization is mainly achieved by structurally adjusting and replacing convolutional operations in the neck layer of the improved YOLO v12 model, aiming to reduce computational cost while maintaining effective feature information transfer.

[0052] First, the neck network structure is simplified. The neck network is responsible for fusing feature maps output from backbone networks of different scales. In this embodiment, a simplified neck network replaces the original neck network. Specifically, only the bidirectional fusion path between the P3 and P4 layer feature maps is retained, i.e., upsampling and fusion operations are performed between the P3 and P4 feature maps. Simultaneously, the direct fusion path between the P5 and P3 layer feature maps is eliminated. This structural adjustment aims to reduce the computational complexity and number of parameters of the neck network by reducing redundant feature fusion paths.

[0053] Secondly, during feature fusion, Gs convolutions replace the standard 3x3 convolutions in the original network. Gs convolutions are used for feature integration, and their core principle lies in significantly reducing computational cost while maintaining feature expressiveness through depthwise separable convolutions and channel shuffling operations. Specifically, a Gs convolution operation typically includes the following steps: Perform a standard convolution operation on the input feature map to generate the first part of the feature map.

[0054] For the first part of the feature map, a series of computationally inexpensive linear transformations (such as pointwise convolution) are applied to generate the second part of the feature map.

[0055] The first part of the feature map is concatenated with the second part of the feature map along the channel dimension to form a complete output feature map.

[0056] Perform a channel shuffling operation to mix the spliced ​​feature map channels in order to enhance information exchange between different feature maps.

[0057] By replacing the standard 3x3 convolution with Gs convolution, the computational resources required for the feature fusion stage can be effectively reduced without sacrificing feature extraction capabilities, thereby improving the overall efficiency of the model.

[0058] The simplified collaborative application of the neck network and Gs convolution enables the efficient integration and transmission of features extracted from the backbone network at a lower computational cost, supporting the accuracy requirements of subsequent color classification and surface feature analysis, while also meeting the need for lightweight models.

[0059] Step S2 further includes model training optimization.

[0060] Model training optimization is achieved by introducing a weighted intersection-union ratio (IU). The loss function replaces the original network's loss function to optimize the model's bounding box regression performance during training. Traditional bounding box regression loss functions, such as those based on... The (Intersection over Union) loss has limitations when handling high-quality anchor boxes or low-quality samples. Specifically, when the model generates predicted boxes that highly overlap with the ground truth bounding boxes, the loss gradient for these high-quality anchor boxes is often large, causing the model to overemphasize small improvements and thus reduce generalization ability. Meanwhile, extremely low-quality samples (those that closely match the ground truth bounding boxes) have limitations. Extremely low prediction bounding boxes (RBOs) can generate harmful gradients, disrupting the training process. This invention introduces... The loss function aims to address these issues by dynamically adjusting the loss weights of anchor boxes of different quality. This reduces the competitiveness of high-quality anchor boxes while mitigating harmful gradients generated by low-quality samples, allowing the model to focus on anchor boxes of average quality and thus improving the overall performance of the detector.

[0061] The formula for calculating the loss function is as follows: ; in, This represents the weighted intersection-union loss function; The basic bounding box regression loss function can be, for example, a... Loss function. When using When the loss function is used, The calculation method is as follows: ; in, It predicts the bounding box. With the true bounding box The crossover ratio between them; Represents the predicted bounding box Center point and true bounding box The square of the Euclidean distance between the centers of; This indicates that the predicted bounding box is included. and the true bounding box The length of the diagonal of the smallest bounding rectangle.

[0062] Represents a non-monotonic dynamic weighting factor used to adjust the quality of the predicted bounding box (i.e., its resemblance to the true bounding box). The loss contribution is adaptively adjusted (value). Its design gives it the following characteristics: When the predicted bounding box matches the true bounding box When the value is extremely high (high-quality anchor frame). The value is relatively small to reduce its contribution to the loss and avoid over-optimization.

[0063] When the predicted bounding box matches the true bounding box When the value is extremely low (low-quality sample). The value is also relatively small to suppress harmful gradients generated by these noisy samples.

[0064] When the predicted bounding box matches the true bounding box When the value is at a medium level (normal quality anchor frame), The relatively large value of allows the improved YOLO v12 model to focus on learning and improving these prediction boxes with enhancement potential.

[0065] By introducing The improved YOLO v12 model of this invention can more effectively balance the optimization requirements of different quality prediction boxes during training, thereby obtaining a more reliable and accurate model for egg color classification and surface feature analysis.

[0066] Step S2 further includes a method for generating a surface feature mask image. The purpose of this method is to generate a binary mask image that accurately marks the defective areas on the eggshell surface based on the detection results of the improved YOLO v12 model, providing spatial guidance for extracting pure spectral information in the subsequent step S3.

[0067] The surface feature mask generation process takes the output of a single visual image of an egg processed by an improved YOLO v12 model as input. The output contains a set of bounding boxes, each with a category label (e.g., "egg" or "defect") and its coordinate position information in the image.

[0068] The specific steps for generating the mask image are as follows: First, initialize an all-zero matrix with the exact same size as the input visual image, as the canvas for the mask image, denoted as . .

[0069] Secondly, based on the detection box labeled "egg," the main region of the egg in the image is determined. Within the rectangular region defined by this detection box, the outline of the eggshell is accurately extracted using image segmentation algorithms, such as image binarization combined with morphological closing operations. Subsequently, the canvas... The values ​​of all pixels corresponding to the area inside the eggshell outline are set to a first preset value (e.g., 255). After this step, an initial mask image containing only the complete eggshell area is obtained. .

[0070] Next, iterate through all bounding boxes labeled "Defect". For each defect bounding box, within the rectangular area it covers, on the canvas... The values ​​of all corresponding pixels are set to the second preset value (e.g., 0).

[0071] Finally, by integrating the above processing results, the final surface feature mask image is generated. Specifically, the final surface feature mask image... By using the initial mask image This was obtained by correcting the defect area information. In the image, the area with a pixel value of the first preset value (255) represents a clean, flawless eggshell surface, while the area with a pixel value of the second preset value (0) represents the image background or the location of blemishes on the eggshell.

[0072] Generated surface feature mask map The intact and defective areas of the eggshell were accurately distinguished at the pixel level, providing a reliable spatial basis for selecting regions of interest (ROI) in subsequent hyperspectral data processing.

[0073] See attached document Figure 5 In step S3, the present invention first clarifies the technical defects of the traditional region of interest (ROI) selection method.

[0074] In the field of non-destructive testing of agricultural products based on hyperspectral imaging, traditional region of interest (ROI) selection methods typically employ manual or predefined fixed shapes (e.g., rectangles, circles, or ellipses) to select the target sample region. These methods aim to extract the average spectral information of the target region from the hyperspectral image. However, for agricultural products such as eggs, which have irregular shapes and may have various surface defects, these traditional ROI selection methods have technical limitations.

[0075] Specifically, traditional ROI selection methods fail to effectively identify and avoid visual defects on the egg surface, such as spots, stains, cracks, or uneven coloring. When a predefined ROI or manually selected area inadvertently covers these defects, the extracted spectral data will contain reflectance information from these defects. The spectral responses of these defects differ fundamentally from the spectral characteristics of the egg's internal quality (such as freshness), leading to distorted spectral curves that fail to accurately reflect the egg's internal composition and structural state. This contamination of spectral information directly affects the accuracy and reliability of subsequent freshness detection models.

[0076] Furthermore, manually selecting ROIs is inefficient, time-consuming, and highly dependent on the operator's experience and subjective judgment, making it difficult to guarantee the consistency and standardization of the selection results. For the egg quality inspection needs of large-scale, automated production lines, this non-automated ROI selection method is clearly impractical. Even using predefined fixed-shape ROIs, due to the large individual differences in eggs, a single fixed-shape ROI cannot perfectly fit the irregular contours of all eggs, and may include background information or omit some effective areas, further introducing interference.

[0077] In summary, traditional ROI selection methods fail to address the interference of surface defects on the purity of spectral data and lack automation and standardization capabilities, severely limiting the accuracy and efficiency of hyperspectral technology in the non-destructive testing of egg freshness.

[0078] Step S3 further includes an adaptive spectral ROI dynamic programming algorithm.

[0079] The adaptive spectral ROI dynamic programming algorithm aims to overcome the shortcomings of traditional ROI selection methods. It utilizes visual feature analysis results to perform refined region selection on hyperspectral images, ensuring that the extracted spectral information is pure and free from surface imperfections. This algorithm takes as input a hyperspectral image after black-and-white correction and a surface feature mask generated by a surface feature mask generation method.

[0080] The core of the adaptive spectral ROI dynamic programming algorithm lies in automatically identifying and determining one or more optimal regions of interest representing the clean surface of an egg in a hyperspectral image based on the surface feature mask map. This avoids including any areas with identified surface defects.

[0081] The specific execution process of the adaptive spectral ROI dynamic programming algorithm is as follows: Input reception and region identification: Receives hyperspectral images after black-and-white correction and surface feature mask images. Surface feature mask It is a binary image, where regions with pixel values ​​of a first preset value (e.g., 255) represent clean eggshell surfaces, while regions with pixel values ​​of a second preset value (e.g., 0) represent background or surface imperfections. The adaptive spectral ROI dynamic programming algorithm is first based on... It identifies all interconnected candidate regions consisting only of clean eggshell pixels.

[0082] Region Optimization and Selection: For the identified clean eggshell candidate regions, the adaptive spectral ROI dynamic programming algorithm adopts the idea of ​​dynamic programming to optimally select the largest continuous flawless region as the optimal region of interest. The optimization objective is to identify the largest contiguous, flawless region to maximize the purity and representativeness of the extracted spectral data.

[0083] Specifically, the adaptive spectral ROI dynamic programming algorithm uses dynamic programming to process surface feature mask images. The analysis is performed. This method efficiently determines the largest rectangular clean area by iteratively calculating and updating the maximum size of the flawless region that can be formed at each pixel location. This process ensures the selection of... It is located entirely within the clean eggshell area and has a sufficiently large area to provide a stable spectral signal.

[0084] If an auxiliary function is defined , indicating in image coordinates If the number of pixels at a given location is consecutively upwards and has a value equal to the first preset value (clean pixel), then the recursive relationship is as follows: ; in, and These represent the row and column indices of the image, respectively, with the boundary conditions being... ; Indicates the coordinates in the image The number of consecutive clean pixels at that location; Represents the surface feature mask in image coordinates The pixel value at that location; the first preset value represents The pixel values ​​of the clean eggshell area; the second preset value represents The pixel value of the defective or background area.

[0085] Indicates if image coordinates If the pixel at that location is a clean eggshell pixel, then execute... The calculation involves adding 1 to the consecutive clean pixel count value at the corresponding position in the previous row; if the image coordinates If the pixel at that location is a defective or background pixel, then execute... The operation involves resetting the consecutive clean pixel count here to 0. By calculating line by line Each row is treated as a histogram, and then a known efficient algorithm (such as an algorithm based on a monotonic stack) is used to calculate the area of ​​the largest rectangle that can be formed in the histogram, thus determining the final area. The largest clean rectangular region in the system. The boundary coordinates of the rectangular region. That is, determined to be .

[0086] Spectral data extraction: Once It has been determined that the adaptive spectral ROI dynamic programming algorithm will extract the ROI only from the black-and-white corrected hyperspectral image. The spectral data of all pixels within the region is extracted. The extracted spectral data will be averaged to obtain the average spectral curve of the region, which will be used for subsequent feature extraction and model training.

[0087] By using an adaptive spectral ROI dynamic programming algorithm, regions affected by surface defects can be automatically and accurately extracted from hyperspectral images, ensuring that the acquired spectral information is pure and reliable, thereby improving the accuracy and reliability of non-destructive testing of egg freshness based on hyperspectral technology.

[0088] Step S3 further includes the extraction and generation of average spectral information. This step aims to optimize the region of interest. The hyperspectral data within the range is processed into a single, representative average spectral curve.

[0089] The process of extracting and generating average spectral information firstly involves... The boundary coordinates are used to locate all pixels belonging to the optimal region of interest within the hyperspectral data cube after black-and-white correction.

[0090] Subsequently, the reflectance values ​​of all pixels within the optimal region of interest are calculated using an arithmetic mean in each spectral band. This calculation generates a one-dimensional vector, which represents the average spectral curve of the egg sample. The formula for calculating the average spectral curve is as follows: ; in, At wavelength The average spectral reflectance value is calculated at that time. It is the optimal region of interest. The total number of pixels contained within; In the hyperspectral data cube, located at the image coordinates The pixel at wavelength is The spectral reflectance value at that time; This indicates that for all regions of interest that belong to the optimal region of interest... The pixels are summed.

[0091] By repeating the above calculations for all spectral bands, a complete average spectral curve is finally generated. This average spectral curve represents the comprehensive spectral characteristics of the clean, flawless surface of the egg sample.

[0092] The generated average spectral curve is a one-dimensional data vector, where each element corresponds to the average reflectance at a specific wavelength. This data vector, as a pure spectral feature, will be used for training and validation of the egg freshness prediction model in subsequent step S4, thus providing a data foundation for achieving high-precision non-destructive testing.

[0093] See attached document Figure 3 In step S4, a spectral data preprocessing method is first performed. This method aims to eliminate the influence of ambient light interference, differences in the physical state of the samples, and equipment noise on the raw spectral data, thereby improving the signal-to-noise ratio for subsequent feature extraction and modeling.

[0094] The spectral data preprocessing method first divides all egg samples to be tested into a dark egg set and a light egg set based on the egg surface color information obtained in step S2. Then, the subsequent spectral preprocessing process is performed independently for each set to eliminate the systematic bias caused by the inherent color differences of the eggshell on the spectral reflectance benchmark.

[0095] For the classified spectral data, a normalization method is first used to map the spectral data to a specific interval to eliminate differences in data dimensions. This invention employs the min-max normalization method, and its calculation formula is as follows: ; in, This represents the normalized spectral reflectance value. This represents the original spectral reflectance value; This represents the minimum reflectance value in the spectral curve of the sample. This represents the maximum reflectance value in the spectral curve of the sample.

[0096] Subsequently, the normalized data undergoes standardization, transforming it into a distribution with a mean of 0 and a standard deviation of 1 to improve data comparability. The calculation formula is as follows: ; in, This represents the standardized spectral data; This represents the normalized spectral reflectance value; This represents the arithmetic mean of all spectral data in the sample. This represents the standard deviation of all spectral data across all bands in the sample.

[0097] Furthermore, to eliminate spectral baseline drift and enhancement effects caused by differences in the scattering characteristics of the egg surface and changes in optical path, this invention employs the Multiplicative Scatter Correction (MSC) method to correct the data. The specific steps of multiplicative scatter correction are as follows: First, calculate the average spectrum of all calibration set samples as the ideal average spectrum. .

[0098] Secondly, for each individual spectral vector The relationship between the univariate linear regression method and the mean spectral vector is established. Linear relationship between them: ; in, Indicates the first spectral vectors of each sample; Represents the average spectral vector of all samples; Indicates the first The regression intercept of each sample reflects the additive shift of the spectrum; Indicates the first The regression slope of each sample reflects the multiplicative shift of the spectrum; This represents the regression residual.

[0099] Finally, using and For spectral vectors Inversion correction is performed to obtain the corrected spectral data. : ; in, This indicates the th element after multivariate scattering correction. Spectral data of each sample.

[0100] Through the above color classification-based grouping process, combined with the cascaded preprocessing strategies of normalization, standardization and multivariate scattering correction, high-frequency noise, baseline drift and light scattering interference in the spectral data are effectively removed, providing a highly consistent data foundation for subsequent feature wavelength screening and freshness determination model construction.

[0101] See attached document Figure 6In step S4, a feature wavelength screening method is further included. This method aims to reduce the dimensionality of the preprocessed high-dimensional full-band spectral data, removing redundant information and irrelevant noise to construct a concise and informative feature set. This invention employs a cascaded screening strategy, first introducing Pearson correlation coefficient analysis for preliminary screening, and then introducing the Extreme Gradient Boosting (XGBoost) classification algorithm for secondary refinement.

[0102] The characteristic wavelength screening method first employs Pearson correlation coefficient analysis to quantify the linear correlation between each spectral band and egg freshness indicators. For each wavelength variable in the preprocessed spectral data matrix, the Pearson correlation coefficient between it and the sample freshness label is calculated.

[0103] The formula for calculating the Pearson correlation coefficient is as follows: ; in, Indicates the first Pearson correlation coefficients between wavelength variables and freshness labels; This represents the total number of samples in the training set; Indicates the first The sample at the th Preprocessed spectral reflectance values ​​at each wavelength; Indicates that all samples are in the first... The arithmetic mean of the spectral reflectance values ​​at each wavelength; Indicates the first The freshness label value of each sample; This represents the arithmetic mean of the freshness label values ​​for all samples. Indicates the first The covariance between the spectral data of each wavelength and the freshness label reflects the synchronicity of their changes; Indicates the first The sum of squared deviations of all sample spectral data at a given wavelength reflects the degree of fluctuation in the data at that wavelength. This represents the sum of squared deviations of all sample freshness label data.

[0104] After the calculation is completed, based on the results... The absolute values ​​of the wavelengths are used to sort all wavelengths. A correlation threshold is set, and wavelengths with absolute values ​​greater than the threshold are retained, or a preset number of wavelengths with the highest absolute values ​​are selected as the initial set of feature wavelengths.

[0105] Subsequently, to further explore the nonlinear relationship between feature wavelengths and freshness and remove collinear features, this invention introduces the XGBoost algorithm to perform a secondary screening of the initially selected feature wavelength set. The XGBoost algorithm is based on a gradient boosting decision tree framework, evaluating the contribution of wavelengths by calculating the importance score of features during the tree model construction process.

[0106] During the training of the XGBoost model, when splitting each leaf node of the decision tree, a gain metric is used to evaluate the merits of candidate features and their split points. The formula for calculating the gain metric is as follows: ; in, This represents the structural fraction gain resulting from splitting at a certain split point of a certain feature. and These represent the sum of the first-order gradients of all samples in the left and right child nodes after the split, respectively. and These represent the sum of the second-order gradients of all samples in the left and right child nodes after the split, respectively. The coefficients of the L2 regularization term are used to control model complexity. This represents the minimum loss reduction required for a node to split, i.e., the node splitting threshold. The structure score of the left child node after splitting reflects the contribution of the left child node to reducing the model loss. The structure score of the right child node after splitting reflects the contribution of the right child node to reducing the model loss. This represents the structural score when no splitting occurs (i.e., the parent node).

[0107] The XGBoost algorithm calculates the final importance score for each feature wavelength by accumulating the average gain from each feature acting as a splitting node across all decision trees. The wavelengths in the initial feature wavelength set are then sorted from highest to lowest importance score. Several wavelengths with a cumulative contribution rate reaching a preset proportion or ranking highly are selected to form the final feature wavelength subset. This feature wavelength subset will serve as input data for the subsequent KNN model construction.

[0108] Step S4 further includes a method for constructing and optimizing a freshness classification model. This step aims to use a selected subset of feature wavelengths as input features, employ the K-Nearest Neighbors (KNN) classification algorithm to construct an egg freshness detection model, and optimize the model's hyperparameters using the Particle Swarm Optimization (PSO) algorithm to achieve high-precision and high-robustness freshness determination.

[0109] The freshness classification model construction and optimization method is first based on the KNN classification algorithm. The core of the KNN algorithm is that, for a new egg sample to be classified (test sample), the distance between it and all known freshness samples in the training set is calculated in the feature space, and the K nearest neighbors are selected. The freshness category of these K neighbors is voted on, and the category with the most votes is taken as the final judgment result of the test sample.

[0110] exist In the 3D feature space, the sample to be classified With any sample in the training set Distance between The calculation is performed using Euclidean distance, and the formula is as follows: ; in, Indicates the sample to be classified With training set samples The Euclidean distance between them; Represents the dimension of the feature space; Indicates the sample to be classified is at the th Spectral reflectance values ​​at each characteristic wavelength; Represents the first in the training set The training sample at the th ... Spectral reflectance values ​​at each characteristic wavelength; Indicates the sample to be tested and the first The sum of squares of the numerical differences of each training sample across all selected feature wavelength dimensions.

[0111] The key to building a KNN model lies in selecting the optimal... The value (number of neighbors) and distance metric methods are used. To accurately determine the optimal combination of these key parameters, this invention introduces the Particle Swarm Optimization (PSO) algorithm for global optimization. The PSO optimization algorithm optimizes the key parameters of the KNN model (mainly...) The combination of values ​​is considered as the position of a particle in the multidimensional search space. Algorithm initialization is performed by... A group of particles, in which the first... Individual particles The position in the dimensional parameter space is denoted as .

[0112] The PSO algorithm uses the classification accuracy of the KNN model on the cross-validation set as the fitness function, and iteratively searches to continuously update the particle velocity. and location The velocity update formula and the position update formula are as follows: ; ; in, and Indicates the first The particle in the first Second and third The velocity vector at the next iteration; and Indicates the first The particle in the first Second and third The position vector at the next iteration (i.e., the current KNN parameter combination); This represents the inertia weight, used to balance global exploration and local development capabilities; and This represents the acceleration factor (learning factor), which adjusts the learning step size of a particle towards its own optimal solution and the group's optimal solution; and Indicates in A random number that is uniformly distributed within a range; Indicates the first The particle up to the [number]th The individual's best position (individual extreme value) experienced in the next iteration; This indicates that all particles up to the [number]th ... The global best position (global extreme value) experienced in the next iteration.

[0113] Finally, the global extremum output by the PSO algorithm is... The corresponding location parameters are used as the optimal parameter combination for the KNN model. Using these optimal parameters and a subset of feature wavelengths, the model is retrained on the full training data to construct the final egg freshness detection model. This model can receive the spectral feature input of the egg to be tested and output its corresponding freshness level or predicted value, thus achieving accurate and non-destructive determination of egg freshness.

Claims

1. A method for detecting egg freshness based on machine vision color classification and hyperspectral imaging, characterized in that, Includes the following steps: S1. For the same egg sample, acquire visual images and hyperspectral images, and perform black and white correction processing on the hyperspectral images to convert the original image data into standard reflectance data; S2. Process the visual image to obtain the color classification result of the eggshell and a surface feature mask map that marks the defective areas on the eggshell surface. S3. Using the surface feature mask, select the region of interest that avoids the defective area on the eggshell surface on the standard reflectance data, and calculate the average spectral reflectance of the region of interest to generate a spectral curve; S4. Preprocess the spectral curve, extract feature wavelength combinations from the preprocessed spectral curve using a feature filtering algorithm, train a classification model using the feature wavelength combinations, establish an egg freshness detection model, and output the judgment result.

2. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, In step S2, generating the surface feature mask image includes: Initialize an all-zero matrix of the same size as the visual image as a mask drawing canvas; Based on the detection box marked as an egg in the color classification result, the main egg area is determined, and the pixel point in the mask canvas corresponding to the main egg area is set to the first preset value; The detection boxes marked as defects are traversed, and the pixels of the detection boxes corresponding to the defects in the mask canvas are set to a second preset value, thereby generating the surface feature mask map.

3. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, Step S2 is implemented using an improved YOLO model, and a mobile multi-query attention module is embedded in the backbone network of the improved YOLO model. The mobile multi-query attention module re-weights the input features through a multi-head attention mechanism.

4. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 3, characterized in that, The neck network of the improved YOLO model is simplified, retaining the bidirectional fusion path of some feature maps, and Gs convolution is used to replace the standard convolution for feature integration, in order to reduce computational complexity and the number of parameters.

5. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 3, characterized in that, The improved YOLO model is trained and optimized by introducing a weighted intersection-over-union (IoU) loss function. The weighted IoU loss function adjusts the loss weights of anchor boxes of different quality through a non-monotonic dynamic weighting factor.

6. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, In step S3, the region of interest is selected by executing an adaptive spectral ROI dynamic programming algorithm. The adaptive spectral ROI dynamic programming algorithm determines the largest continuous flawless region as the region of interest based on the surface feature mask map through dynamic programming.

7. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, Step S4, the preprocessing of the spectral curve includes: First, based on the color classification results, the egg samples are divided into a dark egg set and a light egg set; Then, normalization, standardization, and multivariate scattering correction operations are performed independently on the dark egg set and the light egg set, respectively.

8. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, The feature selection algorithm includes: First, the Pearson correlation coefficient analysis method was used to calculate and sort the correlation between all wavelengths and the freshness label, and to conduct preliminary screening to obtain a preliminary set of characteristic wavelengths. Then, an extreme gradient boosting classification algorithm is introduced to further refine the initial set of feature wavelengths. Based on the gain index, the feature importance score is calculated to determine the final combination of feature wavelengths.

9. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, The classification model is a K-nearest neighbor classification model, and the hyperparameters of the K-nearest neighbor classification model are globally optimized using a particle swarm optimization algorithm to determine the optimal parameter combination.

10. The egg freshness detection method based on machine vision color classification and hyperspectral imaging according to claim 1, characterized in that, The black-and-white correction process for the hyperspectral image includes: Acquire dark current images and standard whiteboard images; The hyperspectral image is then corrected using the dark current image and the standard whiteboard image to obtain the standard reflectance data.

Images