A Machine Vision-Based Method for Detecting Seed Breakage Rate in Sweet Potato Origin Pellet Granulation

Abstract

This invention provides a machine vision-based method for detecting the breakage rate of sweet potato mono-embryo pelleted seeds, belonging to the field of seed detection technology. The invention first acquires visible and near-infrared images of seeds under standard illumination, as well as non-standard visible light images under multiple light sources, and performs reflectance calibration and polarization difference correction processing. Based on the visible and near-infrared reflectance images, features such as color and texture are extracted to construct a multispectral feature vector. Using photometric stereo vision technology, combined with multiple non-standard visible light images, surface normal vectors are calculated. An initial three-dimensional height map is generated through integration, and multiple anomaly judgment features are extracted to mark suspected breakage areas. Further edge sharpness indices are calculated to eliminate mechanically damaged areas, determining the final breakage area and its three-dimensional contour. Finally, combining multi-dimensional features such as volume loss rate and projected area ratio with the multispectral feature vector, a breakage probability prediction model is constructed to achieve accurate and efficient detection of seed breakage rate.

Description

Technical Field

[0001] This invention relates to the field of seed testing technology, specifically to a machine vision-based method for detecting the breakage rate of sweet potato mono-embryo pelleted seeds. Background Technology

[0002] Sweet potato seed pelleting is a crucial step in modern sugar agriculture. It involves coating seeds with a coating agent to form uniform pellets suitable for precision seeders. However, during pelleting, packaging, transportation, and sowing, seeds are prone to surface cracks, internal structural damage, or coating peeling due to mechanical compression or impact. These damages directly affect germination rate, seedling uniformity, and final yield. Therefore, efficient and accurate damage rate detection of pelleted seeds is an important prerequisite for ensuring agricultural production efficiency.

[0003] Existing detection methods primarily rely on manual visual observation or machine vision systems based on two-dimensional visible light images. Manual inspection is not only inefficient and highly subjective, but also incapable of handling the rapid sorting needs of large-scale seed production. While traditional monocular vision systems can identify obvious surface color anomalies or large defects, they lack sensitivity for minute cracks, shallow scratches, and damage masked by the coating material. More importantly, these methods cannot detect changes in seed surface morphology or internal structural damage, making it difficult to distinguish between genuine mechanical breakage and natural color spots or reflections, thus limiting the accuracy and reliability of detection.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based method for detecting the breakage rate of sweet potato mono-embryo pelleted seeds, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A machine vision-based method for detecting the breakage rate of sweet potato mono-embryo pelleted seeds, comprising the following steps:

[0008] Step 1: Collect visible light and near-infrared images of the seeds to be analyzed under standard illumination, as well as non-standard visible light images under k light sources. Process the above images based on the reflectance calibration image group and the polarization difference correction image group to obtain visible light reflectance images, near-infrared reflectance images, and k non-standard visible light reflectance images.

[0009] Step 2: Extract visible light image features and near-infrared image features based on visible light reflectance images and near-infrared reflectance images, and merge them to construct a multispectral feature vector;

[0010] Step 3: Combine each non-standard visible light reflectance image and its corresponding light source direction vector to calculate the normal vector of each pixel in the seed region; generate an initial three-dimensional height map by performing field integration on the normal vector to extract the abnormal judgment features of the seed surface, and mark the suspected damaged area according to the abnormal judgment features and the preset judgment conditions.

[0011] Step 4: Calculate the edge sharpness index of each suspected damaged area, remove the areas determined to be mechanically damaged, and output the final damaged area and its three-dimensional contour data;

[0012] Step 5: Based on the 3D contour data, calculate the maximum value of the total damaged area percentage, total absolute volume loss, and volume loss rate of the final damaged area for each seed, and combine them with the multispectral feature vector as input data to construct a damage probability prediction model and determine the damage probability of the seed to be analyzed.

[0013] Furthermore, the reflectivity calibration image set includes a whiteboard image and a dark field image, and the polarization difference correction image set includes a parallel polarized light image and a vertically polarized light image;

[0014] The acquisition logic for parallel polarized light images and vertical polarized light images is as follows: under standard illumination conditions equipped with a polarizing filter, parallel polarized light images and vertical polarized light images of the seed to be analyzed are acquired respectively to form a polarization difference correction image group.

[0015] The acquisition logic for visible light images, near-infrared images, and non-standard visible light images is as follows: using an RGB color camera and a monochrome near-infrared camera with the same pixel level, visible light images and near-infrared images of the seeds to be analyzed are acquired simultaneously under standard lighting conditions. The two images are then image registered to transform them to the same coordinate system. The standard lighting condition is uniform lighting condition. Then, by controlling k different light sources with known directions to be lit sequentially, k non-standard visible light images of the seeds to be analyzed under different lighting conditions are acquired to form a set of non-standard visible light images used to generate the initial three-dimensional height map.

[0016] The logic for processing the above images based on the reflectance calibration image group and the polarization difference correction image group is as follows: the visible light image, near-infrared image, and non-standard visible light image are processed by the reflectance calibration image group to obtain the visible light reflectance image to be corrected, the near-infrared reflectance image, and the non-standard visible light reflectance image to be corrected. Then, the polarization difference image group is used to perform polarization difference correction on the visible light reflectance image to be corrected and the non-standard visible light reflectance image to be corrected to obtain the visible light reflectance image and the non-standard visible light reflectance image.

[0017] Furthermore, constructing the multispectral feature vector specifically includes:

[0018] The visible light image features include color features and texture features. The color features are hue and saturation, and the texture features are GLCM contrast. The near-infrared image features include transmittance and LBP entropy.

[0019] The visible light reflectance image is converted to the HSV color space and processed using the OTSU thresholding method to initially separate the background and foreground as seed candidate regions. For the initially separated foreground regions, the mean and standard deviation of hue and saturation of all pixels are calculated to define upper and lower thresholds for hue and saturation of the seed regions. The lower threshold for hue is... Hue upper limit threshold , The mean value of hue. The standard deviation of hue, and the lower limit threshold of saturation. Saturation upper limit threshold , The mean value of saturation. The standard deviation of saturation is the hue value in the foreground region falling within the range of saturation. , Within the range, and the saturation value falls within... The region comprised of pixels within the specified range is defined as the final seed region. The mean hue and mean saturation of all pixels within the seed region are calculated as the color features of the seed to be analyzed. The seed region of the visible light reflectance image is converted into a grayscale image, and its gray-level co-occurrence matrix is ​​calculated. The GLCM contrast of the seed region is calculated using the gray-level co-occurrence matrix as the texture feature. (The reason why pixels falling within the threshold range are considered seed regions needs to be explained later; it seems a bit strange.)

[0020] Based on the location of the seed region in the visible light reflectance image, the seed region is determined in the near-infrared reflectance image. In the seed region of the near-infrared reflectance image, the transmittance is calculated based on the red channel reflectance. The seed region in the image is then locally binarized to calculate the LBP entropy.

[0021] Visible light image features and near-infrared image features are concatenated to construct a multispectral feature vector, represented as follows: ,in, This is a multispectral feature vector, where H represents hue and S represents saturation. Transmittance, For GLCM contrast, Let LBP be the entropy.

[0022] Furthermore, generating the initial 3D height map specifically includes:

[0023] Based on k visible light reflectance images, a photometric stereo vision algorithm is used to calculate the unit normal vector of each pixel within the seed region, specifically as follows: For each pixel within the seed region, the pixel brightness value at the same location in the k visible light reflectance images is extracted to form a k-dimensional observation vector. This vector is then simplified based on the reflectance function of the Lambertian surface, resulting in:

[0024] ;

[0025] in, Represents the observation vector. For surface reflectivity, For unit normal vector, The light source direction matrix is ​​k×3. This represents the three-dimensional row vector of the first light source. This represents the three-dimensional row vector of the second light source. This represents the three-dimensional row vector of the k-th light source;

[0026] By solving the following overdetermined system of equations, namely The least-squares solution for the normal vector and albedo is obtained, and after normalization, the unit normal vector is obtained. Then, the surface gradient is extracted from the normal vector field using the following relation: , ,in, The component of the unit normal vector on the X-axis in three-dimensional space. The component of the unit normal vector on the Y-axis in three-dimensional space. Let p be the component of the unit normal vector on the Z-axis in three-dimensional space, p be the partial derivative of the height in the X direction, and q be the partial derivative of the height in the Y direction.

[0027] Then, by solving the following Poisson equation and integrating the gradient field, an initial 3D height map is generated, as shown in the following formula:

[0028] ;

[0029] in, (x, y) represents the height of the pixel (x, y) in three-dimensional space, (p, q) represents the gradient field, and (x, y) is the position of the pixel.

[0030] Furthermore, marking the suspected damaged area specifically includes:

[0031] The anomaly detection features include Gaussian curvature, mean curvature, and local height difference; based on the initial 3D height map, the first and second partial derivatives of each pixel are calculated, and then the Gaussian curvature and mean curvature of each pixel are calculated according to the differential geometry formula; at the same time, the absolute value of the height difference between each pixel and its neighboring pixels in the initial 3D height map is calculated.

[0032] The preset judgment conditions include geometric anomaly conditions and spectral verification conditions. The geometric anomaly conditions are as follows: ,in, Let (x, y) be the Gaussian curvature of the pixel. Let be the average curvature of pixel (x, y). This is the mean of the absolute values ​​of the height differences between a pixel (x, y) and its neighboring pixels. Represents logical OR, , , These represent the preset Gaussian curvature threshold, average curvature threshold, and height difference threshold, respectively.

[0033] The spectral verification conditions are as follows: ,in, This represents the transmittance of a pixel (x, y). This represents the average transmittance of the neighboring pixels of pixel (x, y). This represents the standard deviation of the transmittance of the neighboring pixels of pixel (x,y). These are adjustable parameters;

[0034] Pixels that simultaneously meet the geometric anomaly condition and the spectral verification condition are marked as suspected damaged points, and other pixels are marked as normal points. Output a binary image, setting the pixel value of suspected damaged points to 1 and the pixel value of normal points to 0.

[0035] An 8-neighborhood scan is performed on the binary image, and all spatially adjacent pixels with a value of 1 are grouped into the same connected region. Each connected region is treated as an independent candidate for suspected damaged area. Based on a preset minimum damaged area threshold, candidates for suspected damaged areas that are smaller than the minimum damaged area threshold are removed, and the remaining candidates for suspected damaged areas are taken as suspected damaged areas.

[0036] Furthermore, determining whether it is mechanical damage specifically includes:

[0037] N edge points are collected on the edge contour of each suspected damaged area. Based on the initial 3D height map, the height gradient magnitude of each edge point is calculated using the following formula:

[0038] ;

[0039] in, This represents the magnitude of the height gradient at pixel (x, y).

[0040] Calculate the average and maximum values ​​of the height gradient amplitudes of all edge points in each suspected damaged area, and use them as the edge sharpness index. Based on the set mechanical damage judgment conditions, determine whether it is mechanical damage. The mechanical damage judgment conditions include: Condition 1: The average value of the height gradient amplitudes of all edge points is greater than the preset threshold for the average height gradient amplitude of normal damage; Condition 2: The maximum value of the height gradient amplitudes of all edge points is greater than the preset threshold for the maximum value of the height gradient amplitude of normal damage. If either judgment condition is met, the suspected damaged area is determined to be mechanical damage.

[0041] The areas identified as having mechanical damage are removed from the suspected damage areas, and the remaining suspected damage areas are output as the final damage areas. The three-dimensional contour data corresponding to each final damage area is also output.

[0042] Furthermore, the three-dimensional contour data includes the coordinates and height of all pixels within the region. Based on the binary image of the seed to be analyzed, the number of pixels in each final damaged region is counted to calculate the area ratio, as shown in the following formula:

[0043] ;

[0044] in, This represents the area percentage of the j-th ultimately damaged region within the entire seed to be analyzed. Let be the projected area of ​​the j-th final damaged region on the binary image, which is the number of pixels in that final damaged region. The total projected area occupied by the seed to be analyzed in the binary image is denoted as , which is the number of pixels of the seed to be analyzed in the binary image, and j is the index of the final damaged area.

[0045] Calculate the arithmetic mean of the heights of all pixels in all ultimately damaged areas as the baseline height; obtain the physical area of ​​a single pixel using camera calibration parameters, and compare the height of each pixel with the baseline height to calculate the absolute volume defect. The specific formula is as follows:

[0046] ;

[0047] in, This represents the absolute volume loss of the j-th ultimately damaged region. Let R(j) be the physical area of ​​a single pixel, and let R(j) represent the set of pixels within the j-th final damaged region. As the reference height, This represents the height of the pixel (x, y).

[0048] Based on the initial 3D height map, the total volume of the seed to be analyzed is calculated. Specifically, the reference height is considered as the base height of the seed, and the total volume of the seed surface above the base height is calculated using the following formula:

[0049] ;

[0050] in, Let E represent the total volume of the seed to be analyzed, and let E represent the set of all pixels occupied by the seed in the binary image. This is the lowest point height in the entire seed region;

[0051] Based on the absolute volume loss and the total seed volume, the volume loss rate of each ultimately damaged area is calculated using the following formula:

[0052] ;

[0053] in, Let be the volume loss rate of the j-th final damaged region.

[0054] Furthermore, constructing the damage probability prediction model specifically includes:

[0055] The area percentage and absolute volume loss of all ultimately damaged areas are summed to obtain the total damaged area percentage and total absolute volume loss of seeds; the maximum volume loss rate of all ultimately damaged areas is determined; the total damaged area percentage, total absolute volume loss of seeds, and the maximum volume loss rate of all ultimately damaged areas are combined with the multispectral feature vector to obtain the combined feature vector.

[0056] Based on the above method, a set of sample seeds with known damage status is collected, and the combined feature vector of each sample seed is obtained. The damage assessment of each sample seed is independently performed by the expert scoring method according to the physical condition of the sample seed, and the damage probability value of each sample seed is determined as the training label. Based on the logistic regression model, the combined feature vector of the set of sample seeds is used as input, and its corresponding damage probability value is used as the label to train the model and build a damage probability prediction model.

[0057] The combined feature vector of the seed to be analyzed is input into the trained damage probability prediction model. The damage probability value output by the model is compared with the preset damage threshold to determine whether the seed to be analyzed is damaged.

[0058] The number of seeds identified as damaged in a group of seeds to be analyzed is counted, and their proportion in the group is calculated. This proportion is then used as the damage rate of the group of seeds to be analyzed. The technical effects and advantages provided by this invention in the above technical solution are as follows:

[0059] This solution effectively addresses the problems of low accuracy, poor anti-interference ability, and inability to quantify assessment in the detection of damage in sweet potato monoembryo pelleted seeds by integrating multispectral imaging and three-dimensional morphology reconstruction techniques. Utilizing the sensitivity of near-infrared images to internal structures and the enhancement of surface details by polarization-corrected visible light images, combined with the accurate reconstruction of surface geometric features using three-dimensional height maps, the system can reliably identify various damage types, including microcracks, internal damage, and surface depressions, significantly improving the detection rate and accuracy for complex damage features. By quantitatively calculating multi-dimensional geometric indicators such as volume loss and projected area ratio of the damaged area, and fusing them with multispectral features for decision-making, this method achieves an objective and quantitative evaluation of the degree of seed damage, avoiding misjudgments caused by subjective human error and natural morphological differences. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0062] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0063] Example:

[0064] Please see Figure 1 The present invention provides a technical solution:

[0065] A machine vision-based method for detecting the breakage rate of sweet potato mono-embryo pelleted seeds, comprising the following steps:

[0066] Step 1: Collect visible light and near-infrared images of the seeds to be analyzed under standard illumination, as well as non-standard visible light images under k light sources. Process the above images based on the reflectance calibration image group and the polarization difference correction image group to obtain visible light reflectance images, near-infrared reflectance images, and k non-standard visible light reflectance images.

[0067] In this embodiment, the core of the image acquisition system includes: a standard illumination box, a pixel-level aligned RGB color camera and a monochrome near-infrared camera (or a multispectral camera with corresponding spectral response capabilities), a diffuse white board with known reflectivity (as a reflectivity reference), a polarizing filter that can cover the camera lens, and k independently controllable point light sources with known spatial azimuth angles. The direction of each light source is defined by its zenith angle (the angle with the sample stage normal) and azimuth angle (the angle between its projection onto the sample stage plane and the reference direction). The standard illumination box provides a uniform and stable main illumination environment with known spectral characteristics. The k point light sources are arranged around the sample stage, and their zenith angles and azimuth angles are accurately recorded for subsequent calculation of the three-dimensional direction vectors of the light sources and for photometric stereo vision calculations. To ensure spatial consistency of the image, the RGB camera and the near-infrared camera are rigidly fixed in hardware and pre-calibrated to ensure that their optical axes are parallel and their imaging planes are coplanar, minimizing parallax. Reflectance calibration is a crucial step in eliminating camera dark current noise and illumination inhomogeneity, converting the original image grayscale values ​​into relative reflectance. The reflectance calibration image set includes white board images and dark field images, while the polarization difference correction image set includes parallel polarized light images and perpendicular polarized light images. The white board images are acquired under standard illumination conditions, with a standard diffuse white board of known reflectance placed in the center of the sample stage, using both an RGB color camera and a monochrome near-infrared camera. This image represents the maximum response of the imaging system under the current illumination. The dark field images are acquired under completely darkened conditions (i.e., all light sources are turned off), using both an RGB color camera and a monochrome near-infrared camera. This image characterizes the camera's dark current and readout noise.

[0068] Polarization difference correction aims to effectively suppress the interference of specular reflections (highlights) from the seed surface, which can obscure the texture and damage details of the seed surface. The acquisition logic for parallel-polarized and perpendicular-polarized light images is as follows: Under standard lighting conditions equipped with a polarizing filter, a linear polarizing filter is attached to the front of the camera lens. First, the polarizing filter is rotated so that its polarization direction is parallel to the polarization direction of the incident light (this state needs to be determined experimentally beforehand), and a parallel-polarized light image is acquired. Then, the polarizing filter is rotated 90 degrees so that its polarization direction is perpendicular to the polarization direction of the incident light, and a perpendicular-polarized light image is acquired. This process is repeated to acquire both parallel-polarized and perpendicular-polarized light images of the seed to be analyzed, forming a polarization difference corrected image set.

[0069] The acquisition logic for visible light images, near-infrared images, and non-standard visible light images is as follows: Under standard illumination conditions, the polarizing filter is removed (or adjusted to a state that does not affect imaging), and the RGB color camera and monochrome near-infrared camera are simultaneously triggered to acquire one visible light image and one near-infrared image, respectively. Due to slight differences in position and viewing angle between the two cameras, image registration is required for the acquired visible light and near-infrared images. A registration algorithm based on feature points (such as SIFT, ORB) or a phase correlation-based algorithm is used to calculate the reflection transformation or perspective transformation matrix between the two images, and the near-infrared image is transformed to the coordinate system of the visible light image to achieve pixel-level spatial alignment. The standard illumination condition is uniform illumination. Then, by controlling k different light sources with known directions to be lit sequentially, k non-standard visible light images of the seed to be analyzed under different illumination conditions are acquired, forming a set of non-standard visible light images used to generate the initial 3D height map.

[0070] The logic for processing the above images based on the reflectance calibration image group and the polarization difference correction image group is as follows: The visible light image, near-infrared image, and non-standard visible light image are processed using the reflectance calibration image group, and the formula used to calculate the reflectance image to be corrected is:

[0071] ;

[0072] in, For the original image, For the corresponding dark field image, For the corresponding whiteboard image, The calculated reflectance image is shown below. This operation normalizes the image pixel values ​​to the relative reflectance range of [0,1], effectively eliminating the effects of system fixed-mode noise and uneven illumination. After this step, the visible light reflectance image to be corrected, the near-infrared reflectance image, and k non-standard visible light reflectance images to be corrected are obtained.

[0073] Next, polarization difference correction is performed on the visible light reflectance image to be corrected and the non-standard visible light reflectance image to be corrected using a polarization difference image group to obtain the visible light reflectance image and the non-standard visible light reflectance image. This processing is specifically for images in the visible light band to suppress specular reflection. First, the visible light reflectance image to be corrected and each non-standard visible light reflectance image to be corrected are processed according to the reflectance calibration method described above to obtain the corresponding parallel polarization reflectance image and vertical polarization reflectance image. Then, the polarization difference algorithm is used for correction. The logic of polarization difference correction is as follows: calculate the difference image. The differential image is then contrast-enhanced. Finally, the corrected reflectance image is obtained using one of the following formulas:

[0074] Formula A: ;

[0075] Formula B: ;

[0076] Formula C: ;

[0077] in, For reflectance images, This is a parallel polarization reflectance image. This is a vertically polarized reflectance image. These are the weight coefficients for the corresponding items. This is a correction item.

[0078] Regarding the formula above The specific value is not a fixed preset value, but needs to be determined through experimental calibration. The method for determining the value involves collecting a set of seed sample images with known surface characteristics (including smooth and rough surfaces), and manually identifying areas in the images severely affected by specular reflection. The optimization objective is to effectively suppress the specular reflection component in this area after correction, while preserving diffuse texture information to the greatest extent possible. The optimal value is then fitted onto the experimental data using a grid search method or optimization algorithm (such as least squares). The intensity of specular reflection is closely related to seed surface roughness, water content, and illumination; therefore, a fixed coefficient is difficult to apply to all situations. By calibrating on a representative sample set, an optimal compromise coefficient can be obtained for the current seed type and imaging system, ensuring the overall correction effect is optimal and guaranteeing the stability of subsequent feature extraction. Typically, these coefficients range from 0 to 1, with initial adjustments starting from 0.5. Formula A above embodies a linear subtraction model, assuming that the specular reflection component in the image is proportional to the polarization difference value. The correction process involves directly subtracting the estimated specular reflection component from the original image. The formula is intuitive and simple, with minimal computational cost. However, when When it is very large, it may cause Negative values ​​require truncation (e.g., setting to zero), which may introduce errors. Formula B is a non-linear ratio model based on a physical assumption that the reflectivity of the original image is exaggerated in areas of strong specular reflection. This formula uses a... The relevant denominator is used to compress the values ​​of these highlighted areas. is a normalization constant to prevent the denominator from being zero. It is usually a very small integer, or related to the average reflectance of the image. This controls the intensity of compression. This formula effectively suppresses highlights, preventing negative values ​​and better reflecting the non-linear characteristics of light intensity saturation in certain situations. However, it may over-compress in some cases, leading to loss of detail in highlight areas. It is more suitable for images with very strong specular reflection and near-saturated pixel areas. Formula C is a linear attenuation model, which can be seen as a variant of Formula A. It treats the correction process as applying an attenuation to the original image related to the specular reflection intensity. When... When the specular reflection is high (strong), the attenuation factor is small, thus reducing the brightness of that pixel. This formula is also simple and intuitive; by designing the attenuation factor, it can be naturally ensured that the result will not be negative (provided that...). The value makes (It should not be less than 0). This is similar to formula A, but we want to avoid negative values.

[0079] Step 2: Extract visible light image features and near-infrared image features based on visible light reflectance images and near-infrared reflectance images, and merge them to construct a multispectral feature vector;

[0080] In this embodiment, constructing the multispectral feature vector specifically includes:

[0081] The visible light image features include color features and texture features. The color features are hue and saturation, and the texture features are GLCM contrast. The near-infrared image features include transmittance and LBP entropy.

[0082] The visible light reflectance image is converted to the HSV color space. The specific conversion method is as follows: for each pixel, based on the intensity values ​​of its R, G, and B channels, the corresponding hue (H), saturation (S), and brightness (V) values ​​are calculated using standard conversion formulas. Subsequently, the image is processed using the OTSU thresholding method to initially separate the background (i.e., seed candidate regions) and the foreground, which serves as seed candidate regions. The OTSU thresholding method is an adaptive thresholding method that automatically calculates the optimal segmentation threshold by maximizing the inter-class variance between the foreground and background classes. This process does not require a preset threshold and can adapt to images under different lighting conditions. For the foreground regions initially separated by the OTSU method as seed candidate regions, the mean and standard deviation of the hue and saturation of all its pixels are calculated. These statistics are dynamically calculated based on the foreground region of the image being analyzed, rather than fixed preset values. This ensures that feature extraction is adaptive to seed images from different batches and under different imaging conditions. Using these statistics, upper and lower thresholds for the hue and saturation of the seed regions are defined, where the lower hue threshold... Hue upper limit threshold , The mean value of hue. The standard deviation of hue, and the lower limit threshold of saturation. Saturation upper limit threshold , The mean value of saturation. The standard deviation of saturation is used here; the mean ± standard deviation is chosen as the threshold range based on statistical principles, assuming that the hue and saturation values ​​of the foreground (seed) region roughly follow a normal distribution. This range (i.e., Theoretically, this can cover approximately 95.4% of the data points. This setting is designed to accommodate normal fluctuations in seed surface color while effectively excluding background or noise pixels that significantly differ from the seed color characteristics, thereby achieving accurate extraction of the seed region. It also ensures that the hue values ​​in the foreground region fall within the range of... , Within the range, and the saturation value falls within... The connected region consisting of pixels within the specified range is determined as the final seed region.

[0083] The clustering of healthy seed surfaces in a specific color space is analyzed using an adaptive thresholding method based on data distribution, which naturally excludes damaged parts and foreign objects with abnormal color characteristics. The core idea is that in the HSV color space, due to the granulation process, the surface material and color of seeds are relatively uniform. Most pixels in intact seed areas exhibit a compact clustering distribution in both hue and saturation dimensions. By calculating the mean and standard deviation of hue and saturation for all pixels in the foreground region (i.e., the seed candidate region), the approximate range and density of this main cluster in the color feature space are depicted under data-driven conditions. Subsequently, the threshold range is set to plus or minus two standard deviations of the mean. Pixels whose hue or saturation values ​​fall outside this range have significantly deviated from the normal clustering of the seed body, and therefore have a very high probability of not belonging to the intact seed surface. They may be due to exposed internal tissues from damage, attached highly reflective impurities, or dark mold spots, etc.

[0084] The mean hue and mean saturation of all pixels within the seed region are calculated as the color features of the seed to be analyzed. The seed region of the visible light reflectance image is then converted to a grayscale image using a weighted formula. To match the sensitivity of the human eye to different wavelengths of light, the gray-level co-occurrence matrix (GLCM) is calculated. The GLCM is a method for describing texture by studying the correlation characteristics of gray space. In this embodiment, the calculation parameters of the GLCM are set as follows: step size (d) is 1 pixel, and the directions are selected as 0°, 45°, 90°, and 135°. The average value of the GLCM in each direction is taken as the final GLCM, which serves as the texture feature. The contrast is further calculated using the calculated GLCM. Contrast reflects the sharpness of the image and the depth of the texture grooves; the deeper the texture grooves, the greater the contrast and the clearer the visual effect. This calculated GLCM contrast is used as the texture feature of the seed to be analyzed. Based on the precisely determined position coordinates of the seed region in the visible light reflectance image, the same seed region is located and extracted in the spatially aligned near-infrared reflectance image. This ensures that the visible light and near-infrared features originate from the same physical location of the seed. In the seed region of the near-infrared reflectance image, the transmittance is calculated based on the reflectance value of the red channel. The transmittance is the average reflectance value of the red channel in the seed region of the near-infrared image. The red channel is chosen because in near-infrared imaging, red light in a specific band is highly correlated with information such as the internal structure and moisture content of the seed. Its reflectance characteristics can indirectly reflect the physical state of the seed, such as its integrity or potential damage. Local binary mode analysis is performed on the seed region in the near-infrared reflectance image. LBP is an operator that describes the local texture features of an image. In this embodiment, the basic LBP operator is used, that is, in a 3x3 neighborhood, the gray value of the center pixel is used as a threshold, and it is compared with the gray values ​​of 8 neighboring pixels. If the neighboring pixel value is greater than or equal to the center pixel value, the position is marked as 1, otherwise it is marked as 0, thus generating an 8-bit binary mode. A statistical histogram of the LBP values ​​of all pixels in the seed region is calculated. Then, the information entropy, i.e., LBP entropy, is calculated based on this LBP histogram; the formula for calculating LBP entropy is z. ,in It represents the probability of the k-th LBP pattern appearing in the histogram. LBP entropy reflects the complexity and irregularity of the seed surface texture. Damaged or abnormal areas often lead to changes in texture complexity, thus affecting the entropy value.

[0085] Visible light image features and near-infrared image features are concatenated to construct a multispectral feature vector, represented as follows: ,in, This is a multispectral feature vector, where H represents hue and S represents saturation. Transmittance, For GLCM contrast, Let LBP be the entropy.

[0086] By fusing color and texture information under visible light with transmission characteristics and texture complexity information under near-infrared light, this multispectral feature vector can more comprehensively and accurately characterize the performance and internal physical properties of sweet potato monoembryonic seeds, providing robust and highly discriminative feature inputs for subsequent seed damage state classification.

[0087] Step 3: Combine each non-standard visible light reflectance image and its corresponding light source direction vector to calculate the normal vector of each pixel in the seed region; generate an initial three-dimensional height map by performing field integration on the normal vector to extract the abnormal judgment features of the seed surface, and mark the suspected damaged area according to the abnormal judgment features and the preset judgment conditions.

[0088] In this embodiment, generating the initial three-dimensional height map specifically includes:

[0089] First, construct the light source direction matrix, as follows:

[0090] For each light source that records its zenith angle and azimuth angle, the formula for calculating its three-dimensional unit direction column vector is: ;in Let i be the zenith angle. Let be the azimuth angle, and i be the index of the light source. Transpose the k column vectors calculated above to obtain k 1×3 row vectors. Stack these row vectors as rows to form the light source direction matrix. .

[0091] Based on k visible light reflectance images, a photometric stereo vision algorithm is used to calculate the unit normal vector of each pixel within the seed region, specifically as follows: For each pixel within the seed region, the pixel brightness value at the same location in the k visible light reflectance images is extracted to form a k-dimensional observation vector. This vector is then simplified based on the reflectance function of the Lambertian surface, resulting in:

[0092] ;

[0093] in, Represents the observation vector. For surface reflectivity, For unit normal vector, The light source direction matrix is ​​k×3. This represents the three-dimensional row vector of the first light source. This represents the three-dimensional row vector of the second light source. This represents the three-dimensional row vector of the k-th light source;

[0094] In practical applications, to meet the solution conditions of the photometric stereo vision algorithm, the value of k should be no less than 3 to ensure that the light source direction matrix L has sufficient rank to solve for the normal vector. Preferably, the value of k is in the range of 4 to 6. Increasing the number of light sources can enhance the robustness of the system, reduce noise interference, and improve the accuracy of normal vector estimation.

[0095] By solving the following overdetermined system of equations, the least-squares solutions for the normal vector and albedo are obtained: During the solution process, if the matrix To improve the stability of the solution, a regularization term or numerical stabilization method such as singular value decomposition can be introduced to approximate singularity. Subsequently, the unit normal vector is obtained after normalization, and the surface gradient is then extracted from the normal vector field using the following relation: , ,in, The component of the unit normal vector on the X-axis in three-dimensional space. The component of the unit normal vector on the Y-axis in three-dimensional space. Let p be the component of the unit normal vector on the Z-axis in three-dimensional space, p be the partial derivative of the height in the X direction, and q be the partial derivative of the height in the Y direction.

[0096] Then, by solving the following Poisson equation and integrating the gradient field, an initial 3D height map is generated, as shown in the following formula:

[0097] ;

[0098] in, For the Laplace operator, Let (x, y) represent the height of the pixel (x, y) in 3D space, and (p, q) represent the gradient field. , Let (x, y) be the pixel position. In the discrete image domain, this Poisson equation is usually solved efficiently using discrete cosine transform or iterative methods (such as Gauss-Seidel iteration). Appropriate boundary conditions need to be set during the solution process, such as von Neumann boundary conditions, which assume that the height gradient at the boundary is zero.

[0099] In this embodiment, marking the suspected damaged area specifically includes:

[0100] The anomaly detection features include Gaussian curvature, average curvature, and local height difference. Gaussian curvature reflects the inherent bending degree of the surface at a given point; for areas with severe local deformation such as damage or cracks, the Gaussian curvature typically changes significantly. Average curvature reflects the average bending degree of the surface at a given point and can be used to identify anomalies such as depressions or protrusions. Local height difference calculates the absolute value of the height difference between each pixel and its neighboring pixels, reflecting the local roughness or abrupt changes in the surface. Based on the initial 3D height map, the first and second partial derivatives of each pixel are calculated. In discrete calculations, these derivatives can be approximated using image gradient operators such as the Sobel operator, Scharr operator, or the central difference method. Then, the Gaussian curvature and average curvature of each pixel are calculated using differential geometry formulas, as follows:

[0101] ;

[0102] ;

[0103] Where K is Gaussian curvature, a measure describing the intrinsic geometric properties of a surface at a point (i.e., properties independent of how the surface is embedded in three-dimensional space). It reflects the degree of local curvature of the surface at that point and is a geometric invariant independent of the choice of coordinate system. In seed surface topography analysis, Gaussian curvature is used to identify drastic local deformations that cannot be achieved through simple bending. K > 0 indicates that the point has ellipsoidal features (locally peaks or valleys), K = 0 indicates that the point has cylindrical or planar features (locally flattenable, such as saddle ridges or planes), and K < 0 indicates that the point has hyperboloid features (locally saddle-shaped). H is mean curvature, a measure describing the extrinsic geometric properties of a surface at a point (i.e., related to how the surface is embedded in three-dimensional space). It reflects the tendency of the surface to bend average toward its normal vector at that point and is a direct indicator of surface concavity and convexity. H > 0 indicates that the surface bulges outward on average at that point (such as the outer side of a sphere), H = 0 indicates a minimal curved surface (such as a soap film), and H < 0 indicates that the surface is concave inward on average at that point (such as the inner side of a sphere). , indicating altitude The rate of change of the slope (p) in the X direction. It quantifies the degree of curvature of the surface along the X direction. For example, a bulge in the X direction would cause... It is a positive value. , indicating altitude The rate of change of the slope (q) in the Y direction. It quantifies the degree of curvature of the surface along the Y direction. , indicating altitude The rate of change of the slope (p) in the X direction with respect to the Y direction, or the rate of change of the slope (q) in the Y direction with respect to the X direction (according to Clairaut's theorem, the two are equal). It quantifies the degree of bending coupling of the surface in two orthogonal directions and is a key parameter for identifying complex morphologies such as saddle points.

[0104] Simultaneously, the absolute value of the height difference between each pixel and its neighboring pixels in the initial 3D height map is calculated, and the average of all absolute values ​​is taken as the local height difference. The calculation formula is as follows:

[0105] ;

[0106] in, This represents the local height difference of a pixel (x, y), where i is the index of the pixel in the neighborhood, used to traverse all pixels within that neighborhood window (e.g., a 3×3 or 5×5 window); and N is the number of neighboring pixels. It represents the height of the i-th pixel in the neighborhood of pixel (x, y);

[0107] The preset judgment conditions include geometric anomaly conditions and spectral verification conditions. The geometric anomaly conditions are as follows: ,in, Let (x, y) be the Gaussian curvature of the pixel. Let be the average curvature of pixel (x, y). This is the mean of the absolute values ​​of the height differences between a pixel (x, y) and its neighboring pixels. Represents logical OR, , , These represent the preset Gaussian curvature threshold, average curvature threshold, and height difference threshold, respectively.

[0108] In damage detection, a complete, smooth seed surface typically exhibits a gently varying curvature value. However, a damaged area (such as a crack, notch, dent, or spall) introduces drastic deformation into a localized region. This deformation causes abrupt changes in the region's height (captured by local height differences), and its curvature pattern deviates significantly from that of a normal region. For example, extremely high average and Gaussian curvatures are observed at the edges of cracks, and specific combinations of curvature are produced at the bottom and edges of pits formed by spalling. Therefore, by setting a threshold... By detecting these abnormal curvature extreme points, potential damage areas can be effectively located.

[0109] This is the Gaussian curvature threshold, used to determine whether there is severe, non-developable curvature in a localized area of ​​the surface. Its specific value can be determined by statistically analyzing the Gaussian curvature distribution of a large number of known normal seed samples. For example, the 99th percentile of the Gaussian curvature distribution of normal samples can be used as the threshold. .generally, The value can range from 0.1. Up to 1.0 The specific result depends on the smoothness of the seed surface and the imaging resolution. This is the average curvature threshold, used to determine the overall unevenness or irregularity of a surface. Its specific value can also be determined by statistically analyzing the absolute value distribution of the average curvature of normal samples, for example, by taking its 99th percentile. Typically, The value can range from 0.05. Up to 0.5 . This is a local height difference threshold used to determine whether there are steep height changes on the surface. Its specific value can be set based on the average roughness of a normal seed surface or the maximum allowable height difference. Typically, The value can range from 5% to 15% of the average seed diameter.

[0110] The spectral verification conditions are as follows: ,in, This represents the transmittance of a pixel (x, y). This represents the average transmittance of the neighboring pixels of pixel (x, y). This represents the standard deviation of transmittance among the neighboring pixels of pixel (x, y). In this embodiment, a local window, for example, 5×5 or 7×7 pixels, is defined centered on the currently selected pixel. The window size is an odd integer to ensure central symmetry. The mean transmittance and standard deviation of transmittance are calculated within this local window. This is an adjustable parameter used to control the sensitivity of spectral anomaly detection. The value of determines the strictness of the judgment on whether it is an outlier. The higher the value, the stricter the conditions, the fewer suspected damage points are detected, and the risk of missed detection may increase. The smaller the value, the more lenient the conditions, the more points are detected, and the risk of false positives may increase. The specific optimization values ​​can be determined through ROC curve analysis on a validation set containing known broken and normal seeds, aiming to achieve the optimal balance point. Typically, The value ranges from 1.5 to 3.0.

[0111] Pixels that simultaneously meet both geometric anomaly and spectral verification conditions are marked as suspected damaged points, and other pixels are marked as normal points. A binary image is output, with the pixel values ​​of suspected damaged points set to 1 and the pixel values ​​of normal points set to 0.

[0112] In this embodiment, relevant data values ​​from 30 data points were collected to demonstrate the process of identifying suspected damage points. During the identification process, the Gaussian curvature threshold was set to 0.5. The average curvature threshold is 0.3. The height difference threshold is 0.02 mm, and the spectral sensitivity parameter is set to 2. Specific data are shown in the table below:

[0113] Table 1: Relevant Data for Identifying Suspected Damage Points

[0114] The table above clearly illustrates the process by which these 30 sets of sample data points were identified as suspected damage points. The local window size used in calculating the mean and standard deviation of local transmittance was 7×7 pixels. Choosing this medium size allows for robust estimation of the local background statistical characteristics in most seed damage detection scenarios. Geometric outliers in the table indicate whether they are marked as outliers after geometric anomaly assessment, while spectral outliers indicate whether they are marked as outliers after spectral verification assessment.

[0115] Morphological operations are performed on the obtained binary image, such as using small structuring elements (e.g., 3×3 circles or squares) for closing operations to fill small holes, connect adjacent damaged areas, and smooth boundaries. Subsequently, 8-neighborhood connected component analysis is performed on the binary image, grouping all spatially adjacent pixels (connected by top, bottom, left, right, and four diagonals) with a pixel value of 1 into the same connected component. Each connected component serves as an independent candidate for a suspected damaged area. Based on a preset minimum damaged area threshold, candidate suspected damaged areas smaller than the threshold are eliminated. The minimum damaged area threshold should be set considering the smallest identifiable damaged size in practical applications and avoid a large number of tiny false areas caused by image noise. For example, the minimum damaged area can be set to 0.01% to 0.1% of the total image pixels, or converted according to physical size, such as corresponding to an actual size less than 0.1. Areas that were initially suspected of being damaged were eliminated. Finally, the remaining potential damaged areas were selected as suspected damaged areas.

[0116] Step 4: Calculate the edge sharpness index of each suspected damaged area, remove the areas determined to be mechanically damaged, and output the final damaged area and its three-dimensional contour data;

[0117] In this embodiment, determining whether it is mechanical damage specifically includes:

[0118] N edge points are collected on the edge contour of each suspected damaged area. The value of N should be sufficient to reflect the characteristics of the entire edge contour, usually set to 50 to 200 points. In practice, sampling can be performed along the edge contour at equal arc lengths or equal pixel distances to ensure the uniformity and representativeness of the sampling. Based on the initial 3D height map, the height gradient magnitude of each edge point is calculated using the following formula:

[0119] ;

[0120] in, This represents the magnitude of the height gradient at pixel (x, y). These represent the partial derivatives in the x and y directions, respectively, which are the rates of change of height at that point. In practical numerical calculations, discrete differential operators such as the Sobel operator, Prewitt operator, or central difference method are typically used to approximate these two partial derivatives. For example, using a 3×3 Sobel operator template to perform convolution operations in the neighborhood of the pixel (x, y) can yield approximate values ​​of the height gradient in the X and Y directions at that point. It combines the changes in height in two orthogonal directions. The larger the value, the more drastic the height change at the edge point, and the sharper the edge in three-dimensional space.

[0121] Calculate the average height gradient magnitude of all edge points in each suspected damaged area. and maximum value This is used as an indicator of edge sharpness. The average value reflects the overall average steepness of the edge of the damaged area, while the maximum value reflects the steepness of the sharpest part of the edge. Based on the established mechanical damage judgment criteria, it is determined whether the damage is mechanical. The mechanical damage judgment criteria include: Condition 1: The average height gradient amplitude of all edge points is greater than the preset threshold for the average height gradient amplitude of normal damage. Condition 2: The maximum value of the height gradient magnitude at all edge points is greater than the preset threshold for the maximum value of the height gradient magnitude of normal damage. If any one of the judgment conditions is met, the suspected damaged area is determined to be mechanical damage. and The setting of these parameters is crucial for accurately distinguishing between mechanical damage and natural breakage. Their values ​​depend on the measurement accuracy of the 3D heightmap, the unit (e.g., micrometers / pixel), and the physical properties of the sweet potato cake surface itself. The determination process includes: collecting a representative set of sweet potato cake samples containing known categories (explicitly labeled as natural breakage, mechanical breakage, and intact); performing 3D scanning and the processing described in this method on these samples; and calculating the natural breakage region and mechanical damage region for each sample. and Draw separate maps for areas of natural damage and mechanical damage. and The distribution histogram or scatter plot. Observe the two types of regions. and Divisibility in dimensions. It should be set for natural damage. and the upper limit of distribution and mechanical damage Between the lower bound of the distribution, a value can usually be chosen that minimizes the sum of the misclassification rates of natural damage (classifying natural damage as mechanical damage) and mechanical damage (classifying mechanical damage as natural damage). Similarly, The principles of determination and The same applies to the distribution differences regarding the maximum value feature. The performance of the selected threshold is tested on independent validation sets, and fine-tuned based on the validation results. This value is chosen because mechanical damage is typically caused by scratches or impacts from sharp objects, resulting in steep, cliff-like three-dimensional structures at the edges, causing abrupt changes in height at the edges. and The values ​​are generally higher. Natural damage (such as natural cracking or peeling of the seed coat) typically has gentler edges and more subtle height variations. and The values ​​are relatively low. For example, assuming the data unit of the 3D height map is micrometers, the above statistical analysis may reveal more than 95% of naturally damaged areas. Less than 15 micrometers per pixel Less than 40 micrometers per pixel; and more than 80% of the mechanically damaged area Higher than 15 micrometers / pixel or Higher than 40 micrometers per pixel. Therefore, and Initially, these thresholds can be set to 15 micrometers per pixel and 40 micrometers per pixel, respectively. In practical applications, these two thresholds may need to be adjusted adaptively based on different seed varieties and the precision of the imaging system.

[0122] Finally, areas identified as having mechanical damage are removed from the suspected damage area, and the remaining suspected damage areas are output as the final damage areas. The corresponding 3D contour data for each final damage area is also output. The 3D contour data consists of the set of 3D coordinate points in the initial 3D height map corresponding to each final damage area. This 3D contour data not only contains the planar shape and location of the damage area but also accurately records its depth information, providing a reliable data foundation for subsequent accurate calculation of the damage area (especially on curved seed surfaces), damage volume estimation, and deeper damage morphology analysis.

[0123] Step 5: Based on the 3D contour data, calculate the maximum value of the total damaged area percentage, total absolute volume loss, and volume loss rate of the final damaged area for each seed, and combine them with the multispectral feature vector as input data to construct a damage probability prediction model and determine the damage probability of the seed to be analyzed.

[0124] In this embodiment, the three-dimensional contour data includes the coordinates and height of all pixels within the region. First, based on the binary image of the seed to be analyzed, the number of pixels in each final damaged region is counted to calculate the area percentage, using the following formula:

[0125] ;

[0126] in, This represents the area percentage of the j-th ultimately damaged region within the entire seed to be analyzed. Let be the projected area of ​​the j-th final damaged region on the binary image, which is the number of pixels in that final damaged region. The total projected area occupied by the seed to be analyzed in the binary image is denoted as the number of pixels of the seed to be analyzed in the binary image, and j is the index of the final damaged area.

[0127] In the formula for calculating the area ratio This reflects the relative size of a single damaged area within the total seed surface area, quantifying the two-dimensional spatial influence of the damaged area as a percentage. This indicator objectively assesses the extent to which damage encroaches on the seed surface area, providing an intuitive geometric basis for judging the severity of damage. Independent variable and There is a direct proportional relationship between them; the larger the projected area of ​​the damaged region, the higher its proportion in the total surface area of ​​the seed; while the denominator... As a normalization factor, this ensures consistency in damage assessment results for seeds of different sizes. This allows the area percentage index to reflect both the absolute size of the damaged area and eliminate the influence of individual seed size differences on the assessment results. As the projected area of ​​the damaged region increases, the numerator value increases accordingly, leading to an increase in the area percentage. A linear increase indicates a more severe degree of damage; conversely, when the damaged area decreases, the proportion decreases accordingly. And if the total projected area of ​​the seed... Changes will have a reverse effect on the percentage value; that is, under the same damaged area, the smaller the seed, the larger its area percentage value.

[0128] The arithmetic mean of the heights of all pixels in all final damaged areas is calculated as the reference height. This reference height is set based on the fact that damaged areas typically exhibit concave characteristics; by calculating the average height within the damaged areas, a relative reference plane can be established, thereby accurately calculating the defect volume.

[0129] The physical area of ​​a single pixel is obtained by using camera calibration parameters. The height of each pixel is compared with a reference height to calculate the absolute volume defect, as shown in the following formula:

[0130] ;

[0131] in, This represents the absolute volume loss of the j-th ultimately damaged region. Let R(j) be the physical area of ​​a single pixel, and let R(j) represent the set of pixels within the j-th final damaged region. As the reference height, This represents the height of the pixel (x, y).

[0132] The dependent variable in the formula for calculating absolute volume loss This parameter reflects the amount of physical space missing from the j-th ultimately damaged area relative to the intact seed surface. Its technical advantage lies in its ability to accurately quantify the actual volume of material lost from the seed surface due to damage, dents, or missing parts. This parameter combines two-dimensional damage area with three-dimensional depth information, transforming the visually observed degree of damage into precisely measurable volume loss data. This provides a more accurate and robust geometric feature input for damage probability prediction than simple area measurement, enabling the model to distinguish between large but shallow scratches and small but deep pits, significantly improving its ability to represent different types of damage. Independent variable It directly depicts the depth of the depression at each pixel location. The reference height represents the theoretical height reference surface of the damaged area when it is not damaged, while the actual height... A value below this reference plane indicates a material defect at that location. The calculation sums the differences across all pixels within the region; essentially, it integrates over the entire concave space of the damaged area based on spatial discrete sampling, representing the physical area of ​​a single pixel. The area of ​​each pixel, acting as an integral infinitesimal element, transforms depth information in a two-dimensional plane into volume information in three-dimensional space. Therefore, these independent variables collectively determine the magnitude of the absolute volume defect represented by the dependent variable. As the difference between the height of each pixel and the reference height increases, it means the depression at that pixel is more significant, and its corresponding contribution to the volume defect also increases. Therefore, when the damaged area contains more pixels with greater depth, or when the depression of existing pixels deepens further, accumulating these increased differences and multiplying them by a fixed pixel area will result in the dependent variable, i.e., the absolute volume defect, showing a clear increasing trend.

[0133] The physical area of ​​a single pixel is obtained through camera calibration parameters. The specific calibration method includes: using a standard calibration board, acquiring an image of the calibration board, and determining the actual physical size of each pixel in the image through corner detection and coordinate system transformation. This calibration process needs to be performed under the same imaging conditions as the seed acquisition to ensure measurement accuracy.

[0134] Based on the initial 3D height map, the total volume of the seed to be analyzed is calculated. Specifically, the reference height is considered as the base height of the seed, and the total volume of the seed surface above the base height is calculated using the following formula:

[0135] ;

[0136] in, Let E represent the total volume of the seed to be analyzed, and let E represent the set of all pixels occupied by the seed in the binary image. This represents the lowest point height in the entire seed region.

[0137] Calculate the total volume of the seeds to be analyzed At that time, the reference plane is set to the lowest point height of the entire seed region. This data iterates through the height values ​​of all pixels within a seed region defined by the initial 3D height map. The minimum value among them is This calculation method defines the three-dimensional solid volume of the seed as starting from its lowest contact point on the supporting plane (corresponding to...). This establishes a unified and objective physical benchmark by determining the spatial occupancy of each point on the surface. Compared to using internal reference surfaces such as the average height of the damaged area, this approach... Using this benchmark eliminates basis deviations introduced by different seeds or different placement orientations of the same seed, calculating the absolute and true physical volume of the seed. This makes subsequent calculations of volume loss rate more accurate. It can more accurately reflect the severity of material loss caused by damage relative to the overall size of the seed, enhancing the physical consistency and comparability of the assessment results.

[0138] Based on the absolute volume loss and the total seed volume, the volume loss rate of each ultimately damaged area is calculated using the following formula:

[0139] ;

[0140] in, Let be the volume loss rate of the j-th final damaged region.

[0141] The dependent variable in the formula for calculating volume loss rate Specifically, this reflects the percentage of volume loss caused by the j-th final damaged area relative to the total seed volume. This relative quantitative indicator effectively eliminates the impact of size differences between individual seeds on the evaluation results. By normalizing the absolute volume loss to a proportion relative to the total seed volume, this parameter achieves a standardized measurement of the degree of damage to seeds of different sizes. Its technical effect is to make damage assessment results comparable between seeds of different sizes, providing a reliable quantitative basis for establishing a unified damage judgment standard, and significantly improving the accuracy and consistency of damage detection. In this calculation formula, when the total seed volume remains constant, the volume loss rate shows a linear increasing trend as the absolute volume loss increases, indicating that the degree of damage intensifies; conversely, when the absolute volume loss remains unchanged while the total seed volume increases, the volume loss rate decreases accordingly, indicating that the same amount of damage has a relatively smaller impact on larger seeds.

[0142] In this embodiment, constructing the damage probability prediction model specifically includes:

[0143] The area percentage and absolute volume loss of all ultimately damaged areas are summed to obtain the total damaged area percentage and total absolute volume loss of seeds; the maximum volume loss rate of all ultimately damaged areas is determined; the total damaged area percentage, total absolute volume loss of seeds, and the maximum volume loss rate of all ultimately damaged areas are combined with the multispectral feature vector to obtain the combined feature vector.

[0144] The construction process of the combined feature vector specifically includes: first, extracting three geometric feature parameters for each final damaged region, namely, area percentage, absolute volume defect, and volume loss rate; then, summarizing these features to calculate the overall features at the seed level, including: the sum of the area percentages of all damaged regions, the sum of the absolute volume defects of all damaged regions, and the maximum value of the volume loss rate of all damaged regions; finally, concatenating these overall geometric features with the multispectral feature vector to form the final combined feature vector used for model training.

[0145] The multispectral feature vector is obtained by extracting the reflectance or absorptivity characteristics of seeds at different wavelengths in a multispectral imaging system. Specifically, it includes spectral response values ​​at specific wavelengths such as the visible light band and the near-infrared band. These features can reflect the material properties of the seed surface, humidity distribution, and other physicochemical properties. Combined with geometric features, they can more comprehensively describe the damage state of the seed.

[0146] Based on the above method, a set of seed samples with known damage status was collected, and the combined feature vector of each seed sample was obtained. An expert scoring method was then used to independently assess the damage of each seed sample based on its physical condition. The specific implementation of the expert scoring method is as follows: multiple experienced seed quality assessment experts were invited to independently evaluate each seed sample. During the evaluation, factors such as the damaged area, depth, and location were comprehensively considered through visual observation and tactile inspection, resulting in a damage probability score between 0 and 1, where 0 represents no damage and 1 represents complete damage. For example, a seed with only minor surface scratches received a score of approximately 0.1; a seed with small chips at the edge received a score of approximately 0.35; a seed with obvious cracks in the embryo region received a score of approximately 0.7; and a severely broken seed received a score higher than 0.8. Finally, the average score from multiple experts was used as the training label for that seed sample.

[0147] Based on the logistic regression model, the combined feature vector of the sample seeds is used as input, and its corresponding damage probability value is used as the label for model training to construct a damage probability prediction model. During model training, the optimal values ​​of each weight coefficient are determined using the maximum likelihood estimation method, so that the model prediction results are as close as possible to the expert scores.

[0148] During model training, it is necessary to determine the specific values ​​of several weight coefficients. The method for determining these weight coefficients is as follows: First, collect a sufficient number of sample seeds (usually no less than 200 seeds) to obtain the combined feature vector and corresponding expert rating label for each seed. Then, use optimization algorithms such as gradient descent or Newton's method to minimize the difference between the model's predicted value and the expert rating, and determine the optimal value of each weight coefficient through iterative optimization. The specific value of the weight coefficients depends on the feature distribution of the training dataset; different varieties or batches of seeds may require retraining to obtain suitable weight coefficients.

[0149] The combined feature vector of the seed to be analyzed is input into the trained damage probability prediction model. The damage probability value output by the model is compared with the preset damage threshold to determine whether the seed to be analyzed is damaged.

[0150] The method for determining the breakage threshold is as follows: After model training, the model performance is evaluated using validation set seed samples. The optimal threshold point is selected as the breakage threshold by plotting an ROC curve and calculating the Youden index. Specifically, the true positive rate and false positive rate are first calculated at different thresholds. Then, the Youden index (true positive rate + true negative rate - 1) corresponding to each threshold is calculated, and the threshold with the largest Youden index is selected as the final breakage threshold. This method can control the false detection rate while ensuring detection sensitivity, achieving the best discrimination effect. The breakage threshold typically ranges from 0.3 to 0.7, with the specific value depending on the requirements of the actual application scenario for detection accuracy and recall. For seed quality detection scenarios with high requirements, the threshold can be appropriately increased to between 0.6 and 0.7 to reduce false detections; for preliminary screening scenarios, the threshold can be reduced to between 0.3 and 0.5 to ensure a high detection rate.

[0151] The number of damaged seeds in a batch of seeds to be analyzed is counted, and their proportion in the batch is calculated. This proportion is then used as the damage rate of the batch of seeds to be analyzed. The damage rate is calculated using the formula: Damage rate = (Number of damaged seeds / Total number of seeds tested) × 100%. This indicator objectively reflects the quality status of the entire batch of seeds and provides an important reference for seed production and quality control.

[0152] To ensure the reliability of the detection results, the method also includes a model validation step: the model is periodically validated using standard samples with known damage states. When the model performance degrades beyond a preset range (e.g., accuracy drops by more than 5%), training samples need to be recollected and the model parameters updated. This measure ensures the long-term stable operation of the detection system and adapts to natural changes in seed characteristics. Furthermore, the method considers the differences in characteristics among different seed varieties. For different types of sweet monoembryo pelleted seeds, dedicated prediction models need to be established separately to ensure the accuracy of the detection results.

[0153] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0154] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0155] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0156] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A machine vision-based sugar beet embryo pelletized seed breakage rate detection method, characterized by, The specific steps include: Step 1: Collect visible light images and near-infrared images of seeds to be analyzed under standard light, and non-standard visible light images under k light sources, and process the above images based on the reflectance calibration image set and the polarization difference correction image set to obtain visible light reflectance images, near-infrared reflectance images, and k non-standard visible light reflectance images; Step 2: Extract visible light image features and near-infrared image features based on the visible light reflectance images and the near-infrared reflectance images and combine them to construct a multispectral feature vector; Step 3: Calculate the normal vector of each pixel point in the seed region in combination with each non-standard visible light reflectance image and its corresponding light source direction vector; generate an initial three-dimensional height map through normal vector field integration operation, thereby extracting abnormal judgment features of the seed surface, and marking out suspected damage areas according to the abnormal judgment features and a preset judgment condition; Step 4: Calculate the edge sharpness index of each suspected damage area, exclude the areas determined as mechanical damage, and output the final damage area and its three-dimensional contour data; Step 5: Calculate the total damage area ratio, the total absolute volume loss, and the maximum value of the volume loss rate of the final damage area of each seed according to the three-dimensional contour data, and combine them with the multispectral feature vector as input data to construct a damage probability prediction model to determine the damage probability of the seeds to be analyzed.

2. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 1, characterized in that, The reflectance calibration image set includes a whiteboard image and a dark field image, and the polarization difference correction image set includes a parallel polarized light image and a vertical polarized light image; Wherein, the acquisition logic of the parallel polarized light image and the vertical polarized light image is: under the condition of standard light equipped with a polarizing mirror, the parallel polarized light image and the vertical polarized light image of the seeds to be analyzed are collected respectively to jointly constitute the polarization difference correction image set; Wherein, the acquisition logic of the visible light image, the near-infrared image, and the non-standard visible light image is: using the same pixel-level RGB color camera and monochrome near-infrared camera, the visible light image and the near-infrared image of the seeds to be analyzed are synchronously collected under standard light conditions, and the two images are image-registered to be transformed to the same coordinate system, wherein the standard light condition is a uniform light condition, and then k different and known direction light sources are controlled to be lit in turn to collect k non-standard visible light images of the seeds to be analyzed under different light conditions, thereby constituting a non-standard visible light image set for generating an initial three-dimensional height map; Wherein, the logic of processing the above images based on the reflectance calibration image set and the polarization difference correction image set is: processing the visible light image, the near-infrared image, and the non-standard visible light image through the reflectance calibration image set to obtain the corrected visible light reflectance image, the near-infrared reflectance image, and the corrected non-standard visible light reflectance image, and then performing polarization difference correction on the corrected visible light reflectance image and the corrected non-standard visible light reflectance image through the polarization difference image set to obtain the visible light reflectance image and the non-standard visible light reflectance image.

3. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 1, characterized in that, The multispectral feature vector is constructed specifically by: The visible light image features include color features and texture features, the color features are hue and saturation, and the texture features are GLCM contrast, and the near-infrared image features include transmittance and LBP entropy; The visible light reflectance image is converted to HSV color space and processed based on OTSU threshold method to preliminarily separate the background and the foreground as a seed candidate region from the image; for the preliminarily separated foreground region, the mean value and the standard deviation of the hue and the saturation of all pixel points thereof are calculated to define the upper limit threshold and the lower limit threshold of the hue and the saturation of the seed region, wherein the hue lower limit threshold , , is the mean value of the hue, is the standard deviation of the hue, the saturation lower limit threshold , , is the mean value of the saturation, is the standard deviation of the saturation, the region composed of the pixel points with the hue value falling within the range of , ] and the saturation value falling within the range of is determined as the final seed region; the mean value of the hue and the mean value of the saturation of all pixel points in the seed region are calculated to serve as the color feature of the seed to be analyzed, and the seed region of the visible light reflectance image is converted to a gray scale image and the gray scale co-occurrence matrix thereof is calculated, and the GLCM contrast of the seed region is calculated through the gray scale co-occurrence matrix to serve as the texture feature; Based on the seed region position in the visible light reflectance image, the seed region is determined in the near-infrared reflectance image, the transmittance is calculated based on the red channel reflectance in the seed region of the near-infrared reflectance image, and the seed region in the image is locally binarized to calculate the LBP entropy; The visible light image features and the near-infrared image features are spliced to construct a multispectral feature vector, which is represented as wherein, is the multispectral feature vector, H is the hue, and S is the saturation, is the transmittance, is the GLCM contrast, is the LBP entropy.

4. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 2, characterized in that, The generating of the initial three-dimensional height map specifically comprises: Based on the k visible light reflectance images, the unit normal vector of each pixel point in the seed region is calculated by using photometric stereo algorithm, specifically as follows: for each pixel point in the seed region, the pixel brightness values of the same position in the k visible light reflectance images are extracted to form a k-dimensional observation vector, the reflection function of a Lambert surface is simplified to obtain: ； wherein, represents an observation vector, is a surface reflectance, is a unit normal vector, is a k x 3 light direction matrix, represents a three-dimensional row vector of a first light source, represents a three-dimensional row vector of a second light source, represents a three-dimensional row vector of a kth light source; By solving the following overdetermined equation system, i.e. , to obtain the least square solution of normal vector and albedo, and then by normalization to obtain the unit normal vector, and further by the relationship formula to extract the surface gradient from the normal vector field, the specific relationship formula is , , wherein, is the component of the unit normal vector on the X-axis in three-dimensional space, is the component of the unit normal vector on the Y-axis in three-dimensional space, is the component of the unit normal vector on the Z-axis in three-dimensional space, p represents the partial derivative of height in the X direction, and q represents the partial derivative of height in the Y direction; Then, the gradient field is integrated by solving the following Poisson equation to generate the initial three-dimensional height map, and the formula is as follows: ； wherein represents the height of the pixel point (x, y) in the three-dimensional space, (p, q) represents the gradient field, and (x, y) is the pixel point position.

5. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 3, characterized in that, The marking of the suspected damage area specifically comprises: The abnormality judgment features include Gaussian curvature, average curvature and local height difference; based on the initial three-dimensional height map, the first-order partial derivative and the second-order partial derivative of each pixel point are calculated, and then the Gaussian curvature and the average curvature of each pixel point are calculated according to the differential geometry formula; at the same time, the absolute value of the height difference of each pixel point and the pixel points in its neighborhood in the initial three-dimensional height map is calculated; The preset judging conditions include a geometric abnormal condition and a spectrum verification condition, and the geometric abnormal condition is: wherein, is a Gaussian curvature of the pixel point (x, y), is an average curvature of the pixel point (x, y), is a mean value of absolute values of height differences between the pixel point (x, y) and its neighborhood pixel points, represents a logical OR, , , respectively represent preset Gaussian curvature threshold values, average curvature threshold values and height difference threshold values. The spectral verification condition is: wherein, T(x,y) denotes the transmittance of the pixel point (x,y), T(x,y) denotes the mean of the transmittance of the neighboring pixel points of the pixel point (x,y), T(x,y) denotes the standard deviation of the transmittance of the neighboring pixel points of the pixel point (x,y), is an adjustable parameter; The pixel points that meet the geometric abnormality condition and the spectral verification condition at the same time are marked as suspected damage points, and other pixel points are marked as normal points, and a binary image is output, the pixel value of the suspected damage point is set to 1, and the pixel value of the normal point is set to 0; An 8-neighborhood scanning is performed on the binary image, all spatially adjacent pixel points with a value of 1 are merged into the same connected domain, each connected domain is taken as an independent suspected damage area candidate, based on a preset minimum damage area threshold, the suspected damage area candidates smaller than the minimum damage area threshold are removed, and the remaining suspected damage area candidates are taken as suspected damage areas.

6. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 5, characterized in that, The determination of whether it is mechanical damage specifically comprises: N edge points are collected on the edge profile of each suspected damage area, based on the initial three-dimensional height map, the height gradient amplitude of each edge point is calculated, and the formula is as follows: ； wherein, represents the height gradient amplitude of the pixel point (x, y); The average value and the maximum value of the height gradient amplitude of all edge points of each suspected damage area are calculated, and are taken as edge sharpness indicators; according to the set mechanical damage judgment condition, it is determined whether it is mechanical damage, the mechanical damage judgment condition includes: condition one: the average value of the height gradient amplitude of all edge points is greater than a preset average value threshold of the height gradient amplitude of the normal damage, condition two: the maximum value of the height gradient amplitude of all edge points is greater than a preset maximum value threshold of the height gradient amplitude of the normal damage; if one of the judgment conditions is met, it is determined that the suspected damage area is mechanical damage; And the region determined as mechanical damage is removed from the suspected damage area, the remaining suspected damage area is output as the final damage area, and the three-dimensional profile data corresponding to each final damage area is output.

7. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 5, characterized in that, The three-dimensional profile data includes the coordinates and height of all pixel points in the region. According to the binary image of the seed to be analyzed, the number of pixel points in each final damage area is counted to calculate the area ratio, and the formula is as follows: ； wherein, represents the area proportion of the jth final damaged area in the whole seed to be analyzed, is the projection area of the jth final damaged area on the binary image, that is, the number of pixel points of the final damaged area, is the total projection area of the seed to be analyzed in the binary image, that is, the number of pixel points of the seed to be analyzed in the binary image, and j is the index of the final damaged area. The arithmetic mean of the height of all pixel points in all final damage areas is calculated as the reference height. The physical area of a single pixel is obtained through camera calibration parameters, and the height of each pixel point is compared with the reference height to calculate the absolute volume loss, and the specific formula is as follows: ； wherein, represents the absolute volume loss of the jth final breakage zone, is the physical area of a single pixel, R(j) represents the set of pixel points within the jth final breakage zone, is the reference height, represents the height of the pixel point (x, y); Based on the initial three-dimensional height map, the total volume of the seed to be analyzed is calculated. Specifically, the reference height is regarded as the base height of the seed, and the volume of the seed surface above the base height is summed up to calculate the total volume, and the formula is as follows: ； wherein, is the total volume of the seeds to be analyzed, E represents a set of all pixel points occupied by the seeds to be analyzed in the binary image, is the lowest point height of the entire seed region; According to the absolute volume loss and the total volume of the seed, the volume loss rate of each final damage area is calculated, and the formula is as follows: ； wherein, Vj is the volume loss rate for the jth final fracture zone.

8. A machine vision-based sugar beet embryo pelleting seed breakage rate detection method according to claim 7, characterized in that, The construction of the damage probability prediction model specifically includes: Sum the area ratio and absolute volume loss of all final damage areas to obtain the total damage area ratio and the total absolute volume loss of the seed. Determine the maximum value of the volume loss rate of all final damage areas. Combine the total damage area ratio, the total absolute volume loss of the seed, and the maximum value of the volume loss rate of all final damage areas with the multi-spectral feature vector to obtain a combined feature vector. Based on the above method, a group of sample seeds with known damage states are collected, and the combined feature vector of each sample seed is obtained. According to the sample seeds, each sample seed is independently evaluated for damage by expert scoring method, and the damage probability value of each sample seed is determined as the training label. Based on the logistic regression model, the combined feature vector of the group of sample seeds is input as the input, and the corresponding damage probability value is taken as the label for model training to construct a damage probability prediction model. The combined feature vector of the seed to be analyzed is input into the trained damage probability prediction model, and the damage probability value output by the model is compared with the preset damage threshold to determine whether the seed to be analyzed is damaged. The number of seeds determined to be damaged in a group of seeds to be analyzed is counted, and the proportion of the seeds in the group is calculated and taken as the damage rate of the group of seeds to be analyzed.

Images