Fruit quality detection method and system based on multi-spectral structured light

Abstract

The application relates to the technical field of fruit quality detection, and discloses a fruit quality detection method and system based on a multi-spectrum structured light, which first collects a structured light image sequence and a spectral intensity image of a fruit to be detected, and calculates an initial truncated phase field; then performs inverse compensation of a subsurface scattering phase on the initial truncated phase field by using the spectral intensity image, and reconstructs a unit normal vector field of a surface of the fruit to be detected through unwrapping; then performs physical inverse compensation on the spectral intensity image by using the unit normal vector field and an illumination direction vector, and decouples to obtain a real spectral reflectance distribution; and finally fuses geometric and spectral biochemical features to output a comprehensive grading result of the fruit to be detected. The application corrects the phase by calculating a gray absorption term and a gradient scattering term of the image, effectively solves the phase drift problem caused by the penetration of light on the translucent fruit skin, eliminates the depth measurement error caused by the light diffusion in the medium, and further improves the fruit detection precision.

Description

Technical Field

[0001] This invention relates to the field of fruit quality testing technology, specifically to a fruit quality testing method and system based on multispectral structured light. Background Technology

[0002] In modern fruit post-harvest processing production lines, accurately assessing the external geometry (such as size and deformity) and biochemical quality of the peel (such as sugar content, bruising, and mold) of fruits is crucial for achieving automated grading. Currently, machine vision technology is mainly used to replace manual sorting. Among these technologies, structured light 3D reconstruction technology, due to its non-contact and high precision characteristics, is often used to obtain geometric morphological information of fruits, while multispectral imaging technology is used to analyze the biochemical properties of fruit surfaces.

[0003] However, existing detection technologies have significant limitations when dealing with specific objects like fruits. First, traditional structured light reconstruction algorithms are typically based on the assumption that the surface of the fruit is opaque and diffusely reflective. However, the skin and flesh of most fruits (such as apples, pears, and peaches) are translucent. When structured light stripes are projected onto the fruit surface, the light penetrates the skin and enters the flesh, causing subsurface scattering. This results in reduced contrast and positional shift of the stripe signal acquired by the imaging end. This physical phenomenon means that the calculated phase information cannot accurately reflect the physical outer surface of the fruit, instead forming a virtual surface that is concave inwards, directly reducing the geometric accuracy of the 3D reconstruction.

[0004] Secondly, fruits typically have spherical or irregularly curved surfaces, resulting in significant differences in the angle between the normal direction and the optical axis of the imaging system across different areas of their surface. When acquiring spectral or intensity images, the central region of the fruit is brighter due to Lambert's cosine law and Fresnel reflection effects, while the brightness at the edges decreases sharply. Existing processing methods often confuse this brightness unevenness with the fruit's inherent texture or use only simple two-dimensional image enhancement techniques, failing to isolate the influence of geometry on illumination from a physical optics perspective. This leads to deviations in determining the true reflectance of the fruit peel, easily misjudging normal dark areas at the edges as spectral defects, or overlooking early, minute damage located at areas of high curvature at the edges, making it difficult to maintain consistent quality inspection standards across the entire fruit surface. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a fruit quality detection method and system based on multispectral structured light, which solves the problems of low three-dimensional reconstruction accuracy caused by light penetration when detecting translucent fruits, and distortion in spectral reflectance extraction caused by surface light attenuation due to the curved shape of the fruit.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a fruit quality detection method based on multispectral structured light, comprising the following steps:

[0007] Acquire structured light image sequences and spectral intensity images of the surface of the fruit to be tested;

[0008] The initial truncated phase field and modulation depth are calculated using the structured light image sequence;

[0009] The initial truncated phase field is subjected to subsurface scattering phase inverse compensation using the spectral intensity image to obtain the truncated phase field. The truncated phase field is then unwrapped based on the modulation depth to reconstruct the unit normal vector field of the surface of the fruit to be tested.

[0010] Using the unit normal vector field and the illumination direction vector, physical inverse compensation is performed on the spectral intensity image to decouple and obtain the true spectral reflectance distribution;

[0011] Geometric features are extracted based on the unit normal vector field, and spectral biochemical features are extracted based on the true spectral reflectance distribution. The comprehensive grading result of the fruit to be tested is then output in combination with the preset grading threshold.

[0012] Preferably, before acquiring the structured light image sequence and spectral intensity image of the surface of the fruit to be tested, a calibration step is performed: the translation vector of the projection unit relative to the imaging unit is calculated using a stereo calibration algorithm, the spatial coordinates of the center of the measurement field of view where the fruit to be tested is located are set, the vector from the optical center of the projection unit to the spatial coordinates of the center of the measurement field of view is calculated, and the vector is normalized to obtain the illumination direction vector as a global illumination direction constant.

[0013] Preferably, the steps of acquiring the structured light image sequence and spectral intensity image of the surface of the fruit to be tested specifically include: controlling the projection unit to project a phase-shifted sinusoidal fringe pattern onto the surface of the fruit to be tested, and the imaging unit acquiring the structured light image to form a structured light image sequence; then controlling the projection unit to switch to a uniform illumination mode, and the imaging unit acquiring the diffuse reflectance image of the surface of the fruit to be tested covering the visible light and near-infrared regions under the same exposure time and gain settings as the spectral intensity image, which corresponds to the DC component in the structured light image.

[0014] Preferably, the subsurface scattering phase inverse compensation step specifically involves: constructing a phase drift compensation model based on the Beer-Lambert law, calculating the logarithmic term of the pixel gray value in the spectral intensity image (absorption depth compensation component) and the spatial gradient magnitude (lateral scattering compensation component), weighted summing to obtain a phase drift estimate, and subtracting the phase drift estimate from the initial truncated phase field to obtain the truncated phase field.

[0015] Preferably, the reconstruction process further includes the following steps: generating an effective binary mask based on the lower threshold of modulation depth and the upper threshold of saturation intensity; establishing a local search window for invalid pixels within the effective binary mask; and using a Gaussian weighting function combining the phase value of the effective pixels within the local search window with the spatial distance to perform weighted neighborhood interpolation repair on the invalid pixels.

[0016] Preferably, the reconstruction step includes: calculating the phase gradient components of the unwrapped truncated phase field, mapping the phase gradient components to geometric gradients using a sensitivity coefficient; constructing a normal vector based on differential geometry principles and geometric gradients, and performing Euclidean normalization to obtain a unit normal vector field.

[0017] Preferably, the physical inverse compensation steps include: calculating the dot product of the unit normal field and the illumination direction vector to obtain the local incident angle cosine distribution; constructing a geometric attenuation correction factor using the Schlick approximation algorithm and setting an illumination threshold constant to truncate and protect the geometric attenuation correction factor; and using the geometric attenuation correction factor to normalize the spectral intensity image and decouple it to obtain the true spectral reflectance distribution.

[0018] Preferably, the steps for extracting geometric features and spectral biochemical features include: performing morphological erosion operations on the effective binary mask to generate the region of interest; calculating the average curvature field of the region of interest using the divergence operator of the unit normal vector field, and statistically analyzing the mean and standard deviation of the average curvature field as geometric features; and calculating the spectral difference response value based on the true reflectance distribution to generate spectral biochemical features.

[0019] Preferably, the steps for outputting the comprehensive grading result include: setting a geometric defect binarization criterion based on the geometric morphology features and statistical principles; generating a spectral defect criterion based on spectral biochemical features combined with an adaptive segmentation threshold; performing a logical OR operation on the geometric defect binarization criterion and the spectral defect criterion to obtain the comprehensive defect area ratio of the surface of the fruit to be tested; calculating the ratio of the average reflectance of the region of interest at the characteristic wavelength to the reference wavelength to obtain the spectral feature index; comparing the comprehensive defect area ratio and the spectral feature index with the maximum allowable defect area threshold and the quality spectral index threshold, respectively; determining the fruit grade (second-grade fruit, premium fruit, standard fruit, or processed fruit) based on the comparison results to obtain the comprehensive grading result.

[0020] The fruit quality detection system based on multispectral structured light includes a projection unit, an imaging unit, and a processor. The processor includes:

[0021] The calibration processing module is configured to calculate the relative spatial position of the imaging unit and the projection unit, and to calculate the illumination direction vector based on the relative spatial position;

[0022] The acquisition and control module is configured to control the imaging unit to acquire structured light image sequences and synchronized spectral intensity images;

[0023] The phase calculation module is configured to calculate the initial truncated phase field and modulation depth;

[0024] The geometric reconstruction module is configured to use the spectral intensity image to perform subsurface scattering phase inverse compensation on the initial truncated phase field to obtain the truncated phase field, and then unwrap the truncated phase field by the modulation depth to reconstruct the unit normal vector field.

[0025] The physical compensation module is configured to perform physical inverse compensation on the spectral intensity image by combining the illumination direction vector and the unit normal vector field, thereby decoupling and obtaining the true spectral reflectance distribution.

[0026] The comprehensive grading module is configured to output comprehensive grading results based on geometric morphology features and spectral biochemical features.

[0027] This invention provides a method and system for fruit quality detection based on multispectral structured light. It has the following beneficial effects:

[0028] 1. This invention solves the phase drift problem caused by the penetration of structured light stripe signals into the interior caused by the translucent fruit peel by using the spectral intensity image to perform subsurface scattering phase inverse compensation of the initial truncated phase field. The phase is corrected by calculating the gray-scale absorption term and gradient scattering term of the spectral intensity image, eliminating the depth measurement error caused by the diffusion of light in the fruit pulp medium. This allows the reconstructed unit normal vector field to accurately represent the real physical outer surface of the fruit, thereby improving the geometric shape restoration accuracy of translucent fruits such as apples and pears.

[0029] 2. This invention utilizes the reconstructed unit normal vector field and the calibrated illumination direction vector to perform physical inverse compensation on the spectral intensity image, solving the problem of uneven surface illumination caused by the curved shape of fruit. By calculating the local incident angle cosine distribution of the entire field of view and constructing a geometric attenuation correction factor, the brightness distribution in the spectral intensity image affected by the object shape is restored to the true spectral reflectance distribution. This achieves decoupling of geometric shape information and spectral texture information, thereby ensuring that the large curvature area at the edge of the fruit and the flat area in the center have consistent spectral response characteristics, avoiding the situation where the edge area is misjudged as a spectral defect due to natural brightness attenuation. Attached Figure Description

[0030] Figure 1 This is a flowchart of the fruit quality detection method based on multispectral structured light according to the present invention;

[0031] Figure 2 This is a schematic diagram of the architecture of the fruit quality testing system of the present invention.

[0032] Figure 3 This is a flowchart of the data synchronization acquisition timing of the present invention;

[0033] Figure 4 This is a flowchart of the initial phase calculation and time-domain processing of the present invention;

[0034] Figure 5 This is a flowchart of the subsurface scattering phase inverse compensation process of the present invention;

[0035] Figure 6 This is a flowchart of the microscopic surface normal vector reconstruction process of the present invention;

[0036] Figure 7 This is a flowchart of the geometry-driven spectral radiometric correction process of the present invention;

[0037] Figure 8 This is a flowchart of the quality feature extraction and comprehensive grading process of the present invention;

[0038] Figure 9 This is a heatmap showing the three-dimensional reconstruction error distribution of the present invention.

[0039] Figure 10 This is a statistical chart showing the defect identification performance of the present invention;

[0040] Figure 11 This is a linear regression diagram of the spectral characteristic index and the actual sugar content value of the present invention.

[0041] Among them, 101 is the acquisition and control module; 102 is the phase calculation module; 103 is the geometric reconstruction module; 104 is the physical compensation module; 105 is the comprehensive classification module; and 106 is the calibration processing module. Detailed Implementation

[0042] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Please refer to the appendix. Figure 1 This invention provides a fruit quality detection method based on multispectral structured light, comprising the following steps:

[0044] Step 1: Acquire structured light image sequences and spectral intensity images of the surface of the fruit to be tested;

[0045] Step 2: Calculate the initial truncated phase field and modulation depth using the structured light image sequence;

[0046] Step 3: Perform subsurface scattering phase inverse compensation on the initial truncated phase field using the spectral intensity image to obtain the truncated phase field, and unwrap the truncated phase field based on the modulation depth to reconstruct the unit normal vector field of the surface of the fruit to be tested.

[0047] Step 4: Using the unit normal vector field and the illumination direction vector, perform physical inverse compensation on the spectral intensity image to decouple and obtain the true spectral reflectance distribution;

[0048] Step 5: Extract geometric features based on the unit normal vector field, extract spectral biochemical features based on the true spectral reflectance distribution, and output the comprehensive grading result of the fruit to be tested by combining the preset grading threshold.

[0049] Please refer to the appendix. Figure 2This embodiment provides a fruit quality detection system for implementing the fruit quality detection method based on multispectral structured light. The system includes a projection unit, an imaging unit, and a processor at the hardware level. The projection unit and imaging unit constitute an active vision acquisition front-end. The processor is connected to the active vision acquisition front-end via a data bus and is used to execute the core algorithm logic. Specifically, the processor includes: a calibration processing module 106, used to perform system geometric calibration, calculate the relative spatial positions of the projection unit and the imaging unit; set the spatial coordinates of the measurement field of view center according to a preset working distance; calculate the vector pointing from the optical center of the projection unit to the measurement field of view center; normalize this vector; and generate a global illumination direction vector with a constant illumination direction. An acquisition control module 101 is used to control the projection unit to first project a phase-shifted sinusoidal fringe pattern, triggering the imaging unit to synchronously acquire a structured light image sequence; then switch to a uniform illumination mode to acquire spectral intensity images under the same exposure parameters, ensuring the spatiotemporal alignment of geometric and spectral data. Phase calculation module 102: Used to process structured light image sequences using a multi-step phase-shifting algorithm to calculate the initial truncated phase field; simultaneously, it calculates the modulation depth of the stripe signal to identify low signal-to-noise ratio regions and construct an effective binary mask. Geometric reconstruction module 103: Used to estimate the phase drift caused by subsurface scattering using grayscale and gradient information from the spectral intensity image, and performs inverse subtraction correction on the initial truncated phase field; subsequently, it unwraps the corrected truncated phase field based on the modulation depth and calculates the geometric gradient based on the differential principle to reconstruct the unit normal vector field of the surface of the fruit under test. Physical compensation module 104: Used to calculate the local incident angle cosine distribution using the dot product of the illumination direction vector and the unit normal vector field, and constructs a geometric attenuation correction factor based on Fresnel reflection characteristics; using this geometric attenuation correction factor, it performs pixel-by-pixel normalization processing on the spectral intensity image, decoupling to obtain the true spectral reflectance distribution unaffected by curvature. The comprehensive grading module 105 is used to perform morphological erosion on the effective binary mask; calculate the average curvature field based on the unit normal vector field to obtain the geometric defect binarization criterion; calculate the spectral difference response value based on the true spectral reflectance distribution to obtain the spectral defect criterion; and finally compare the comprehensive defect area ratio and spectral feature index with the preset grading threshold to output the comprehensive grading result.

[0050] Please refer to the appendix. Figure 1Before detection, geometric calibration and parameter initialization are first performed to establish a unified measurement coordinate system and obtain a physical reference for illumination conditions. This reference is used to determine the relative spatial position between the imaging unit (e.g., camera) and the projection unit (e.g., illumination lamp, projector), and the optical center of the projection unit is mapped to the coordinate system of the imaging unit. Specifically, the calibration process uses a planar calibration plate with a known geometric feature array, preferably a black and white checkerboard calibration plate or a high-precision dot array calibration plate. The calibration plate is placed at multiple different positions and angles within the depth of field. At each position, the imaging unit first acquires a natural light image of the calibration plate; subsequently, the projection unit projects a preset phase-shifted fringe sequence (e.g., longitudinal and transverse phase-shifted sinusoidal fringe patterns) onto the calibration plate, while the imaging unit simultaneously acquires a structured light image modulated by the calibration plate. The calculation is performed using stereo calibration algorithms commonly used in the field (e.g., Zhang Zhengyou calibration method and its variants). For the calibration of the projection unit, the acquired structured light image is processed using a phase resolution algorithm to establish a phase mapping relationship between the pixel coordinates of the imaging unit and the pixel coordinates of the projection unit. This treats the projection unit as a camera operating in reverse, obtaining its corresponding virtual corner coordinates, and then calculating the internal parameter matrix of the projection unit. For the imaging unit, the corner coordinates are extracted from the natural light image, and its internal parameter matrix (including focal length and principal point coordinates) and distortion coefficients are calculated.

[0051] Furthermore, in another embodiment, the distortion coefficient is used to digitally characterize the nonlinear optical errors of the imaging lens, and it mainly includes two types: radial distortion coefficient and tangential distortion coefficient. The radial distortion coefficient is used to correct barrel or pincushion distortion caused by the shape of the lens itself, which bends the straight lines at the image edges. For the industrial-grade machine vision lens used in this embodiment, a polynomial model is typically used to characterize the radial distortion coefficient. The radial distortion coefficient typically ranges from -1.0 to 1.0, and decreases exponentially with increasing polynomial order. The tangential distortion coefficient is used to correct the centrifugal position deviation caused by the lens optical axis not being perfectly perpendicular to the sensor imaging plane. Due to the precision assembly process of industrial cameras, the tangential distortion coefficient is extremely small, typically ranging from -0.05 to 0.05.

[0052] Based on this, the calibration processing module 106 calculates the rotation matrix and translation vector T of the projection unit coordinate system relative to the imaging unit coordinate system using a stereo matching algorithm. The translation vector T represents the spatial position of the optical center (light source center) of the projection unit in the imaging unit coordinate system. Setting the optical center of the imaging unit as the origin (0, 0, 0), the spatial coordinates of the optical center of the projection unit (X...) are... L ,Y L Z L That is, equivalent to the components (t) of the translation vector T. x ,ty ,t z Simultaneously, the light source position parameters need to be converted into a direction vector form illuminating the fruit to be tested. Considering that within a specific detection field of view, the distance between the fruit to be tested and the light source is much greater than the surface undulations of the fruit itself, a far-field illumination model is used as an approximation. First, the spatial coordinates P of the center of the measurement field of view in the imaging unit coordinate system are set. center For P center =(0,0,Z work ), where Z work Let l be the system's working distance (i.e., the average distance from the optical center of the imaging unit to the surface of the fruit being measured). Then, the vector pointing from the optical center of the projection unit to the center of the measurement field of view is calculated and normalized to obtain the illumination direction vector l. The formula for calculating this illumination direction vector l is as follows:

[0053] ;

[0054] In the formula, l represents the normalized illumination direction vector with a magnitude of 1; (X L ,Y L Z L () represents the spatial coordinates of the optical center of the projection unit in the imaging unit coordinate system; the denominator term is... This represents the Euclidean distance between the optical center of the projection unit and the optical center of the imaging unit. Furthermore, in this embodiment, based on the far-field illumination assumption, the differences in light angularity within the field of view are ignored, and the illumination direction vector is treated as a global illumination direction constant.

[0055] Therefore, the illumination direction vector l obtained through calculation, along with the distortion coefficients of the imaging unit, can be used in subsequent real-time detection steps. This illumination direction vector l can be read as a global illumination direction constant and multiplied by the real-time reconstructed unit normal vector field to quantify the local incident angle cosine distribution. As for controlling the calibration environment, it can be performed in a laboratory with uniform ambient lighting and no strong light interference. The flatness error of the calibration plate should be better than 0.05 mm to ensure that the geometric accuracy of the parameter calculation can support subsequent micron-level topography measurements and high dynamic range spectral correction.

[0056] See attached document Figure 3After calibration, the acquisition control module 101 executes step 1, and the invention enters the data synchronization acquisition stage. By controlling the timing coordination of the projection unit and the imaging unit, a structured light image containing the geometric phase information of the fruit under test and a spectral intensity image containing the physical material information of the fruit under test are acquired in the same field of view and at the same time. Specifically, the projection unit and the imaging unit can be connected through a hardware trigger signal line (such as a TTL level signal) to establish a master-slave synchronization mechanism. Specifically, the projection unit can use spatial light modulation based on a digital micromirror device (DMD) or liquid crystal (LCD) to project a preset encoded phase-shift sinusoidal fringe pattern onto the surface of the fruit under test. The total number of phase shift steps N is set to an integer between 3 and 12, preferably N=4, to balance acquisition efficiency and phase calculation accuracy. For each phase shift projection step, the imaging unit acquires a structured light image highly modulated by the surface of the fruit under test in linear response mode, denoted as I. pat (u,v,k), the formula is expressed as:

[0057] I pat (u,v,k)=R(u,v){I amb (u,v)+I mod (u,v)cos[Φ init (u,v)+δ k ]}+D noise (u,v);

[0058] In the formula, (u,v) represents the pixel coordinates on the photosensitive plane of the imaging unit; k is the phase shift index, ranging from 0 to N-1, where N is the total number of phase shift steps; R(u,v) represents the spectral reflectance distribution of the surface of the fruit under test in the current band, and this term acts as a multiplicative factor on both the ambient background light and the projected fringe light; I amb (u,v) represents the superimposed light intensity of the ambient background light and the DC component of the projector at pixel coordinates (u,v); I mod (u,v) represents the system modulation amplitude corresponding to the projected stripe light at pixel coordinates (u,v); Φ init (u,v) represents the initial truncation phase value at pixel coordinates (u,v); δ k δ is the preset phase shift for step k. k =2πk / N;D noise (u,v) represents the dark current noise and thermal noise of the imaging sensor inside the camera.

[0059] After acquiring N structured light images, the maximum grayscale image (full white image) of the entire field of view is immediately loaded into the spatial light modulator of the projection unit, causing the projection unit to switch to uniform illumination mode. At this time, the projection unit operates as a uniform surface light source, with its spectral range covering the visible and near-infrared regions from 600nm to 1100nm. The reason for using this band is that fruit epidermal tissue exhibits subsurface scattering (SSS) characteristics in this band, and the transmission depth is sufficient to carry information about early subcutaneous damage. Meanwhile, the imaging unit, maintaining the same exposure time and gain settings as the structured light image acquisition, acquires the diffuse reflectance image of the fruit surface as the spectral intensity image, denoted as I. raw The spectral intensity image I raw Corresponding to the DC component in the above formula, i.e.: I raw ≈R(u,v)(I amb +I mod Furthermore, this spectral intensity image can record the two-dimensional texture of the fruit surface under test, and its gray-level distribution gradient can characterize the lateral scattering distance of photons inside the fruit peel, providing physical input data for subsequent phase inverse compensation based on light intensity gradient. Moreover, the entire acquisition time span is controlled within the minimum period allowed by the imaging unit frame rate (e.g., within 100ms) to eliminate pixel misalignment caused by minor vibrations of the fruit on the production line, ensuring I... pat with I raw The same physical point on the surface of the fruit to be tested corresponds to the pixel coordinates (u,v).

[0060] See attached document Figure 4 The phase calculation module 102 executes step 2, after obtaining I pat After (u,v,k), pixel-by-pixel temporal dephase operation is required. Specifically, the orthogonality of the multi-step phase-shifting algorithm is utilized to decouple the background light component, the reflectivity component of the fruit surface, and the phase component carrying geometric depth information from the temporal signal. For any pixel coordinate (u,v) on the target surface of the imaging sensor inside the camera, its grayscale value in N structured light images can be extracted to form a temporal grayscale vector. Then, the initial truncated phase value Φ of the pixel coordinate is calculated using the least squares method. init (u,v). The calculation formula is as follows:

[0061] ;

[0062] In the formula, k is the phase shift index, ranging from 0 to N-1, and N is the total number of phase shift steps; arctan2(,) is the bivariate four-quadrant arctangent function, whose return value ranges from (-π,π]. Due to the periodic truncation characteristic of the arctangent function, the calculated Φ initThe initial truncated phase field exhibits a sawtooth distribution that cycles between -π and π. Mathematically, the phase value of this initial truncated phase field corresponds to the phase delay of the sinusoidal fringes in the imaging optical path. However, physically, for translucent materials such as fruits, this phase value actually corresponds to the average optical path center after multiple scatterings of light within the fruit peel, rather than the precise location on the fruit's geometric surface. This physical phase lag caused by the material's translucency is the subsurface scattering error that needs to be corrected in subsequent steps.

[0063] Simultaneously, the present invention also calculates the modulation depth M(u,v) of the pixel coordinates (u,v). Specifically, in this embodiment, the modulation depth is the amplitude of the AC component of the received sinusoidal fringe signal, used to characterize the effective intensity of the structured light signal at the current pixel coordinates, and its calculation formula is as follows:

[0064] ;

[0065] In the formula, k is the phase shift index, and the value of M(u,v) reflects the signal-to-noise ratio of the pixel coordinate (u,v). Its physical meaning is that when the projected light shines on a surface area with high reflectivity and at the depth of focus, the reflected sinusoidal fringes have clear contrast, and the calculated value of M(u,v) is large, indicating that the initial truncation phase value Φ calculated at this time is relatively large. init M(u,v) has a high confidence level; conversely, in areas with strong ambient light interference, severe light absorption on the surface of the fruit under test (such as black spots), or areas obscured by shadows, the stripe signal is submerged in noise, and the value of M(u,v) approaches zero. Therefore, this invention uses the modulation depth M(u,v) not only to evaluate phase quality, but also as a direct input parameter for subsequently constructing geometric confidence weights.

[0066] See attached document Figure 5 In step 3, after calculating and obtaining the initial truncated phase field, the geometric reconstruction module 103 enters the subsurface scattering phase inverse compensation step based on the spectral intensity image. Using the spectral intensity physical information of the synchronously acquired spectral intensity image, the geometric phase that has drifted due to the influence of the semi-transparent material is corrected at the pixel level.

[0067] Specifically, for thin fruit peels (such as apples and cherries), when structured light stripes are projected onto the surface of the fruit, photons do not undergo simple surface reflection at the physical interface. Instead, some photons are refracted into the interior of the fruit, undergo multiple scattering and absorption processes, and then exit from the area around the incident point. This results in an incident, scattering, and exiting physical process, causing low-pass blurring in the spatial distribution of the stripe image received by the camera, and manifesting as phase lag in the phase field. In the 3D reconstruction results, this phase lag directly corresponds to the reconstructed surface shrinking inward, leading to a smaller measured geometric size than the actual size, loss of high-frequency texture details, and, due to differences in material thickness, maturity, and internal damage in different areas, the phase error caused by this scattering is spatially non-uniformly distributed.

[0068] To correct this non-uniformly distributed phase error, this embodiment performs phase compensation. Specifically, based on the Beer-Lambert law and its generalization to scattering media, the average optical path length (corresponding to the phase shift) in the medium exhibits a logarithmic attenuation characteristic with respect to the intensity of the emitted light. Simultaneously, at the edges of the medium or where the texture changes abruptly, the lateral diffusion of photons causes a change in the gradient. Based on this physical relationship, the phase shift compensation model can be expressed as:

[0069] ;

[0070] In the formula, ΔΦ sss (u,v) represents the estimated phase shift caused by subsurface scattering at pixel coordinates (u,v); For logarithmic decay term, I raw (u,v,λ) represents the grayscale value of pixel coordinates (u,v) in the spectral intensity image at a specific wavelength λ, wherein the specific wavelength λ is preferably 850nm±10nm; max I represents the theoretical saturation grayscale value of the camera at the current bit depth (e.g., for an 8-bit image). max =255); It is a small positive real number (typically taking values ​​in the range of 10). -6 Up to 10 -3 This is used to avoid invalid calculations due to a zero denominator or a zero logarithmic argument; This represents the spatial gradient magnitude of the spectral intensity image; It is a two-dimensional discrete difference operator. In specific implementation, a 5×5 Scharr operator is used, which has pairs of I in the horizontal u direction and the vertical v direction respectively. raw Perform convolution operations and calculate the gradient components G. u and G v Then, the spatial gradient magnitude can be obtained: ;k absThis is the absorption depth compensation coefficient, which is related to the absorption coefficient μ of the medium. a and reduced scattering coefficient μ s 'Positive correlation' characterizes the weight of light intensity attenuation on depth measurement; k scat The lateral scattering compensation coefficient represents the weight of the effect of edge gradient changes on phase blurring.

[0071] Regarding the coefficient k abs and k scat The determination method is as follows: A sample of the same variety and similar maturity as the fruit to be tested is selected as the calibration material. First, a high-precision contact measurement device (such as a coordinate measuring machine, CMM) or a high-precision optical scan after opaque spraying treatment is used to obtain the true three-dimensional point cloud of the sample surface, and then converted into the true phase distribution field Φ. true Then, using the initial truncated phase field Φ init and spectral intensity image I raw Construct the error objective function, which is expressed by the following formula:

[0072] ;

[0073] In the formula, E(k) abs ,k scat ) represents the objective function for parameter optimization; ΔΦ sss (k abs ,k scat ) indicates substituting the current iteration parameter k abs With k scat The phase drift estimate obtained after calculation is compensated by the aforementioned (logarithmic term). With gradient term k scat ||(∇I raw (u,v,λ))||)Dynamically generated;

[0074] Then, the error objective function is iteratively solved using the least squares method or the Levenberg-Marquardt optimization algorithm. When the error E converges to the preset residual threshold, the optimal solution obtained is the fixed parameter k applicable to this batch of fruits to be tested. abs and k scat .

[0075] At this point, the initial truncated phase field can be corrected, as expressed by the formula:

[0076] Φ corr (u,v)=Φ init (u,v)-ΔΦ sss (u,v);

[0077] In the formula, Φ corr(u,v) represents the phase value of pixel coordinates (u,v) in the corrected truncated phase field; thus, the corrected truncated phase field Φ corr This allows the virtual subsurface of the fruit to be pulled back to its actual physical outer surface. Subsequently, the invention can perform spatial phase unwrapping. A quality-map-guided path integral algorithm is used, where the quality map is the modulation depth M and the spectral intensity image I in this step. raw The normalized product is formed, and the unwrapping path prioritizes traversing high-quality regions, thus unfolding the truncated phase field into a spatially continuous continuous phase field Φ. It should be noted that if the calculation result exceeds the interval (-π, π], it needs to be rewrapped, that is, added or subtracted by an integer multiple of 2π, so that its value returns to the principal value interval, in order to ensure the input validity of subsequent unwrapping algorithms.

[0078] Furthermore, embodiments of the present invention also utilize the modulation depth M and the spectral intensity image I raw To identify and repair singular points in the continuous phase field Φ, specifically in optical measurements of fruit surfaces, the reliability of phase distribution data is limited by two physical boundary conditions: first, low signal-to-noise ratio regions, such as backlit edges of fruit or black rotten spots, where reflected light is weak, modulation depth M is extremely low, and the calculated phase value is dominated by random noise; second, high light saturation regions, such as specular reflection points on smooth fruit peel surfaces, where light intensity exceeds the camera's range, the top of the sine fringes is truncated, leading to phase calculation failure. Therefore, to simultaneously eliminate these two types of anomalous data, a pixel-level effective binary mask Mask is first calculated. This mask defines the reliable area of ​​phase data in the entire field of view, and the formula for calculating the effective binary mask value Mask(u,v) corresponding to pixel coordinates (u,v) can be:

[0079] ;

[0080] In the formula, This is a logical indicator function. It takes the value 1 when the condition within the parentheses is true, and 0 otherwise. When Mask(u,v)=1, it indicates that the pixel at coordinates (u,v) is a valid pixel; when Mask(u,v)=0, it indicates that the pixel at coordinates (u,v) is an invalid pixel that needs repair. mod The lower limit threshold for modulation depth is determined by an adaptive histogram statistical method. Specifically, the gray-level histogram of the modulation depth M of the current frame is calculated, and the first trough position between the low gray-level area (background noise area) and the high gray-level area (effective signal area) of the histogram is found. The corresponding gray-level value is set as T. mod In one implementation, for 8-bit deep data, T mod It is usually between 5 and 15, where T is in the formula. satThe upper limit threshold for saturation light intensity is determined based on the upper limit of the camera's linear response range and is used to identify pixels with charge overflow. For 8-bit quantized images, T... sat Set to 250 to 254 to ensure that all pixels in the non-linear saturation region are removed.

[0081] Secondly, based on the calculated mask, weighted neighborhood interpolation is performed on the continuous phase field Φ. Specifically, this involves: traversing all invalid pixels marked as Mask(u,v)=0, establishing a local search window Ω of size (2w+1)×(2w+1) centered on this invalid pixel (where w is the window radius, typically w=2), and then using all valid pixels within the local search window (i.e., the neighboring pixels of Mask=1) to perform weighted reconstruction on the center pixel. The calculation formula is as follows:

[0082] ;

[0083] In the formula, Φ(u',v') represents the original, unrepaired (or unwrapped) continuous phase value at the neighboring pixel (u',v'); Mask(u',v') is the valid binary mask value corresponding to the neighboring pixel (u',v'). When the valid binary mask value is 1, it indicates that the neighboring pixel is a valid pixel; when the valid binary mask value is 0, it indicates that the neighboring pixel is an invalid pixel (not included in the weighted calculation); Φ repaired (u,v) represents the repaired continuous phase values, and Φ repaired (u,v) can be used as the cutoff phase value Φ(u,v) of the cutoff phase field to be solved; (u',v') are the local coordinates within the local search window; e is a small constant to prevent the denominator from being zero; W(u',v') is a Gaussian weighting function based on spatial distance, expressed as W(u',v')=exp(-((u'-u)). 2 +(v'-v) 2 ) / (2σ 2 In this process, σ is the Gaussian standard deviation, which is set to half the window radius. If there are no valid pixels in the local search window Ω (i.e., the denominator is zero), the local search window radius is gradually expanded until it contains at least 3 valid pixels, or the pixel of the invalid pixel is directly marked as a permanent invalid background. Therefore, after this process, noise and specular holes in the phase distribution can be eliminated, and the drastic change in the normal vector direction caused by noise can be prevented.

[0084] See attached document Figure 6 Furthermore, in obtaining Φ repairedAfter (u,v), the geometric reconstruction module 103 further performs reconstruction calculations of the microscopic surface normal vectors to extract the local geometric orientation information of the fruit surface from the truncated phase field and construct a pixel-by-pixel unit normal vector field. This unit normal vector field can not only represent the uneven texture of the fruit surface but also provide input variables for subsequent geometrically driven spectral radiometric correction. Furthermore, unlike the traditional indirect calculation path of phase, depth point cloud, meshing, and normal vectors, this embodiment adopts a calculation path of phase gradient, geometric gradient, and normal vectors, which avoids the accumulated error in the phase-to-depth integration process and the smoothing loss in the meshing process, thereby preserving the small microscopic defect features of the fruit skin.

[0085] Considering that differential operations are sensitive to high-frequency noise, before calculating the gradient, an isotropic Gaussian filter with a 5×5 window is first applied to Φ. repaired (u,v) undergoes a slight smoothing process to suppress random speckle noise. At this time, the phase field Φ repaired Phase gradient component Φ u and Φ v The calculation formula is as follows:

[0086] ;

[0087] ;

[0088] In the formula; Φ u (u,v) and Φ v (u,v) represent Φ respectively. repaired (u,v) represents the phase gradient components along the horizontal (u-axis) and vertical (v-axis) directions of the image at pixel coordinates (u,v). and Φ(u+1,v) and Φ(u-1,v) represent the gradient components of the phase field Φ in the pixel coordinate system, in radians per pixel; Φ(u+1,v) and Φ(u-1,v) represent the unwrapped continuous phase values ​​at the right and left adjacent pixels of the current pixel coordinate (u,v), respectively; Φ(u,v+1) and Φ(u,v-1) represent the unwrapped continuous phase values ​​at the lower and upper adjacent pixels of the current pixel coordinate (u,v), respectively.

[0089] For edge pixels in the image, the forward or backward differencing operator is automatically switched to ensure that the gradient field calculation covers the entire image area. Secondly, a physical mapping relationship is established between the phase gradient and the geometric gradient of the fruit surface under test. This is specifically based on the structured light triangulation principle, where the phase change and the change in object height have an approximately linear relationship, specifically defined as the sensitivity coefficient S. geo(Unit: mm / radian), and this coefficient characterizes the physical height change corresponding to a unit phase change. Its value is determined by the baseline distance between the projector and the camera, the working distance, and the fringe frequency. Therefore, the geometric gradient g of the surface of the fruit under test in the physical spatial coordinate system (X,Y,Z) is... x and g y The calculation formula is:

[0090] g x (u,v)=(S geo Φ u (u,v)) / Δd;

[0091] g y (u,v)=(S geo Φ v (u,v)) / Δd;

[0092] In the formula, g x (u,v) and g y (u,v) represent the slopes of the tangents along the horizontal and vertical directions at the coordinates (u,v) on the surface of the fruit to be measured; parameter S geo To obtain a constant through calibration of a standard plane, in practice, this is achieved by measuring the phase difference ΔΦ of steps with known heights. step The difference between the actual height and ΔZ step The ratio is determined, i.e., S geo =ΔZ step / ΔΦ step Δd represents the lateral spatial resolution, which indicates the actual physical size represented by a pixel of the imaging unit (camera) on the focal plane of the object under test. It can be obtained through the calibration process (e.g., photographing a checkerboard calibration board of known size and calculating the ratio of the physical distance between adjacent corner points to the pixel distance), or calculated based on the camera pixel size, lens focal length, and working distance (i.e., Δd≈p). size D work / f, where D work where f is the working distance, p is the focal length, and f is the focal length. size (This refers to the physical size of a pixel).

[0093] Finally, based on the principles of differential geometry, the unit normal vector n(u,v) is calculated using the geometric gradient. Specifically, since the normal vector at any point on the surface of the fruit to be measured is perpendicular to the tangent plane at that point, a non-normalized normal vector can be constructed, expressed by the formula: n raw =(-g x ,-g y 1). Then, perform Euclidean normalization on it to obtain the unit normal vector n(u,v), which is expressed by the formula:

[0094] ;

[0095] In the formula, As a normalization factor, it ensures that the magnitude of the output vector is 1. The calculated n is a three-dimensional vector field image that satisfies ||n||=1. The vector direction of each pixel coordinate corresponds to the spatial orientation of the micro-element on the surface of the fruit to be tested. For the surface of the fruit to be tested, the normal vector changes smoothly and continuously in the normal arc area, while in the dents, dents or scars, the normal vector will deflect sharply in the local neighborhood.

[0096] See attached document Figure 7 In step 4, after obtaining the unit normal vector field n, the physical compensation module 104 proceeds to the physical inverse compensation stage. Specifically, the unit normal vector field n is used to adjust the acquired spectral intensity image I. raw The uneven illumination in the light is physically reverse-compensated at the pixel level, thereby decoupling the true material reflectivity distribution of the surface of the fruit under test.

[0097] For spherical or irregularly curved fruits (such as apples, oranges, cherries, or pears), under unidirectional illumination, the surface light intensity follows the cosine radiation law. As the angle between the surface normal and the incident light direction increases, the luminous flux received per unit surface area decreases cosinely, resulting in a brightness distribution in the spectral intensity image that is brighter at the center and darker at the edges. This luminous intensity attenuation caused by geometric curvature can severely confuse defect detection algorithms based on grayscale thresholds and introduce nonlinear spectral reflectance errors. Therefore, compensation is necessary. Specifically: First, calculate the local incident angle cosine distribution across the entire field of view. For each pixel coordinate (u, v) in the image space, by calling the aforementioned illumination direction vector l and the real-time reconstructed unit normal vector n(u, v), calculate the cosine value C of the incident angle θ according to vector algebra principles. inc (u,v), its formula can be expressed as:

[0098] ;

[0099] In the formula, This represents the vector dot product operation. (C) inc The numerical range of (u,v) is [-1, 1]. In actual calculations, for C... inc For regions where the value is less than 0 (i.e., the backlit side), set the value to 0 to indicate that the region cannot receive direct light; cosθ(u,v) represents the cosine of the incident angle θ; n x n y n z Let represent the component values ​​of the unit normal vector n(u,v) along the X, Y, and Z axes in the three-dimensional coordinate system, respectively; x , l y , l zThese represent the component values ​​of the illumination direction vector l along the X, Y, and Z axes in the three-dimensional spatial coordinate system, respectively.

[0100] Secondly, a geometrical attenuation correction factor K that conforms to the optical properties of fruit peel is constructed. corr (u,v). Considering that the skin of most fruits (such as the cuticle of apples) is not an ideal Lambertian diffuse reflector, the natural waxy layer covering its surface will produce Fresnel reflections that depend on the viewing angle and weak specular highlights, thus causing overcorrection in the edge areas (i.e., whitening). Therefore, this embodiment uses Fresnel coefficients for correction, which can be expressed by the following formula:

[0101] K corr (u,v)=C inc (u,v)[(1-k s )+k s (1-C inc (u,v)) 5 ];

[0102] In the formula, (1-C inc (u,v)) 5 This is the Fresnel reflection term in the Schlick approximation algorithm, used to simulate the enhanced reflection effect of light at the grazing angle; k s K is the skin gloss coefficient, which characterizes the degree of reflection enhancement of the fruit skin at a grazing angle. For fruits with rough surfaces (such as kiwifruit), k s The value of k approaches 0. For fruits with smooth surfaces (such as Red Delicious and cherries), k... s The value ranges from 0.1 to 0.3. Before actual deployment, k can be adjusted by photographing a standard sample of the same batch of fruit. s The value that minimizes the gray-level variance between the center and edge regions of the corrected fruit is used to determine the optimal k for this batch. s value.

[0103] Finally, the calculated geometric attenuation correction factor K is used. corr (u,v), for the spectral intensity image I raw Perform pixel-by-pixel normalization to calculate the true spectral reflectance distribution R. true True spectral reflectance R true The formula for calculating (u,v,λ) can be expressed as:

[0104] ;

[0105] In the formula, the denominator is truncated using the maximum value function max(); The illumination threshold constant is denoted by K, which ranges from 0.1 to 0.15. When the incident angle is close to 90 degrees (i.e., near the object's edge contour), the denominator K... corrApproaching zero, direct division would amplify the corrected noise infinitely, producing artifacts. Therefore, by setting... It performs full compensation only on the effective lighting area (incident angle less than about 80 degrees), while numerically suppressing the extremely dark edge areas to prevent the algorithm from diverging.

[0106] Therefore, after the above processing steps, the true spectral reflectance distribution R is obtained. true The geometric lighting shadows in the image can be removed, and in the corrected true spectral reflectance distribution, the differences in pixel values ​​are no longer affected by the surface curvature, but only reflect the true differences in the physical state of the fruit peel surface (such as browning, abrasions, and dents). This provides a physically consistent data foundation for subsequent inversion of internal quality using spectral indices or identification of surface defects using threshold segmentation.

[0107] See attached document Figure 8 The comprehensive grading module 105 executes step 5, specifically, by calculating the geometric morphology statistics and spectral absorption index, and logically comparing them with preset physical thresholds, the comprehensive grading result of the fruit to be tested is output. The specific steps are as follows: First, the validity constraint processing of the region of interest (ROI) is performed. Although a validity binary mask has been generated in the previous steps, considering that the edges of the object (i.e., areas with a viewing angle close to 90°) are easily affected by background stray light interference and that the normal vector calculation has numerical instability, the statistical region needs to be spatially shrunk. Specifically, a morphological erosion algorithm is used, defining the structuring element B as a circular structure with a diameter of 5 to 9 pixels or a rectangular structure of 5×5 pixels. Then, this structuring element is used to perform convolutional erosion operations on the validity binary mask, expressed by the formula: This is used to shrink the boundary of the effective binary mask by approximately 2-3 mm, thereby stripping away the high-noise contour loops and generating the final mask for feature statistics; where, The symbol B represents the erosion operator in mathematical morphology; B represents the structuring element used in morphological operations, the size of which is usually set according to the spatial resolution of the camera, and it is necessary to ensure that the stripped area covers the high curvature area and scattering blur area of ​​the object's edge.

[0108] In terms of geometric features, the average curvature field H of the surface is calculated based on the principles of differential geometry to detect physical dents and deformities. The surface of normal fruit exhibits a smooth manifold microscopically, with a curvature value that is a low-frequency constant; however, bruises or scars cause local normal vector divergence, resulting in high-frequency abrupt changes in curvature. The average curvature field H is calculated using the divergence operator of the unit normal vector field, as shown in the following formula:

[0109] ;

[0110] In the formula, H(u,v) is the average curvature at pixel coordinates (u,v); n x(u+1,v)-n x (u-1,v) represents the central difference of the normal vector X components in the horizontal direction (u-axis); n y (u,v+1)-n y (u,v-1) represents the central difference of the normal vector Y component in the vertical direction (v-axis); subsequently, a binary criterion D for geometric defects is constructed based on the statistical distribution of the average curvature. geo (u,v), specifically: statistics M ROI The arithmetic mean μ of the average curvature of all pixel coordinates within the region of interest. H With standard deviation σ H Then, the adaptive decision logic is set using the statistical 3σ principle, expressed by the formula:

[0111] ;

[0112] In the formula, when D geo (u,v)=1 indicates that the pixel coordinates (u,v) are a geometric defect point; k σ The defect sensitivity coefficient is set to a value range of 3.0 to 4.0. The logic for setting this range is based on the normal distribution assumption, and areas that deviate from the mean by more than a certain multiple of the standard deviation are identified as areas of physical damage.

[0113] In the spectral feature dimension, spectral difference response values ​​need to be calculated to identify surface biochemical lesions (such as browning and decay). The input data is Ra data after eliminating the illumination gradient. true (u,v,λ), and then calculate the spectral absorption index S of a single band or a combination of multiple bands. diff (u,v), and combine with adaptive segmentation threshold to generate spectral defect criterion D spec (u,v), the formula is expressed as:

[0114] ;

[0115] ;

[0116] In the formula, L represents the total number of spectral channels used by the system. For example, if it includes four channels: R (red), G (green), B (blue), and NIR (near-infrared), then L = 4; i T is the preset spectral sensitivity coefficient corresponding to the i-th spectral channel, which is determined based on the absorption characteristics of the target defect (such as browning) (for example, for browned regions, reflectivity decreases in the 550nm band, so the corresponding coefficient takes a positive value); Otsu The adaptive segmentation threshold is calculated in real time by S. diff The grayscale histogram was obtained and calculated using the Otsu's method to accommodate the differences in the basic color of different batches of fruit.

[0117] After completing pixel-level inspection, it is also necessary to calculate the overall defect area ratio R on the surface. defect The formula is expressed as:

[0118] ;

[0119] In the formula, ∨ represents a logical OR operation, meaning that if pixel coordinates (u,v) exhibit either geometric anomalies or spectral anomalies, they are counted as defect points; N ROI The total number of pixels in the region of interest is given, and a spectral characteristic index I, which characterizes internal quality (such as sweetness), is also calculated. qual Specifically, based on Beer-Lambert's law, calculations are performed using the selective absorption characteristics of matter at specific wavelengths. Specifically, M is first calculated. ROI Average spectral reflectance within the region Then, the spectral characteristic index of the characteristic band is calculated, expressed by the formula:

[0120] ;

[0121] In the formula, I qual It is a spectral characteristic index; The average reflectance at a characteristic wavelength sensitive to the target component (such as soluble solids) is selected (for example, the near-infrared band around 970nm or 880nm is selected, which is significantly affected by the absorption of sugar water molecule bonding vibrations). The average reflectance at a reference wavelength insensitive to the target components (e.g., selecting a band around 750 nm or 800 nm). When the internal sugar content of the fruit is high... The spectral characteristic index I decreases due to enhanced absorption. qual The value increases.

[0122] Finally, a comprehensive classification judgment based on multi-level thresholds is performed. Specifically, preset classification threshold parameters can be read, and the calculated comprehensive defect area ratio R is used as the basis for the classification. defect With spectral characteristic index I qual Perform a logical comparison:

[0123] If R defect >T area If it is determined to be a secondary result, the removal action is triggered.

[0124] If R defect ≤T area And I qual ≥T premium The fruit is classified as a top-grade fruit, triggering a level one exit action.

[0125] If R defect ≤T area And T standard ≤Iqual <T premium If the result is a standard fruit, a secondary export action is triggered; otherwise, the fruit is classified as a processed fruit.

[0126] In the formula, T area The maximum permissible defect area threshold (e.g., 5%); T premium With T standard These are the quality spectral index thresholds corresponding to premium quality and standard quality, respectively, and the above threshold parameters (T) premium ,T standard The method for determining the spectral characteristic index I can be as follows: First, collect a batch of representative sample fruits (e.g., 50 fruits), and then calculate their spectral characteristic index I. qual Then, the true sugar content value was measured using standard physicochemical instruments (such as a refractometer), an index-sugar content mapping table was established, and the corresponding index boundary value was found in the mapping table according to market standards (such as sugar content greater than 14 being extra grade), and set as the spectral index threshold.

[0127] In one application embodiment, this embodiment demonstrates the actual application configuration and processing flow of the above method on a Fuji apple production line. Details are as follows:

[0128] Hardware configuration and environment parameters:

[0129] Imaging Unit: Employs a 5-megapixel global shutter CMOS industrial camera equipped with a low-distortion lens with a 16mm focal length. A narrow-band filter with a center wavelength of 850nm (bandwidth ±10nm) is mounted in front of the lens to eliminate interference from ambient visible light and utilize the penetrability of the near-infrared band for early damage.

[0130] Projection unit: Employs a DMD chip-based digital projector with a radiant flux set to 3000mW and a light source wavelength adjusted to cover the near-infrared band of 800-900nm, matching the imaging unit. The optical axis angle between the imaging unit and the projection unit is set to 15°, and the X-axis component of the translation vector (baseline distance) T is set to 120mm (i.e., the projection unit is located 120mm to the right of the imaging unit). The system's standard working distance is set to 450mm.

[0131] Computing platform: Industrial control computer equipped with Intel i7 processor and NVIDIA-RTX series graphics card, running Windows 10 IoT Enterprise Edition.

[0132] Parameter initialization: Before detection, the system reads the following preset parameters: total phase shift steps N=4;

[0133] Illumination direction vector calculation: Based on the baseline distance (120mm) and working distance (450mm), a vector is constructed from the optical center of the projection unit (120,0,0) to the center of the field of view (0,0,450). After normalization, the global illumination direction vector l≈(-0.258,0,0.966) is obtained, which is used for subsequent illumination compensation.

[0134] Subsurface scattering compensation coefficient: k abs =0.35, k scat =0.12 (calibrated for the translucent characteristic of Fuji apple peel); defect sensitivity coefficient k σ =3.5;

[0135] Quality grading threshold:

[0136] Maximum permissible defect area threshold T area =3.0%, the spectral index threshold T of premium-grade fruit premium =0.85 (corresponding to a refractive sugar content of approximately 14.0°Brix).

[0137] Data processing flow:

[0138] Acquisition: The projection unit projects four phase-shifted fringe patterns within 40ms, and the camera synchronously acquires I... pat Immediately afterwards, a full white field was projected, and one spectral intensity image was acquired. raw .

[0139] Phase calculation and correction: The processor calculates the initial truncated phase field and uses I... raw The logarithmic and gradient terms are used to calculate the phase drift estimate ΔΦ. sss For example, in the translucent region at the edge of the fruit, the calculated ΔΦ sss It is approximately 0.15 radians. The corrected truncated phase field not only eliminates the edge shrinkage phenomenon, but also restores the true depth information of the tiny dents on the fruit surface (approximately 3 mm in diameter and 0.5 mm in depth).

[0140] Decoupling normal vector and reflectivity: Reconstructing the unit normal vector field n using the gradient method. At this point, for a backlit region (incident angle ≈ 75°) below the side of the fruit, the original image I... raw The grayscale value is only 45 (relatively dark), which is easily misjudged as a browning defect by traditional algorithms. However, in this embodiment, the geometric attenuation correction factor K is calculated by substituting the pre-calculated illumination direction vector. corr The reflectivity of the backlit area was compensated back to approximately 180, thus avoiding misjudgment.

[0141] Feature fusion and decision-making:

[0142] Two anomalies were detected: one was a strong curvature mutation (H>μ). H+3.5σ H The first injury was determined to be an old impact damage; the second injury had normal curvature but a spectral absorption index (S). diff The levels were significantly lower than normal, indicating early-stage moldy core disease.

[0143] Calculate the overall defect area ratio R defect =4.2%, exceeding T area (3.0%).

[0144] Output result: The apple is determined to be substandard, and a pulse signal is sent to the pneumatic rejection mechanism via the I / O card.

[0145] To further verify the practical effectiveness and reliability of the fruit quality grading method based on geometrically driven spectral correction of the present invention, this embodiment constructs a test set containing 200 Fuji apple samples for verification testing. The test set includes 50 intact fruits, 50 samples with old physical bruising (metachromatic), 50 samples with early core lesions (hyperchromatic), and 50 samples with surface water droplets or stains. Specifically, it includes:

[0146] 1. Verification of the accuracy of microscopic morphology measurement of translucent material surfaces:

[0147] To verify the effect of subsurface scattering phase inverse compensation in this invention on improving the accuracy of three-dimensional reconstruction of translucent fruit, a translucent resin ball with known geometric truth value (optical properties simulating apple peel, diameter 80.00 mm) was selected as a standard for testing.

[0148] Experimental procedure: The system of this invention was used to collect light intensity and phase data, and the corrected phase Φ was calculated using the compensation formula. corr And reconstruct the 3D point cloud.

[0149] Experimental results: See appendix Figure 9 The results show that, after physical inverse compensation, the geometric error of the reconstructed model in the high curvature region at the edge of the sphere (incident angle > 60°) is effectively controlled within ±0.1mm, and the inward shrinkage and collapse phenomenon common in translucent objects is not observed. The root mean square error (RMSE) across the entire field of view is as low as 0.08mm.

[0150] Therefore, this invention can accurately reproduce the true geometric shape of the surface of translucent fruit and effectively detect tiny depressions as shallow as 0.3 mm, thus establishing the geometric data basis for subsequent feature extraction.

[0151] 2. Comprehensive defect identification performance under complex working conditions:

[0152] For complex defects that are difficult to distinguish in the sample set (such as old wounds with no color change but with indentation, and shadow areas / brown areas with darker color but no indentation), the effectiveness of the geometric + spectral dual threshold discrimination logic is verified.

[0153] Experimental procedure: Execute the detection process and calculate the geometric defect binarization criterion D. geo With spectral defect criterion D spec And calculate the final classification accuracy.

[0154] Experimental results: See appendix Figure 10 Data shows that for old dents: thanks to the sensitive capture of local curvature abrupt changes by the microscopic unit normal vector field n, the system successfully identified 48 dent samples, achieving a recognition rate of 96.0%; for early core lesions: by introducing the correct illumination direction vector for geometrically driven illumination correction, the system effectively eliminated the interference of shadows in the dark areas of the fruit surface, accurately identifying 49 lesion samples, achieving a recognition rate of 98.0%; anti-interference capability: for 50 samples with water droplets / stains, the system's overall false alarm rate was only 1.5%. Therefore, this invention, by integrating geometric morphology and material properties, achieves precise decoupling and classification of different types of defects, possessing high robustness in complex industrial environments.

[0155] 3. Verification of the linearity of the internal quality (sweetness) spectral index:

[0156] To verify the spectral characteristic index I after geometric illumination correction qual Effectiveness in predicting internal physicochemical properties (saccharin content).

[0157] Experimental procedure: Spectral reflectance data of all samples were collected, processed by the correction algorithm of this invention, and the spectral characteristic index I was calculated. qual Subsequently, a handheld digital refractometer was used to determine the true sugar content of each sample, and correlation regression analysis was performed.

[0158] Experimental results: See appendix Figure 11 As shown, the vertical axis represents the calculated spectral characteristic index I. qual The data points are closely distributed around the fitted line, showing a positive correlation. Statistical analysis shows that the coefficient of determination (R²) between the two is high. 2 The accuracy reached 0.88, and the root mean square error of prediction (RMSEP) was 0.65°Brix.

[0159] Verification conclusion: This result demonstrates that, after eliminating the effects of uneven illumination caused by surface curvature and Fresnel reflection, the spectral characteristic index I based on the single-band ratio is... qual This can reflect the soluble solids content inside the fruit with high fidelity, proving the scientific nature and physical interpretability of the threshold determination logic in the scheme.

[0160] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fruit quality detection method based on multispectral structured light, characterized in that, Includes the following steps: Acquire structured light image sequences and spectral intensity images of the surface of the fruit to be tested; The initial truncated phase field and modulation depth are calculated using the structured light image sequence; The initial truncated phase field is subjected to subsurface scattering phase inverse compensation using the spectral intensity image to obtain the truncated phase field. The truncated phase field is then unwrapped based on the modulation depth to reconstruct the unit normal vector field of the surface of the fruit to be tested. Using the unit normal vector field and the illumination direction vector, physical inverse compensation is performed on the spectral intensity image to decouple and obtain the true spectral reflectance distribution; Geometric morphology features are extracted based on the unit normal vector field, and spectral biochemical features are extracted based on the true spectral reflectance distribution. Combined with a preset grading threshold, a comprehensive grading result for the fruit to be tested is output. The step of performing subsurface scattering phase inverse compensation on the initial truncated phase field using the spectral intensity image includes the following steps: Based on the Beer-Lambert law and its generalization in scattering media, a phase drift compensation model is constructed. The logarithmic term of the pixel grayscale value in the spectral intensity image is used as the absorption depth compensation component; The spatial gradient magnitude of the spectral intensity image is used as the lateral scattering compensation component. The absorption depth compensation component and the transverse scattering compensation component are weighted and summed using the absorption depth compensation coefficient and the transverse scattering compensation coefficient to obtain the phase drift estimate. Subtracting the phase drift estimate from the initial truncated phase field yields the corrected truncated phase field; The step of performing physical inverse compensation on the spectral intensity image using the unit normal vector field and the illumination direction vector specifically includes: Calculate the dot product of the unit normal vector field and the illumination direction vector to obtain the local incident angle cosine distribution of the entire field of view; Based on the local incident angle cosine distribution and the preset epidermal gloss coefficient, a geometric attenuation correction factor is constructed using the Schlick approximation algorithm; The spectral intensity image is normalized pixel by pixel according to the geometric attenuation correction factor to obtain the true spectral reflectance distribution; In the pixel-by-pixel normalization process, an illumination threshold constant is set to protect the geometric attenuation correction factor by truncation.

2. The fruit quality testing method according to claim 1, characterized in that, Before acquiring the structured light image sequence and spectral intensity image of the surface of the fruit to be tested, a calibration step is also included to obtain the illumination direction vector. This calibration step specifically includes: The translation vector of the projection unit coordinate system relative to the imaging unit coordinate system is calculated using a stereo calibration algorithm; Set the spatial coordinates of the center of the measurement field of view where the fruit to be measured is located; Calculate the vector of spatial coordinates from the optical center of the projection unit to the center of the measurement field of view, and normalize the vector to obtain the illumination direction vector as a global illumination direction constant.

3. The fruit quality testing method according to claim 1, characterized in that, The acquisition of the structured light image sequence and spectral intensity image of the surface of the fruit to be tested specifically includes: The control projection unit projects a pre-coded phase-shifted sinusoidal stripe pattern onto the surface of the fruit to be tested, and the control imaging unit simultaneously acquires structured light images highly modulated by the surface of the fruit to be tested, thus forming a structured light image sequence. After the structured light image sequence acquisition is completed, the projection unit is switched to uniform illumination mode, and the imaging unit is controlled to acquire the diffuse reflectance image of the surface of the fruit to be tested as the spectral intensity image while maintaining the same exposure time and gain setting. The spectral intensity image corresponds to the DC component in the structured light image, and the spectral range of the spectral intensity image covers the visible and near-infrared regions.

4. The fruit quality testing method according to claim 1, characterized in that, Before reconstructing the unit normal vector field of the surface of the fruit to be tested, the process also includes a step of identifying and repairing singular points in the continuous phase field. This step specifically includes: A pixel-level effective binary mask is calculated based on the spatial mapping relationship between the modulation depth and the spectral intensity image. Pixels with a modulation depth higher than the lower threshold of modulation depth and a spectral intensity lower than the upper threshold of saturation light intensity are marked as effective pixels in the effective binary mask, while the remaining pixels are marked as invalid pixels. A local search window is established with each invalid pixel in the validity binary mask as the center pixel; By utilizing the phase values ​​of effective pixels within the local search window and combining them with a Gaussian weighting function based on spatial distance, weighted neighborhood interpolation is performed to repair the center pixel.

5. The fruit quality testing method according to claim 1, characterized in that, The specific steps for reconstructing the unit normal vector field of the surface of the fruit to be tested include: Calculate the phase gradient components of the unwrapped continuous phase field in the horizontal and vertical directions in the pixel coordinate system; Using a pre-calibrated sensitivity coefficient, the phase gradient component is mapped to a geometric gradient in the physical space coordinate system; Based on the principles of differential geometry, a non-normalized normal vector is constructed using the geometric gradient, and then Euclidean normalization is applied to the normal vector to obtain the unit normal vector field.

6. The fruit quality testing method according to claim 4, characterized in that, The steps of extracting geometric features based on the unit normal vector field and extracting spectral biochemical features based on the true spectral reflectance distribution specifically include: Morphological erosion operations are performed on the effective binary mask to generate the region of interest. The average curvature field within the region of interest is calculated using the divergence operator of the unit normal vector field, and the mean and standard deviation of the average curvature field are statistically analyzed as geometric features. The spectral difference response value is calculated using the true spectral reflectance distribution to generate spectral biochemical characteristics.

7. The fruit quality testing method according to claim 6, characterized in that, The output of the comprehensive grading results of the fruit to be tested includes: Based on the geometric morphology features and statistical principles, a binary criterion for geometric defects is set, and a spectral defect criterion is generated based on spectral biochemical features combined with an adaptive segmentation threshold. Logical OR operation is performed on the geometric defect binarization criterion and spectral defect criterion in the region of interest to calculate the comprehensive defect area ratio on the surface of the fruit to be tested. The ratio of the average reflectance at a characteristic wavelength to the average reflectance at a reference wavelength within the region of interest is calculated to obtain the spectral characteristic index characterizing the internal quality. The comprehensive defect area ratio is compared with the preset maximum allowable defect area threshold, and the spectral characteristic index is compared with the preset quality spectral index threshold. Based on the comparison results, the fruits to be tested were classified as secondary fruits, premium fruits, standard fruits, or processed fruits.

8. A fruit quality detection system based on multispectral structured light, used to implement the method as described in any one of claims 1-7, characterized in that, include: Projection unit, imaging unit, and processor; The processor includes: The calibration processing module is used to perform system calibration, obtain the relative spatial position of the imaging unit and the projection unit, and calculate the illumination direction vector based on the relative spatial position. The acquisition control module is used to control the projection unit to cooperate with the imaging unit to acquire structured light image sequences and spectral intensity images of the surface of the fruit to be tested under the same field of view. The phase calculation module is used to calculate the initial truncated phase field and modulation depth using the structured light image sequence; The geometric reconstruction module is used to perform subsurface scattering phase inverse compensation on the initial truncated phase field using the spectral intensity image to obtain the truncated phase field, and to unwrap the truncated phase field by modulation depth to reconstruct the unit normal vector field of the surface of the fruit to be tested. The physical compensation module is used to perform physical inverse compensation on the spectral intensity image using the unit normal vector field and the illumination direction vector, thereby decoupling and obtaining the true spectral reflectance distribution. The comprehensive grading module is used to extract geometric features based on the unit normal vector field, extract spectral biochemical features based on the true spectral reflectance distribution, and output the comprehensive grading result of the fruit to be tested by combining the preset grading threshold.

Images