A method and system for identifying the maturity grade of durians based on deep learning

Abstract

The application discloses a durian maturity grade recognition method and system based on deep learning, and belongs to the technical field of grade recognition, and comprises the following steps: multi-source sensing data synchronous acquisition, dynamic threshold correction, hierarchical classification decision quantization output and standardized report generation. The application synchronously acquires multi-band radio frequency dielectric images, hyperspectral images and single fruit weight data of the durians to be measured, and performs noise reduction preprocessing. The images are input into a cross-modal deep learning feature extraction network, dynamic threshold truncation correction is performed, the image features are adaptively optimized, and a unified dimension durian quality deep feature vector is output. Then, the feature vector is input into a hierarchical deep learning classification model, the hierarchical decision threshold can be dynamically adjusted in a step adaptive manner, the durian maturity grade label is accurately output, and internal health indicators such as mold area, cavity volume and pulp fullness are synchronously quantized. Finally, multi-class detection data is integrated to generate a standardized structured quality data report.

Description

Technical Field

[0001] This invention relates to the field of grade recognition technology, and in particular to a method and system for recognizing the ripeness grade of durian based on deep learning. Background Technology

[0002] Based on the degree of fruit ripeness, internal integrity, and flesh fullness, a multi-level grading standard of A, B, and C is established. Domestic and international supermarkets and import / export trade markets have different requirements for grading thresholds and quality record dimensions. The industry urgently needs an integrated image-based intelligent grading system that can simultaneously output the weight and ripeness level of a single fruit.

[0003] Current non-destructive image recognition technologies for durian ripeness and internal quality suffer from numerous image processing limitations, hindering accurate multi-level assessments and automated digital report output. Existing solutions fall into three categories: visible light image detection, single-band spectral image analysis, and shallow machine learning image discrimination. Visible light imaging captures only a two-dimensional surface image of the peel, extracting only external features such as thorns and peel color. Visible light lacks penetrating power and is obstructed by the thick, uneven peel, preventing the acquisition of internal pulp imaging information. It can only achieve binary classification of ripeness, failing to provide the precise A, B, and C levels suitable for various markets. Single hyperspectral imaging acquires only a single-dimensional, narrow-band spectrum, without integrating multi-band radio frequency imaging. Uneven peel thickness, thorn shadows, and air in the fruit cavity cause pixel distortion noise and a low effective pixel ratio, resulting in insufficient identification of feature pixels corresponding to subtle ripeness gradients. Traditional preprocessing only performs grayscale stretching and threshold segmentation, failing to autonomously select key band pixels related to ripeness and internal defects. Invalid data creates feature redundancy, reducing grading accuracy. Traditional models rely on shallow machine learning and lack the ability to mine multi-scale features using deep learning. They can only extract shallow grayscale and texture features and cannot mine cross-modal deep correlation features between radio frequency imaging and hyperspectral images. The supporting image dataset is incomplete, with incomplete coverage of production areas, varieties, storage and transportation cycles, maturity gradients, and defect samples. The imbalance in the number of A and C grade samples causes training class bias, and scarce grade samples are underlearned. The existing image processing architecture lacks hierarchical classification logic adapted to A / B / C grades, only outputting a binary label of maturity, and lacks adjustable multi-level judgment thresholds, making it difficult to match different market grading standards. The image inference, weight acquisition, and data parsing modules are independent and cannot automatically integrate fruit weight, maturity grade, and internal defect quantification data. There is no automated report post-processing link, and it is impossible to generate standardized reports that connect to the production and sales system. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method and system for identifying durian maturity levels based on deep learning. This method can simultaneously acquire multi-band radio frequency dielectric images, hyperspectral images, and single-fruit weight data of the durian to be tested, and perform noise reduction preprocessing. The images are then input into a cross-modal deep learning feature extraction network for dynamic threshold truncation correction, adaptive optimization of image features, and output of a uniform-dimensional durian quality deep feature vector. The feature vector is then input into a hierarchical deep learning classification model, which can dynamically adjust the grading threshold through adaptive step size, accurately outputting durian maturity level labels, and simultaneously quantifying internal health indicators such as mold area, cavity volume, and flesh fullness. Finally, multiple types of detection data are integrated to generate a standardized structured quality data report.

[0005] On the one hand, a deep learning-based method for identifying durian maturity levels is provided. This method includes: Step S1, acquiring multi-band radio frequency dielectric imaging and hyperspectral images of the durian to be tested and performing noise reduction preprocessing, while simultaneously acquiring single-fruit weight sensing data; Step S2, constructing a cross-modal deep learning feature extraction network, inputting the noise-reduced multi-band radio frequency dielectric imaging and hyperspectral images into the cross-modal deep learning feature extraction network, and determining whether dynamic threshold truncation correction is performed. If yes, outputting a fused durian quality depth feature vector of a unified dimension after correction; otherwise, directly outputting the fused durian quality depth feature vector of a unified dimension; Step S3, ... A hierarchical deep learning classification model is pre-built. The deep feature vector of durian quality is input into the hierarchical deep learning classification model, and it is determined whether to perform dynamic adjustment of the step size adaptive grading threshold. If yes, the current adjustable grading judgment threshold is retrieved after adjustment. If no, the current adjustable grading judgment threshold is retrieved, and the maturity level label corresponding to the durian to be tested is output. Simultaneously, the internal health quantitative indicators of moldy area, cavity volume, and flesh fullness are quantitatively output. In step S4, the single fruit weight sensor data, maturity level label and the output internal health quantitative indicators of moldy area, cavity volume, and flesh fullness are correlated and integrated to generate a structured and standardized data report.

[0006] On the other hand, a deep learning-based durian maturity level recognition system is provided. This system includes: a multi-source sensor data synchronous acquisition module, a dynamic threshold correction module, a hierarchical classification decision quantization output module, and a standardized report generation module. The multi-source sensor data synchronous acquisition module acquires multi-band radio frequency dielectric imaging and hyperspectral images of the durian under test and performs noise reduction preprocessing, while simultaneously acquiring single-fruit weight sensor data. The dynamic threshold correction module constructs a cross-modal deep learning feature extraction network, inputs the noise-reduced multi-band radio frequency dielectric imaging and hyperspectral images into the network, and determines whether to perform dynamic threshold truncation correction. If yes, it outputs a fused, unified-dimensional durian quality deep feature vector after correction; otherwise, it directly outputs... The system integrates a unified dimension of durian quality deep feature vectors; a hierarchical classification and decision quantification output module is used to pre-build a hierarchical deep learning classification model, input the durian quality deep feature vectors into the hierarchical deep learning classification model, and determine whether to perform dynamic adjustment of the step size adaptive grading threshold. If so, it retrieves the appropriate adjustable grading threshold after adjustment; otherwise, it retrieves the appropriate adjustable grading threshold and outputs the maturity level label corresponding to the durian to be tested, and simultaneously quantifies and outputs internal health quantitative indicators such as moldy area, cavity volume, and flesh fullness; a standardized report generation module is used to associate and integrate single fruit weight sensor data, maturity level labels, and output internal health quantitative indicators such as moldy area, cavity volume, and flesh fullness to generate structured standardized data reports.

[0007] Beneficial effects The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: At the data acquisition and preprocessing level, multi-band radio frequency dielectric imaging and hyperspectral images were acquired simultaneously. Pixel grayscale correction and invalid band removal were performed to address typical interferences such as fruit spine shadows and uneven peel thickness. This effectively eliminated image distortion and noise pollution caused by the thick shell of durian, providing high-quality, low-noise input data for subsequent feature extraction and ensuring recognition accuracy from the source. At the cross-modal feature fusion and dynamic correction level, the constructed cross-modal deep learning feature extraction network can perform pixel-level deep fusion of radio frequency dielectric imaging and hyperspectral images, fully exploring the complementary information of the two heterogeneous images in the spatial-frequency domain. Simultaneously, through a dynamic correction mechanism that adaptively adjusts the truncation threshold based on the single-band spectral reflectance pixel value distribution skewness coefficient, it can intelligently remove specular reflection high-brightness pixels and shadow-occluded low-value invalid pixels, ensuring that the effective spectral reflectance data is constrained within the optimal dynamic range. This eliminates baseline drift caused by differences in origin and fluctuations in ambient light, improving the robustness and consistency of feature representation. At the hierarchical classification and quantitative output level, a hierarchical deep learning classification model was pre-constructed and trained using a high-quality dataset covering multiple production areas, multiple varieties, the entire storage and transportation cycle, complete maturity gradients, and various internal defects. Sample equalization effectively eliminated class bias and enhanced the model's ability to identify scarce grade samples. A step-size adaptive grading threshold dynamic adjustment mechanism was adopted, using a sliding window confusion matrix to statistically analyze the misclassification rate of grade A and the missed classification rate of grade C in real time. Combined with the trend slope and momentum-based step-size adaptive algorithm, the adjustable judgment thresholds for grades A / B and B / C were dynamically optimized, allowing the grading standards to adaptively tighten or loosen with fluctuations in raw material quality. This ensured grading accuracy while also considering the differentiated needs of different sales markets. Furthermore, while outputting maturity grade labels, the model simultaneously quantified and output internal health indicators such as the area of ​​moldy regions, the volume of fruit cavities, and the degree of pulp fullness, breaking through the limitations of traditional solutions that only output single-grade conclusions and achieving a multi-dimensional and refined description of quality information. At the data integration and output level, by automatically linking and integrating single fruit weight sensor data, maturity grade labels and internal health quantitative indicators, structured and standardized data reports are generated. No manual secondary sorting and entry is required. The data can be directly connected to the warehouse management system and traceability platform, which improves the automation and informatization level of the sorting line. Attached Figure Description

[0008] Figure 1 A flowchart of a deep learning-based method for identifying durian maturity levels is provided for embodiments of this application; Figure 2 A flowchart of dynamic threshold truncation correction for a deep learning-based method for identifying durian maturity levels is provided in an embodiment of this application. Figure 3 This is a schematic diagram of a deep learning-based durian ripeness level recognition system provided in an embodiment of this application. Detailed Implementation

[0009] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0010] like Figure 1 The diagram shown is a flowchart of a deep learning-based method for identifying durian maturity levels, provided in an embodiment of this application. The method includes the following steps: Step S1: Acquire multi-band radio frequency dielectric imaging and hyperspectral images of the durian to be tested and perform noise reduction preprocessing. Simultaneously acquire single-fruit weight sensing data of the durian to be tested. The multi-band radio frequency dielectric imaging is used to characterize the dielectric distribution pixel features corresponding to defects such as cavities, insect infestation, and rot in the durian pulp. The hyperspectral image contains full-band spectral pixel information covering starch saccharification and moisture changes in the durian pulp. During the acquisition process, noise reduction preprocessing is performed on both types of original images simultaneously. The noise reduction preprocessing includes pixel grayscale correction and invalid band pixel removal for pixel distortion noise caused by fruit spine shadows and uneven peel thickness.

[0011] Step S2: Construct a cross-modal deep learning feature extraction network. Input the multi-band radio frequency dielectric imaging image and hyperspectral image, which have undergone noise reduction preprocessing, into the cross-modal deep learning feature extraction network and determine whether to perform dynamic threshold truncation correction. If yes, output the fused durian quality depth feature vector of the same dimension after correction. If no, directly output the fused durian quality depth feature vector of the same dimension.

[0012] It should be explained that the cross-modal deep learning feature extraction network is constructed using a dual-branch asymmetric encoder structure, which independently extracts features for the different data characteristics of multi-band radio frequency dielectric imaging maps and hyperspectral images. The first branch is a hyperspectral feature encoder, whose input is a three-dimensional hyperspectral data cube (spatial height H × spatial width W × number of bands B, typically 256 × 256 × 150) after noise reduction preprocessing in step S1. This encoder employs a hybrid architecture combining a three-dimensional convolutional neural network (3D-CNN) and a spectral attention mechanism: First, it performs spatial-spectral joint downsampling on the hyperspectral data cube using three 3D convolutional layers (kernel sizes of 7×7×7, 5×5×5, and 3×3×3, strides of 2, 2, and 1, and output channels of 32, 64, and 128 respectively), compressing spatial resolution while preserving spectral continuity between adjacent bands. Then, the feature map output from the 3D convolution is reshaped into a two-dimensional feature sequence along the band dimension and input to the spectral attention module based on the Transformer architecture. This module utilizes a multi-head self-attention mechanism (8 heads) to calculate the global dependency weights between channels in each band dimension, thereby automatically identifying and enhancing key feature bands strongly correlated with maturity and internal quality (such as 1200). (e.g., 1450 nm sugar absorption peak, 1450 nm moisture absorption peak), while suppressing redundant band information introduced by causal skin thickness unevenness and environmental interference. Finally, through global average pooling and pointwise convolution operations, a hyperspectral depth feature map F_hyp with a spatial dimension of 16×16 and 256 channels is output (dimension 16×16×256).The second branch is the radio frequency dielectric feature encoder. Its input is a multi-band radio frequency dielectric image (containing amplitude and phase maps of at least 32 discrete frequency points, with a spatial resolution of H×W after registration, typically 256×256, and 64 channels, corresponding to the real and imaginary parts of the complex dielectric constant at each frequency point) after noise reduction preprocessing in step S1. This encoder adopts an architecture combining a two-dimensional convolutional neural network (2D-CNN) and deformable convolution: firstly, a standard 2D convolution layer (kernel size 5×5, stride 2, output channels 64) is used to perform preliminary dimensionality reduction on the input multi-channel radio frequency image; then, four deformable convolutional residual blocks are concatenated. Each residual block contains a deformable convolutional layer (3×3 kernel size), a batch normalization layer, and a ReLU activation function. The deformable convolutional layer learns additional spatial offsets, enabling the kernel sampling points to adaptively fit the irregular dielectric boundaries and defect edges of the durian fruit body. Compared with standard convolution, it can extract the geometric features of dielectric abnormalities such as cavities, insect infestation, and rot more accurately. The number of output channels of the four residual blocks are 128, 256, 512, and 256, respectively. After progressive downsampling, the spatial resolution is finally output as a radio frequency depth feature map F_rf with a dimension of 16×16×256, ensuring that its spatial size is completely consistent with the hyperspectral depth feature map output by the first branch, laying the foundation for dimension alignment for subsequent cross-modal fusion.

[0013] It should be noted that, as Figure 2 The flowchart shown is a dynamic threshold truncation correction process for a deep learning-based durian maturity level recognition method provided in this application embodiment. The specific process is as follows: First, input a single-band spectral reflectance pixel matrix and calculate the pixel distribution skewness coefficient. Adjust the adjustable scale coefficient differently according to the relationship between the skewness coefficient and the positive and negative skewness tolerance thresholds: when the skewness is greater than the positive threshold, use an attenuation factor to reduce the coefficient to tighten the upper limit and widen the lower limit truncation range; when the skewness is less than the negative threshold, use an amplification factor to increase the coefficient to widen the upper limit and tighten the lower limit truncation range; when the skewness is between the two thresholds, retain the original adjustable scale coefficient. Then, solve the upper and lower limit truncation thresholds of reflectance based on the adjusted adjustable scale coefficients. Iterate through all pixels in the matrix one by one and process the pixel values ​​into intervals. Pixels exceeding the upper limit are reset to the upper limit threshold or marked and removed; pixels below the lower limit are reset to the lower limit threshold or marked and removed; pixel values ​​within the interval remain unchanged. After all pixels have been truncated, output a standardized single-band spectral reflectance pixel matrix in which all effective pixel values ​​are constrained within the adaptive upper and lower limit intervals.

[0014] It should be understood that the specific process for determining whether to perform dynamic threshold truncation correction is as follows: A matrix of single-band spectral reflectance pixel values ​​under preset feature bands is obtained from a hyperspectral image. The single-band spectral reflectance pixel value represents the quantized value output after analog-to-digital conversion of the ratio of reflected light intensity to incident light intensity at each spatial pixel coordinate in the hyperspectral image data cube under the preset feature band. It is used to characterize the degree of absorption or scattering of photons of specific wavelengths by chemical groups (including hydroxyl and hydrocarbon groups) inside the pulp at that spatial location. Its value directly reflects the sugar accumulation or moisture state of the pulp area corresponding to that pixel. Among them, the preset feature band is a characteristic wavelength that is strongly correlated with the soluble solids content of the pulp, and its wavelength range is selected from a single specific wavelength channel in the near-infrared band of 900 nm to 2500 nm. The upper and lower limit truncation thresholds of reflectance are dynamically adjusted based on the pixel values ​​of single-band spectral reflectance. The upper and lower limit truncation thresholds of reflectance include a preset upper limit threshold and a preset lower limit threshold, which together constitute the upper and lower boundaries for filtering the effective range of spectral reflectance values. The upper limit threshold is used to suppress extremely high abnormal pixels caused by specular reflection from the tips of durian thorns, excessive brightness of the peel surface, or saturation effect of the hyperspectral sensor. The lower limit threshold is used to remove extremely low invalid pixels caused by excessive absorption of water film on the peel surface, shading by thorns, or dark current drift of the sensor. The reflectance values ​​of all effective spatial pixels within the effective peel area of ​​the durian to be tested after noise reduction preprocessing are statistically analyzed under a preset characteristic band. The arithmetic mean of reflectance is calculated, and an initial upper and lower limit threshold for reflectance is set. The initial upper and lower limit thresholds for reflectance are dynamically adjusted through a preset adjustable scaling coefficient. The preset adjustable scaling coefficient is used to control the width of the truncation boundary relative to the distribution of the arithmetic mean of reflectance. The distribution skewness coefficient of reflectance values ​​within the current effective area is calculated. The distribution skewness coefficient is used to quantitatively describe the degree of symmetry shift of the distribution of single-band spectral reflectance pixel values ​​of the current batch relative to the normal distribution.

[0015] In this embodiment, the feature bands of the hyperspectral image data cube completed by noise reduction preprocessing in step S1 are accurately extracted. Based on the correlation analysis results of soluble solids content and full-band spectral reflectance of a large number of durian samples during the model training stage, the feature bands are screened by combining partial least squares regression and variable projection importance index. Finally, the 1200 nm±10 nm near-infrared band is selected as the exclusive preset feature band for judging durian maturity. This band corresponds to the absorption characteristics of the first-order overtone stretching vibration of CH bond in durian pulp sugar molecules. Its spectral reflectance and pulp sugar accumulation show a strong negative correlation, with an absolute value of correlation coefficient of more than 0.85, which can accurately characterize the differences in pulp maturity gradient. Subsequently, the single-band spectral reflectance pixel values ​​corresponding to the spatial pixel coordinates of all effective flesh areas of durian under this characteristic band were extracted. This value is the ratio of reflected light intensity to incident light intensity collected by the hyperspectral sensor, and is a dimensionless quantization parameter in the 0-1 range obtained by 12-bit or 14-bit analog-to-digital conversion. The lower the pixel reflectance value, the higher the photon absorption intensity of sugars in the corresponding flesh area, the more sufficient the sugar accumulation, and the better the maturity. Conversely, it indicates insufficient sugar accumulation in the flesh and low maturity. To accurately filter out various abnormal noise pixels during image acquisition, this method sets up a dual-layer reflectivity cutoff threshold system, including an upper and lower threshold. The upper reflectivity threshold is used to specifically suppress abnormally high reflectivity pixels above 0.9 caused by specular reflection from fruit thorn tips, reflection from the thick waxy layer of the peel, and sensor pixel saturation. The lower reflectivity threshold is used to remove invalid pixels with low values ​​below 0.05 caused by strong light absorption from the water film on the peel surface, light shading from fruit thorns, and sensor dark current drift, thus completely avoiding spectral data distortion caused by the physical structure of the peel and the operating conditions of the equipment. In the specific threshold calculation process, the reflectivity values ​​of all pixels in the 1200 nm band of the effective flesh area of ​​the durian to be tested are first statistically analyzed, and the arithmetic mean and standard deviation of the reflectivity are calculated. Based on a preset initial adjustable scale coefficient of 2.5, the initial upper and lower cutoff thresholds are constructed. This scale coefficient can adjust the width of the threshold range. The larger the coefficient, the higher the noise tolerance and the more pixels are retained; the smaller the coefficient, the more stringent the noise filtering and the stronger the noise reduction effect. Meanwhile, the pixel reflectance distribution characteristics are statistically analyzed using a dedicated skewness coefficient calculation formula, which quantitatively characterizes the offset characteristics of spectral data relative to the normal distribution. Positive skewness indicates that the proportion of bright abnormal specular reflection pixels is relatively high, negative skewness indicates that the proportion of shadow low-value invalid pixels is relatively high, and skewness close to zero indicates that the image spectral quality is good and there is little abnormal interference.This method sets ±0.5 as the skewness tolerance threshold to achieve adaptive threshold adjustment. When the skewness coefficient is greater than 0.5, indicating a significant positive skewness, a 0.7 attenuation factor is used to reduce the scale coefficient, narrowing the upper threshold and widening the lower threshold to accurately remove high-brightness noise from the specular reflection of fruit thorns. When the skewness coefficient is less than -0.5, indicating a significant negative skewness, a 1.43 amplification factor is used to expand the scale coefficient, widening the upper threshold and tightening the lower threshold to effectively filter out low-value noise caused by shadows and water films. When the skewness coefficient is within the range, the parameters remain unchanged, and the correction process is skipped. Finally, the final truncation threshold is determined based on the adaptively adjusted scale coefficient, completing the dynamic screening and correction of spectral pixels. This adaptive threshold correction method abandons the limitations of traditional fixed thresholds and can dynamically adapt the correction strategy according to the noise distribution characteristics of the actual spectral image of a single sample. While preserving the effective maturity spectral feature information of the fruit pulp to the maximum extent, it accurately filters out various scene-specific abnormal noises, significantly improving the accuracy and robustness of subsequent multimodal feature fusion and maturity grading determination.

[0016] It should be further explained that determining whether to perform dynamic threshold truncation correction also includes: If the distribution skewness coefficient is greater than the preset positive skewness tolerance threshold (i.e., positive skewness, which indicates that there are significantly more high reflectivity abnormal pixels than low reflectivity abnormal pixels), the adjustable scale coefficient is reduced according to the preset attenuation factor β (0<β<1) to tighten the upper limit cutoff range and widen the lower limit cutoff range, thereby adaptively eliminating high brightness abnormal pixels introduced by specular reflection. If the distribution skewness coefficient is less than the preset negative skewness tolerance threshold (i.e., negative skewness, which indicates that there are significantly more low reflectivity shadow abnormal pixels than high reflectivity abnormal pixels), the adjustable scale coefficient is increased according to the preset magnification factor (1 / β) to widen the upper limit cutoff range and tighten the lower limit cutoff range, thereby adaptively eliminating low-value abnormal pixels introduced by shadow occlusion. If the distribution skewness coefficient is between the positive tolerance threshold and the negative tolerance threshold, then the current adjustable scaling coefficient remains unchanged; Based on the adjusted adjustable scale coefficient, the final upper limit cutoff threshold and the final lower limit cutoff threshold are determined.

[0017] The specific process for determining the final upper and lower limit cutoff thresholds for reflectivity is as follows: Traverse each pixel value in the single-band spectral reflectance pixel value matrix and perform a truncation operation: if the pixel value is greater than the upper limit truncation threshold of reflectance, reset the pixel value to the upper limit truncation threshold of reflectance, or directly mark the pixel as an invalid pixel and remove it. If a pixel value is less than the lower limit of reflectivity truncation threshold, then the pixel value is reset to the lower limit of reflectivity truncation threshold, or the pixel is directly marked as an invalid pixel and removed. If the pixel value is between the lower and upper reflectance cutoff thresholds, then the current pixel value remains unchanged. The output is a normalized single-band spectral reflectance pixel value matrix after truncation. All valid pixel values ​​in the normalized single-band spectral reflectance pixel value matrix are constrained within the adaptively adjusted upper and lower truncation threshold range.

[0018] In this embodiment, before feature extraction, selective correction is performed on the hyperspectral data based on the pre-judgment results: if the distribution skewness coefficient γ is greater than the preset positive skewness tolerance threshold (e.g., +0.5), it indicates that the current image has a large number of abnormally bright pixels due to specular reflection. In this case, the adjustable scaling coefficient α is reduced by the preset attenuation factor β=0.7, narrowing the truncation boundary from μ±ασ to tighten the upper truncation range and widen the lower truncation range; if γ is less than the preset negative skewness tolerance threshold (e.g., -0.5), it indicates that there are a large number of low-value invalid pixels due to shadow occlusion. In this case, α is increased by the preset amplification factor 1 / β=1.43 to widen the upper truncation range and tighten the lower truncation range; if γ is between the two, α is kept unchanged. A cross-modal deep learning feature extraction network with a dual-branch asymmetric encoder structure is constructed. The first branch is a hyperspectral feature encoder. Its input is a three-dimensional hyperspectral data cube (spatial H×W, number of bands B) after dynamic threshold correction. First, it performs spatial-spectral joint downsampling through three 3D convolutional layers (convolutional kernels 7×7×7, 5×5×5, 3×3×3, output channels 32, 64, 128). Then, the output feature map is reshaped into a two-dimensional feature sequence along the band dimension and input into the Transformer-based spectral attention module. The module uses an 8-head self-attention mechanism to calculate global dependency weights in the band dimension, automatically enhancing key bands strongly related to sugar and water content (such as 1200 nm and 1450 nm) and suppressing redundant bands. Finally, it outputs a hyperspectral depth feature map with a spatial dimension of 16×16×256 through global average pooling and pointwise convolution. The second branch is an RF dielectric feature encoder. Its input is a multi-band RF dielectric imaging map (64 channels, corresponding to the real and imaginary parts of the complex dielectric constant at each frequency). It first undergoes a preliminary dimensionality reduction through a 5×5 convolution (stride 2, output channels 64), and then four deformable convolutional residual blocks (3×3 deformable convolution + batch normalization + ReLU) are connected in series. The deformable convolution learns the spatial offset to adaptively fit the irregular dielectric boundaries and defect edges of the durian fruit body. The output channels of the four residual blocks are 128, 256, 512, and 256 respectively, and the final output is an RF depth feature map with a spatial dimension of 16×16×256. A cross-modal attention fusion module is constructed to perform deep interaction between two types of features: the hyperspectral deep feature map is used to generate a query matrix through linear mapping, and a key matrix and a value matrix are generated through linear mapping. Cross-modal attention weights are calculated, and the hyperspectral features enhanced by radio frequency features are obtained using these weights. At the same time, reverse attention is performed symmetrically to obtain radio frequency features enhanced by hyperspectral features. The residuals of the original features and the enhanced features are concatenated along the channel dimension, and then fused by 1×1 convolution to output a fused feature map (16×16×512).Finally, a global feature aggregation layer is connected, and spatial global average pooling and max pooling are performed. The two 512-dimensional feature vectors are concatenated and then refined through a fully connected layer (input 1024, output 512). The final output is a durian quality deep feature vector (1×512) with a unified dimension. This vector aggregates the quality information of each region of the whole durian in the spatial dimension and integrates the spectral response of hyperspectral chemical components and the internal physical structure response of radio frequency dielectric in the channel dimension. It has cross-modal complementarity and global representativeness, providing accurate input features for subsequent hierarchical classification and quantitative output.

[0019] Step S3: A hierarchical deep learning classification model is pre-constructed. The deep feature vector of durian quality is input into the hierarchical deep learning classification model, and it is determined whether to perform dynamic adjustment of the step size adaptive grading threshold. If yes, the appropriate grading threshold is retrieved after adjustment. If no, the appropriate grading threshold is retrieved, and the maturity level label corresponding to the durian to be tested is output. Simultaneously, the internal health quantitative indicators of moldy area area, cavity volume, and flesh fullness are quantitatively output. The hierarchical deep learning classification model is trained on a high-quality dataset of standardized labeled multimodal images covering multiple production areas, multiple varieties, the entire storage and transportation cycle, complete maturity gradient, and various internal defects. During the training process, sample equalization is performed to eliminate the class bias by addressing the imbalance in the number of A-grade high-quality fruits and C-grade defective fruits in the dataset.

[0020] It should be understood that when pre-constructing a hierarchical deep learning classification model, a hierarchical dual-branch network architecture is first built. One branch is used for maturity probability prediction, and the other branch is used for internal defect quantification regression. The network input is set with a fully connected mapping layer adapted to the dimension of the deep feature vector of durian quality. The back end is set with two-level classification output heads to output the classification probabilities corresponding to grades A, B, and C respectively. At the same time, a regression output head is provided to output the quantitative values ​​of moldy area, cavity volume, and flesh fullness. Secondly, a high-quality multimodal labeled dataset was constructed. This dataset collected data covering various production areas in Southeast Asia, different durian varieties, different storage and transportation cycles, and ripeness gradients from immature to fully overripe and fully ripe. It also included multi-band radio frequency dielectric imaging and hyperspectral images of various internal defects such as cavities, insect infestation, and mold. Each sample was individually opened for verification, and the ripeness level and defect quantification ground truth were manually labeled to form a standardized labeled dataset. To address the class imbalance problem caused by the significantly smaller number of Grade A high-quality fruit samples and Grade C defective fruit samples compared to the intermediate Grade B fruits, an equalization processing scheme was adopted, which combined oversampling to expand scarce samples, undersampling to compress redundant Grade B samples, and weighted class loss. This eliminated the class bias that easily occurred during model training, preventing the model from favoring the more prevalent Grade B samples and reducing the accuracy of identifying high-quality and defective fruits. Subsequently, the training set, validation set, and test set were divided, and the deep feature vectors were uniformly standardized and preprocessed. A joint composite loss function of cross-entropy classification loss and mean squared error regression loss was set. An adaptive optimizer was selected to carry out multiple rounds of iterative training. During the training process, the validation set was continuously used to monitor the classification accuracy and defect quantification error. When the validation index did not improve for several consecutive rounds, an early stopping mechanism was triggered to save the optimal weight parameters, thus completing the offline pre-construction and solidification of the hierarchical deep learning classification model. After the model is deployed to the recognition process, it receives the deep feature vector of durian quality output from the cross-modal network to complete forward inference. The program automatically determines whether to enable the step size adaptive grading threshold dynamic adjustment logic. If enabled, it iteratively corrects the first and second adjustable grading judgment thresholds based on the sliding window misjudgment, omission indicators and change slope, and then retrieves the updated grading thresholds. If disabled, it directly retrieves the system default benchmark grading thresholds. Based on the two adjustable thresholds, it completes the A / B / C three-level stratification judgment and outputs a unique maturity level label. Simultaneously, it calculates and outputs three internal health quantitative indicators: moldy area, fruit cavity volume, and flesh fullness through the defect regression branch. The stratified model trained on a balanced dataset takes into account both the maturity grading accuracy and the internal defect quantification accuracy, adapting to the differentiated grading and management needs of A / B / C in multiple markets.

[0021] It needs to be explained that the specific process for determining whether to perform adaptive step-size hierarchical threshold dynamic adjustment is as follows: A first adjustable grading threshold is preset between grade A and grade B to distinguish between grade A and grade B, and a second adjustable grading threshold is preset between grade B and grade C to distinguish between grade B and grade C. The adjustable grading judgment threshold is dynamically adjusted based on the threshold update step size. The adjustable grading judgment threshold refers to the critical value of the probability score between adjacent maturity levels in the output layer of the hierarchical deep learning classification model, including the first adjustable grading judgment threshold and the second adjustable grading judgment threshold. When the probability value of grade A output by the model is greater than or equal to the first adjustable grade determination threshold, the sample is determined to be grade A; when the probability value of grade A is less than the first adjustable grade determination threshold and the probability value of grade B is greater than or equal to the second adjustable grade determination threshold, the sample is determined to be grade B. When the probability value of grade B is less than the second adjustable grading threshold, the sample is determined to be grade C. The adjustable grading threshold is used to convert the continuous probability distribution of the output into discrete multi-level labels. The first adjustable grading threshold and the second adjustable grading threshold are both continuous real numbers with a value range of [0,1].

[0022] The threshold update step size is defined as the numerical unit for incremental adjustment of the adjustable grading threshold within each iteration adjustment cycle. It includes a first threshold update step size for the first adjustable grading threshold and a second threshold update step size for the second adjustable grading threshold. The threshold update step size is used to control the speed and magnitude of threshold change during each adjustment process. Both the first and second threshold update step sizes are preset positive real numbers, ranging from 0.01 to 0.05.

[0023] Determining whether to perform step-size adaptive hierarchical threshold dynamic adjustment also includes: Step 1: During the continuous operation of the sorting line, a fixed number N (N greater than or equal to 50) of durian fruits are used as a sliding batch window. The confusion matrix between the model's grading judgment result and the actual destructive verification label is calculated within the current window. Based on the confusion matrix, the false positive rate for Grade A and the false negative rate for Grade C are calculated respectively. The false positive rate for Grade A is defined as the ratio of the number of samples in the current window that are judged as Grade A by the model but are actually verified as not Grade A to the total number of samples in the window. The false negative rate for Grade C is defined as the ratio of the number of samples in the current window that are actually Grade C but are judged as not Grade C by the model to the total number of samples in the window. Step 2: Obtain the misclassification rate sequence of level A and the omission rate sequence of level C for L consecutive historical windows (L is greater than or equal to 3). Calculate the slope of the trend of level A and the slope of the trend of level B respectively. The slope of the trend is obtained by linearly fitting the misclassification rate / omission rate sequence of the current window and the previous L-1 windows using the least squares method. Step 3: Based on the comparison between the Level A misjudgment rate and the preset Level A misjudgment tolerance upper limit, and the sign of the slope of the Level A change trend, determine the adjustment direction and adjustment amount of the first adjustable classification judgment threshold, specifically as follows: If the misjudgment rate of Class A is greater than the preset tolerance limit for misjudgment of Class A and the slope of the change trend of Class A is greater than or equal to 0, then the adjustment is made in the direction of increasing the first adjustable classification judgment threshold according to the current first threshold update step size. That is, the adjusted first adjustable classification judgment threshold is the sum of the initial first adjustable classification judgment threshold and the first threshold update step size. If the misjudgment rate of Class A is less than or equal to the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of change of Class A is less than 0, then the adjustment is made in the direction of reducing the first adjustable classification judgment threshold according to the current first threshold update step size. That is, the adjusted first adjustable classification judgment threshold is the difference between the initial first adjustable classification judgment threshold and the first threshold update step size. If the misjudgment rate of Class A is less than or equal to the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of Class A is greater than or equal to 0, or if the misjudgment rate of Class A is greater than the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of Class A is less than 0, then the first adjustable classification judgment threshold remains unchanged.

[0024] In this embodiment, the basic network parameters and grading threshold system are preset. First and second adjustable grading thresholds with values ​​ranging from [0,1] are pre-set as the probability score thresholds for A / B and B / C maturity levels, respectively. Simultaneously, positive real numbers in the range of 0.01 to 0.05 are preset as the update step size for the two thresholds to control the amplitude and speed of threshold iteration adjustment, laying the foundation for dynamic correction of the network's grading output. Secondly, a cross-modal feature extraction backbone network is constructed, fusing visual modal features such as texture, color, and shape of durian fruit images with physical modal features such as fruit density and hardness. Single-modal deep feature extraction is completed through a multi-layer convolution module. The cross-modal feature fusion module performs dimensional alignment and adaptive weight fusion of the two types of features, eliminating redundant interference features and generating a globally fused feature that combines visual and physical representations. Subsequently, connecting the network output layer with the grading logic, the fused features are input into the classification output layer, outputting probability scores for maturity levels A, B, and C. A preset grading threshold is used to transform the continuous probability distribution into discrete level labels, completing the basic grading determination. Finally, an adaptive threshold dynamic adjustment mechanism is embedded to achieve network optimization iteration. Using at least 50 fruit samples as a sliding batch window, the confusion matrix between the model's grading results and the actual sampled labels is statistically analyzed window by window, calculating the misclassification rate for level A and the missed classification rate for level C. Error rate sequences from at least three consecutive windows are selected, and the slope of the error change trend is obtained through least squares linear fitting. The comparison results of the error rate and the preset tolerance threshold, along with the positive or negative slope, determine the direction and magnitude of threshold adjustment. When the level A misclassification rate exceeds the limit and the trend does not decrease, the first threshold is increased by a preset step size; when the level A misclassification rate is within the acceptable range and the error shows a decreasing trend, the first threshold is decreased by a step size; in other cases, the threshold remains stable, and the second threshold synchronously follows this adaptive adjustment logic. This network effectively solves the problem of insufficient accuracy in fixed threshold grading by accurately extracting cross-modal features and dynamically iteratively optimizing grading thresholds, thus meeting the needs for accurate grading of fruit maturity in continuous sorting scenarios on production lines.

[0025] It should be further explained that determining the adjustment direction and amount of the first adjustable grading threshold also includes: Step four: Based on the comparison between the Level C false negative rate and the preset Level C false negative tolerance upper limit, and the sign of the slope of the Level B change trend, determine the adjustment direction and adjustment amount of the second adjustable grading judgment threshold, specifically as follows: If the C-level false negative rate is greater than the preset C-level false negative tolerance limit and the slope of the B-level change trend is greater than or equal to 0, then the adjustment is made in the direction of reducing the second adjustable classification judgment threshold according to the current second threshold update step size. That is, the adjusted second adjustable classification judgment threshold is the difference between the initial second adjustable classification judgment threshold and the second threshold update step size. If the C-level false negative rate is less than or equal to the preset C-level false negative tolerance limit and the slope of the B-level change trend is less than 0, then the second adjustable classification judgment threshold will be adjusted in the direction of increasing the second adjustable classification judgment threshold according to the current second threshold update step size. That is, the adjusted second adjustable classification judgment threshold is the sum of the initial second adjustable classification judgment threshold and the second threshold update step size. If the C-level false negative rate is less than or equal to the preset C-level false negative tolerance limit and the slope of the B-level change trend is greater than or equal to 0, or if the C-level false negative rate is greater than the preset C-level false negative tolerance limit and the slope of the B-level change trend is less than 0, then the second adjustable classification judgment threshold remains unchanged. Step 5: After executing steps 3 and 4, dynamically update the first threshold update step size and the second threshold update step size.

[0026] The specific process for dynamically updating the first threshold update step size and the second threshold update step size is as follows: The number of times the adjustment direction of the first adjustable grading judgment threshold and the second adjustable grading judgment threshold changes within multiple consecutive adjustment cycles is counted. If the adjustment direction does not change within P consecutive adjustment cycles (P is greater than or equal to 3), it indicates that the threshold convergence speed is insufficient at the current step size. Then, the corresponding threshold update step size is increased according to the preset acceleration factor η (η is greater than 1). That is, the adjusted first threshold update step size is the product of the first threshold update step size and the preset acceleration factor, or the adjusted second threshold update step size is the product of the second threshold update step size and the preset acceleration factor. And the adjusted step size does not exceed the preset maximum step size upper limit. If the number of times the adjustment direction reverses within a continuous adjustment period exceeds the preset oscillation tolerance number, the corresponding threshold update step size is reduced according to the preset attenuation factor δ (0 is less than δ and less than 1). That is, the adjusted first threshold update step size is the product of the first threshold update step size and the preset attenuation factor, or the adjusted second threshold update step size is the product of the second threshold update step size and the preset attenuation factor, and the adjusted subsequent step size is not lower than the preset minimum step size lower limit. If the change in adjustment direction does not trigger any of the above conditions, the current threshold update step size remains unchanged; Step 6: Store the adjusted final adjustable grading threshold, the first adjustable grading threshold, and the second adjustable grading threshold into the cache and apply them to the real-time grading determination of the next sliding batch window.

[0027] In this embodiment, the network pre-parameters and the grading threshold system are initialized. A first adjustable grading threshold for distinguishing between A and B maturity levels and a second adjustable grading threshold for distinguishing between B and C maturity levels are pre-set. Both thresholds are continuous real numbers in the interval [0,1], corresponding to the probability score thresholds of adjacent maturity levels in the network output layer. This can transform the continuous probability distribution output by the model into discrete A, B, and C level fruit labels. At the same time, two sets of threshold update step sizes are initialized, namely the first and second threshold update step sizes used to fine-tune the first and second grading thresholds. The initial values ​​are preset positive real numbers in the range of 0.01 to 0.05, used to control the magnitude and speed of threshold adjustment in a single iteration. Hyperparameters such as acceleration factor, decay factor, oscillation tolerance number, and convergence determination period required for step size iteration are configured synchronously, defining the upper and lower limit constraints of the threshold step size. Secondly, a core cross-modal deep learning feature extraction backbone network is constructed, integrating multi-dimensional modal information such as durian appearance images and internal quality. Through independent single-modal feature extraction branches, deep extraction of basic features such as visual texture, color contour, and internal maturity properties is completed. Then, through the cross-modal feature fusion module, the dimensions of multi-branch features are aligned and the weights are adaptively weighted and fused to filter out invalid noise features. The cross-modal fusion features with both global representation ability and detail recognition are condensed and input into the network classification output layer to calculate the predicted probability values ​​of the sample corresponding to the three maturity levels A, B, and C, thus completing the basic hierarchical inference. Secondly, an online error statistics and trend analysis module was built to achieve dynamic data monitoring of the production line. A sliding batch window was constructed with a fixed sample size of no less than 50 durian fruits. During continuous sorting operation, the correspondence between the model's automatic grading results and the actual labels obtained from manual destructive sampling was statistically analyzed window by window, generating a grading confusion matrix. This accurately calculated two core error indicators within the window: the misclassification rate for Grade A and the missed classification rate for Grade C. The misclassification rate for Grade A was the proportion of non-Grade A samples misclassified as Grade A out of the total samples in the window, while the missed classification rate for Grade C was the proportion of actual Grade C samples misclassified as non-Grade C. Subsequently, error data from no less than three consecutive historical windows were selected to construct an error sequence. Least squares linear fitting was used to generate the slopes of the error change trends for Grade A and Grade B, accurately determining the dynamic fluctuation trend of the model's grading error. Then, a dual-threshold differential adaptive adjustment was implemented, updating the two grading thresholds based on both the error threshold and the trend slope. For the first adjustable classification threshold, if the misclassification rate of Class A exceeds the preset tolerance limit and the slope of the error trend of Class A is non-negative, it indicates that the misclassification problem of Class A continues to worsen, and the threshold will be adjusted upward according to the first threshold step size; if the misclassification rate of Class A is controlled within the tolerance range and the slope of the error of Class A is negative, it indicates that the classification accuracy continues to improve, and the threshold can be adjusted downward according to the step size to adapt to the sample distribution; for other cross-balancing conditions, the first threshold remains unchanged.Regarding the second adjustable classification threshold, if the C-level false negative rate exceeds the preset tolerance limit and the B-level error trend slope is non-negative, it indicates that the C-level false negative problem continues to worsen. The threshold needs to be lowered by updating the step size according to the second threshold to reduce false negatives. If the C-level false negative rate meets the standard and the B-level error slope is negative, the classification status is stable and improving. The classification range can be optimized by increasing the threshold by the step size. For other steady-state conditions, the second threshold is kept constant. Based on this, a second dynamic iterative optimization of the threshold update step size is completed. The step size parameter is adaptively updated based on the change law of the threshold adjustment direction over at least three consecutive adjustment cycles. If the threshold adjustment direction remains consistent for multiple consecutive cycles, it indicates that the model threshold convergence rate is slow and the optimization efficiency is insufficient under a fixed step size. In this case, a preset acceleration factor greater than 1 will be used to amplify the corresponding threshold update step size to accelerate the model convergence speed. The optimized step size will strictly not exceed the preset maximum step size upper limit. If the threshold adjustment direction frequently reverses within consecutive cycles and the number of oscillations exceeds the preset tolerance value, it is determined that the current step size is too large, causing model instability. The corresponding step size will be reduced by a decay factor in the range of 0 to 1 to suppress parameter oscillations. The optimized step size will not be lower than the preset minimum step size lower limit. If there are no obvious slow convergence or parameter oscillation problems, the original step size parameter will remain unchanged to ensure adjustment stability. Finally, the parameter update and closed-loop iteration are completed. The new first and second adjustable grading thresholds, after threshold adjustment and step size optimization, are cached and stored, replacing the original grading threshold parameters of the network. They are then directly applied to the fruit maturity grade determination in the next sliding batch window. Through the closed-loop process of "feature extraction - grading inference - error statistics - threshold tuning - step size iteration - parameter update", the cross-modal network can continuously adapt to the changes in the distribution of fruit samples in the pipeline, effectively improving the accuracy and robustness of durian maturity three-level grading under complex working conditions.

[0028] It needs to be explained that the specific process for outputting the ripeness level label corresponding to the durian to be tested is as follows: Retrieve the first adjustable grading threshold and the second adjustable grading threshold at the current moment. If the probability value of grade A is greater than or equal to the first adjustable grading threshold, directly determine that the durian to be tested is grade A, output the maturity level label A, and terminate the subsequent grading process. If the probability value of grade A is less than the first adjustable grading threshold, then the probability value of grade B is further compared with the second adjustable grading threshold: if the probability value of grade B is greater than or equal to the second adjustable grading threshold, then the durian to be tested is determined to be grade B, the maturity level label B is output, and the subsequent judgment process is terminated. If the probability value of grade A is less than the first adjustable grading threshold and the probability value of grade B is less than the second adjustable grading threshold, then the durian to be tested is determined to be grade C, and the maturity level label C is output. The correspondence between each grade label, the model output probability value, and the adjustable threshold is represented by the following interval mapping rule: Class A decision interval: PA ∈ [T_AB, 1], and PA ≥ PB and PA ≥ PC; Class B decision interval: PA ∈ [0, T_AB), and PB ∈ [T_BC, 1], and PB ≥ PC; Class C decision interval: PA ∈ [0, T_AB), and PB ∈ [0, T_BC).

[0029] In this embodiment, the initial configuration of the network hyperparameters and hierarchical control system is completed. A first adjustable hierarchical judgment threshold for distinguishing between A-level and B-level maturity and a second adjustable hierarchical judgment threshold for distinguishing between B-level and C-level maturity are pre-set. Both thresholds are continuous real numbers in the range [0,1], corresponding to the probability score critical standards of the network output layer for the three maturity levels. At the same time, the update step size of the first and second thresholds is initialized. The initial step size is a positive real number from 0.01 to 0.05, which is used to control the amplitude and speed of threshold fine-tuning during a single iteration. Simultaneously, the upper limit of tolerance for misjudgment of A-level, the upper limit of tolerance for omission of C-level, the number of historical windows required for least squares fitting, the step size iteration acceleration factor, the decay factor, the number of oscillation tolerances, and the upper and lower limit constraints of the step size are preset to provide basic constraints for network adaptive optimization. Secondly, a cross-modal deep learning feature extraction backbone network was constructed. Addressing the multi-dimensional feature requirements for durian maturity recognition, a multi-branch independent feature extraction structure was built. Deep semantic extraction was performed on the texture, color, and morphological features of the durian appearance image modality, as well as the key maturity features of the internal quality property modality. A cross-modal feature fusion module completed dimensionality correction, adaptive weight allocation, and redundant information removal for features of different dimensions, fusing them to obtain a high-dimensional global feature vector that combines appearance and internal quality representation. This fused feature was input into the classification output layer at the end of the network. The activation function was used to calculate the predicted probability values ​​for the durian samples to be tested, corresponding to maturity levels A, B, and C, respectively, providing accurate probability output for subsequent level determination. Secondly, an online error monitoring and trend analysis module for the production line is constructed. Using at least 50 durian fruits as a sliding batch window, sample data is continuously collected while the sorting line is running continuously. The model's automatic grading results are compared window by window with the actual labels from manual destructive sampling to construct a grading confusion matrix. The misclassification rate of grade A and the missed classification rate of grade C in the current window are accurately statistically calculated. The misclassification rate of grade A is the proportion of non-grade A samples misclassified as grade A by the model out of the total samples in the window, and the missed classification rate of grade C is the proportion of true grade C samples that are judged as non-grade C by the model. Based on this, error indicators from at least three consecutive historical windows are selected to form a time series. Least squares linear fitting is used to obtain the slopes of the grade A and grade B trends, reflecting the dynamic changes in grading errors, accurately capturing the dynamic fluctuations in the model's grading accuracy.Subsequently, based on the dual constraint rules of error threshold and trend slope, adaptive adjustment of the two-level classification thresholds is completed. For the first adjustable classification judgment threshold, when the misjudgment rate of Class A exceeds the preset tolerance limit and the error trend slope is non-negative, the threshold is increased by updating the step size according to the preset first threshold. When the misjudgment rate of Class A meets the standard and the error trend slope is negative, the threshold is decreased. The threshold remains unchanged in other balanced operating conditions. For the second adjustable classification judgment threshold, when the missed judgment rate of Class C exceeds the standard and the error trend slope is non-negative, the threshold is decreased by updating the step size according to the second threshold to suppress the missed judgment problem. When the missed judgment rate of Class C meets the standard and the error trend slope continues to decrease, the threshold is increased to optimize the classification range. The threshold remains constant in other steady-state operating conditions. After the threshold adjustment is completed, the dynamic iterative update of the threshold update step size is further realized. By statistically analyzing the change pattern of the threshold adjustment direction in multiple consecutive adjustment cycles, for cases where the adjustment direction has not changed for no less than 3 consecutive cycles and the model convergence speed is insufficient, a preset acceleration factor greater than 1 is used to amplify the corresponding step size and strictly limit the upper limit of the maximum step size. For cases where the adjustment direction frequently reverses and the parameter oscillation exceeds the standard, a decay factor in the range of 0 to 1 is used to reduce the step size and ensure that the step size is not lower than the minimum lower limit. When there is no abnormal fluctuation, the current step size parameter remains unchanged, so that the adjustment accuracy and convergence efficiency are dynamically adapted to the sorting conditions. Finally, the updated first and second adjustable grading thresholds are loaded to accurately output the maturity grade label of the durian to be tested. The system automatically retrieves the latest cached two-level thresholds and completes the grade identification according to the priority-based grading rule: First, the probability value of grade A output by the model is compared with the first adjustable grading threshold. If the probability value of grade A is greater than or equal to the threshold, the durian to be tested is directly determined to be grade A, the grade label A is output, and the current grading process is terminated. If the probability value of grade A is less than the first adjustable grading threshold, the grade B grading is further initiated. The probability value of grade B is compared with the second adjustable grading threshold. When the probability value of grade B is greater than or equal to the threshold, the durian to be tested is determined to be grade B, the grade label B is output, and the grading process is terminated. If the conditions of the probability value of grade A being less than the first adjustable grading threshold and the probability value of grade B being less than the second adjustable grading threshold are met simultaneously, the durian to be tested is determined to be grade C, and the grade label C is output. The entire network achieves high-precision, dynamic, and intelligent graded recognition of durian maturity levels in a production line scenario through cross-modal feature deep fusion, graded error temporal analysis, threshold and step size dual-layer adaptive optimization, and graded priority determination output mechanism.

[0030] Step S4 involves linking and integrating the single fruit weight sensor data, maturity grade labels, and internal health quantitative indicators such as moldy area, fruit cavity volume, and fruit flesh fullness to generate a structured and standardized data report.

[0031] It should be understood that the global initialization of the system's initial parameters and hierarchical control system is completed. The first and second adjustable hierarchical judgment thresholds with a value range of [0,1] are pre-set to define the model probability critical standards for maturity levels A and B, and B and C, respectively. At the same time, the update step size of the first and second thresholds in the range of 0.01 to 0.05 is initialized, and a complete set of hyperparameters such as error tolerance threshold, number of historical fitting windows, step size acceleration and decay factors, upper and lower limits of step size, and number of oscillation tolerances are configured to establish a constraint benchmark for network adaptive iterative control. Secondly, a cross-modal deep learning feature extraction backbone network based on multi-source information collaboration was constructed, and a multi-branch independent feature extraction structure was built. On the one hand, deep semantic mining was performed on the visual features of durian appearance image modality, such as texture, color, and surface defects. On the other hand, single-fruit weight sensing data collected by the device was accessed, and three types of internal health quantification indicators, namely the area of ​​moldy region, the volume of fruit cavity, and the degree of fruit flesh fullness, were fused simultaneously with the model inference output. Multi-dimensional feature alignment and fusion processing of visual image modality, weight sensing numerical modality, and internal health quantification modality were completed. Redundant noise information was removed through feature screening and weight adaptive fusion mechanism, and a high-dimensional cross-modal fusion feature vector that takes into account appearance maturity features, physical weight features, and internal quality health features was constructed, providing comprehensive and multi-dimensional feature support for subsequent accurate grading. Secondly, an online error monitoring and time-series trend analysis module for the sorting line was constructed. Using a fixed-capacity sliding batch window of no less than 50 durian fruits, sample data was continuously collected during the continuous operation of the sorting line. The automatic grading results of the network model were compared window by window with the actual labels from manual destructive sampling to generate a refined confusion matrix. Based on this, the two core error indicators, the misclassification rate of grade A and the missed judgment rate of grade C, were calculated within the window. Simultaneously, error sequences from no less than three consecutive historical windows were selected, and the slopes of the error change trends for grades A and B were obtained through least-squares linear fitting, accurately judging the dynamic evolution of the model's grading accuracy. Based on this, a dual-layer threshold adaptive adjustment logic was implemented. Combining the error exceeding the standard state and the positive or negative trend slope, the two-level grading thresholds were dynamically optimized: for the first adjustable grading judgment threshold, the threshold was adjusted upwards, downwards, or kept steady based on the constraint of the grade A misclassification rate threshold and the error change trend; for the second adjustable grading judgment threshold, the threshold was adaptively and dynamically corrected based on the upper limit of the grade C missed judgment rate tolerance and the slope of the grade B error trend, accurately adapting to the dynamic changes in the sample distribution of the sorting line. After the threshold adjustment is completed, the iterative optimization of the threshold update step size is further implemented. To address the problem of slow convergence caused by the unchanged adjustment direction for multiple consecutive cycles, the step size is increased by a preset acceleration factor to improve convergence efficiency. To address the problem of frequent oscillations and reversals in the adjustment direction, the step size is reduced by a decay factor to improve model stability. The step size is strictly constrained to always be within the preset upper and lower limits. When there are no abnormal working conditions, the original step size parameter is kept unchanged to ensure the efficiency and stability of model optimization.The system then retrieves the latest updated two-level judgment thresholds from the cache and outputs maturity labels according to the hierarchical priority judgment logic: it first compares the predicted probability of grade A of the sample with the first adjustable grading judgment threshold. If the threshold is met, it outputs the grade A label and terminates the judgment; if the threshold is not met, it compares the predicted probability of grade B with the second adjustable grading judgment threshold. If the threshold is met, it outputs the grade B label and terminates the judgment; otherwise, it is uniformly judged as grade C and the corresponding label is output, thus completing the intelligent identification of the maturity level of a single fruit. Finally, the system completes the integration and standardized report generation of all-dimensional data. It links and matches the measured weight of each durian, the maturity level label output by the model, and the three core internal health indicators obtained by the model's quantitative inference: the area of ​​moldy area, the volume of the fruit cavity, and the degree of flesh fullness. It unifies the data format, standardizes the indicator dimensions and statistical standards, and removes abnormal and invalid data. Finally, it integrates and generates a standardized structured data report containing physical parameters of each fruit, maturity grading results, and internal health quality parameters. This not only realizes the high-precision intelligent grading function of the cross-modal deep learning network, but also completes the standardized accumulation of sorting data.

[0032] like Figure 3 The diagram shows a structural schematic of a deep learning-based durian maturity level identification system provided in this application embodiment. It includes: a multi-source sensor data synchronous acquisition module, a dynamic threshold correction module, a hierarchical classification decision quantization output module, and a standardized report generation module. The multi-source sensor data synchronous acquisition module is used to acquire multi-band radio frequency dielectric imaging and hyperspectral images of the durian to be tested and perform noise reduction preprocessing, while simultaneously acquiring single-fruit weight sensor data of the durian to be tested. The dynamic threshold correction module is used to construct a cross-modal deep learning feature extraction network, inputting the noise-reduced multi-band radio frequency dielectric imaging and hyperspectral images into the cross-modal deep learning feature extraction network, and determining whether to perform dynamic threshold truncation correction. If yes, it outputs a fused, unified-dimensional durian quality deep feature vector after correction; otherwise, it... The system directly outputs a unified dimension durian quality depth feature vector after fusion. A hierarchical classification and quantification output module is used to pre-build a hierarchical deep learning classification model. The durian quality depth feature vector is input into the hierarchical deep learning classification model, and it determines whether to perform dynamic adjustment of the step size adaptive grading threshold. If so, it retrieves the appropriate adjustable grading threshold after adjustment; otherwise, it retrieves the appropriate adjustable grading threshold and outputs the maturity level label corresponding to the durian under test. Simultaneously, it quantifies and outputs internal health indicators such as the area of ​​moldy regions, the volume of the fruit cavity, and the degree of flesh fullness. A standardized report generation module is used to correlate and integrate single-fruit weight sensor data, maturity level labels, and the output internal health indicators such as the area of ​​moldy regions, the volume of the fruit cavity, and the degree of flesh fullness to generate a structured and standardized data report.

[0033] In this embodiment, independent modality-specific coding branches are designed for the multi-band radio frequency dielectric imaging map and hyperspectral image after noise reduction preprocessing. For the radio frequency imaging map, a three-dimensional convolutional neural network (3D-CNN) is used to extract the local dielectric response pattern in the frequency-spatial dimension, and dilated convolution is used to expand the receptive field to capture differences in deep tissues. For the hyperspectral image, a one-dimensional convolutional kernel with residual connections is used to extract pixel-by-pixel spectral features along the spectral dimension, while a spatial attention mechanism is introduced to enhance the response of key regions at the interface between the peel and pulp. Subsequently, the primary feature maps output by the two branches are unified to the same spatial resolution through bilinear interpolation and input into the cross-modal interactive fusion module. This module calculates the pixel-by-pixel correlation weights between radio frequency features and spectral features based on multi-head cross-attention (Cross-Modal Transformer), realizing adaptive alignment and complementary enhancement of heterogeneous information in the channel and spatial dimensions. Building upon this foundation, a contrastive loss function is introduced to constrain the intermodal consistency of the intermediate features before fusion, forcing the multimodal representations of the same durian sample to converge in the latent space. Simultaneously, a gradient inversion layer suppresses modality-specific noise, ensuring that the fused features retain the discriminative information of each modality while eliminating redundant interference. Finally, the fused high-dimensional feature map is compressed into a fixed-length one-dimensional vector using global average pooling and concatenated with a lightweight feature refinement network consisting of fully connected layers. This network employs a layer-by-layer dimensionality reduction strategy (e.g., gradually reducing from 2048 dimensions to 512 dimensions) and embeds batch normalization and ReLU activation, outputting a durian quality depth feature vector with a unified dimension (e.g., 512 dimensions). The entire network is trained end-to-end, with a loss function that integrates classification supervision loss and cross-modal reconstruction loss. During training, the fusion weights are dynamically adjusted based on the feature distribution dispersion on the validation set, ensuring that the final output feature vector possesses both modal complementarity and stable quality representation capabilities.

[0034] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope and intent of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and variations.

Claims

1. A method for identifying durian ripeness levels based on deep learning, characterized in that, Includes the following steps: Step S1: Acquire multi-band radio frequency dielectric imaging and hyperspectral images of the durian to be tested and perform noise reduction preprocessing; simultaneously acquire single fruit weight sensing data of the durian to be tested. Step S2: Construct a cross-modal deep learning feature extraction network. Input the multi-band radio frequency dielectric imaging image and hyperspectral image after noise reduction preprocessing into the cross-modal deep learning feature extraction network, and determine whether to perform dynamic threshold truncation correction. If yes, output the fused durian quality depth feature vector of unified dimension after correction. If no, directly output the fused durian quality depth feature vector of unified dimension. Step S3: Pre-build a hierarchical deep learning classification model, input the deep feature vector of durian quality into the hierarchical deep learning classification model, and determine whether to perform dynamic adjustment of the step size adaptive grading threshold. If yes, then retrieve the appropriate adjustable grading judgment threshold after adjustment. If no, retrieve the appropriate adjustable grading judgment threshold, output the maturity level label corresponding to the durian to be tested, and simultaneously quantify and output the internal health quantitative indicators of moldy area, cavity volume, and flesh fullness. Step S4 involves linking and integrating the single fruit weight sensor data, maturity grade labels, and internal health quantitative indicators such as moldy area, fruit cavity volume, and fruit flesh fullness to generate a structured and standardized data report.

2. The method for identifying durian ripeness levels based on deep learning as described in claim 1, characterized in that: The specific process for determining whether to perform dynamic threshold truncation correction is as follows: Obtain the single-band spectral reflectance pixel value matrix under the preset feature band in the hyperspectral image. The single-band spectral reflectance pixel value represents the quantized value output after analog-to-digital conversion of the ratio of reflected light intensity to incident light intensity at each spatial pixel coordinate in the hyperspectral image data cube under the preset feature band. The upper and lower limit truncation thresholds of reflectance are dynamically adjusted based on the pixel value of single-band spectral reflectance. The upper and lower limit truncation thresholds of reflectance include a preset upper limit threshold of reflectance and a preset lower limit threshold of reflectance, which together constitute the upper and lower boundaries for filtering the effective numerical range of spectral reflectance. The reflectance values ​​of all effective spatial pixels within the effective peel area of ​​the durian to be tested after noise reduction preprocessing are statistically analyzed under the preset characteristic band. The arithmetic mean of reflectance is calculated, and an initial upper and lower limit threshold for reflectance is set. The initial upper and lower limit thresholds for reflectance are dynamically adjusted by a preset adjustable scaling coefficient. The preset adjustable scaling coefficient is used to control the width of the truncation boundary relative to the distribution of the arithmetic mean of reflectance. The distribution skewness coefficient of reflectance values ​​within the current effective area is calculated. The distribution skewness coefficient is used to quantitatively describe the degree of symmetry shift of the distribution of single-band spectral reflectance pixel values ​​of the current batch relative to the normal distribution.

3. The method for identifying durian ripeness levels based on deep learning as described in claim 2, characterized in that: The determination of whether to perform dynamic threshold truncation correction also includes: If the distribution skewness coefficient is greater than the preset positive skewness tolerance threshold, the adjustable scale coefficient is reduced according to the preset attenuation factor to tighten the upper limit cutoff range and widen the lower limit cutoff range, thereby adaptively eliminating the high-brightness abnormal pixels introduced by specular reflection. If the distribution skewness coefficient is less than the preset negative skewness tolerance threshold, the adjustable scale coefficient is increased according to the preset magnification factor to widen the upper limit cutoff range and tighten the lower limit cutoff range, thereby adaptively eliminating low-value abnormal pixels introduced by shadow occlusion. If the distribution skewness coefficient is between the positive tolerance threshold and the negative tolerance threshold, then the current adjustable scaling coefficient remains unchanged; Based on the adjusted adjustable scale coefficient, the final upper limit cutoff threshold and the final lower limit cutoff threshold are determined.

4. The method for identifying durian ripeness levels based on deep learning as described in claim 3, characterized in that: The specific process for determining the final upper limit cutoff threshold and the final lower limit cutoff threshold is as follows: Traverse each pixel value in the single-band spectral reflectance pixel value matrix and perform a truncation operation: if the pixel value is greater than the upper limit truncation threshold of reflectance, reset the pixel value to the upper limit truncation threshold of reflectance, or directly mark the pixel as an invalid pixel and remove it. If a pixel value is less than the lower limit of reflectivity truncation threshold, then the pixel value is reset to the lower limit of reflectivity truncation threshold, or the pixel is directly marked as an invalid pixel and removed. If the pixel value is between the lower and upper reflectance cutoff thresholds, then the current pixel value remains unchanged. The output is a standardized single-band spectral reflectance pixel value matrix after truncation. All valid pixel values ​​in the standardized single-band spectral reflectance pixel value matrix are constrained within the adaptively adjusted upper and lower truncation threshold range.

5. The method for identifying durian ripeness levels based on deep learning as described in claim 1, characterized in that: The specific process for determining whether to perform step-size adaptive hierarchical threshold dynamic adjustment is as follows: A first adjustable grading threshold is preset between grade A and grade B to distinguish between grade A and grade B, and a second adjustable grading threshold is preset between grade B and grade C to distinguish between grade B and grade C. The adjustable grading judgment threshold is dynamically adjusted based on the threshold update step size. The adjustable grading judgment threshold refers to the critical value of the probability score between adjacent maturity levels in the output layer of the hierarchical deep learning classification model, including the first adjustable grading judgment threshold and the second adjustable grading judgment threshold. When the probability value of grade A output by the model is greater than or equal to the first adjustable grade determination threshold, the sample is determined to be grade A; when the probability value of grade A is less than the first adjustable grade determination threshold and the probability value of grade B is greater than or equal to the second adjustable grade determination threshold, the sample is determined to be grade B. When the probability value of grade B is less than the second adjustable grading threshold, the sample is determined to be grade C. The adjustable grading threshold is used to convert the continuous probability distribution of the output into discrete multi-level labels. The threshold update step size is defined as the numerical unit for incremental adjustment of the adjustable grading judgment threshold within each iteration adjustment cycle, including a first threshold update step size for the first adjustable grading judgment threshold and a second threshold update step size for the second adjustable grading judgment threshold. The threshold update step size is used to control the speed and magnitude of threshold change during each adjustment process.

6. The method for identifying durian ripeness levels based on deep learning as described in claim 5, characterized in that: The determination of whether to perform step-size adaptive hierarchical threshold dynamic adjustment also includes: Step 1: During the continuous operation of the sorting line, a fixed number of durian fruits are used as a sliding batch window. The confusion matrix between the model's grading judgment result and the actual destructive verification label is calculated within the current window. Based on the confusion matrix, the false positive rate for Grade A and the false negative rate for Grade C are calculated respectively. The false positive rate for Grade A is defined as the ratio of the number of samples in the current window that are judged as Grade A by the model but are actually verified as not Grade A to the total number of samples in the window. The false negative rate for Grade C is defined as the ratio of the number of samples in the current window that are actually Grade C but are judged as not Grade C by the model to the total number of samples in the window. Step 2: Obtain the misclassification rate sequence of level A and the omission rate sequence of level C for continuous windows, and calculate the slope of the trend of level A and the slope of the trend of level B respectively. The slope of the trend is obtained by linearly fitting the misclassification rate / omission rate sequence of the current window and the previous window using the least squares method. Step 3: Based on the comparison between the Level A misjudgment rate and the preset Level A misjudgment tolerance upper limit, and the sign of the slope of the Level A change trend, determine the adjustment direction and adjustment amount of the first adjustable classification judgment threshold, specifically as follows: If the misjudgment rate of Class A is greater than the preset tolerance limit for misjudgment of Class A and the slope of the change trend of Class A is greater than or equal to 0, then the adjustment is made in the direction of increasing the first adjustable classification judgment threshold according to the current first threshold update step size. That is, the adjusted first adjustable classification judgment threshold is the sum of the initial first adjustable classification judgment threshold and the first threshold update step size. If the misjudgment rate of Class A is less than or equal to the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of change of Class A is less than 0, then the adjustment is made in the direction of reducing the first adjustable classification judgment threshold according to the current first threshold update step size. That is, the adjusted first adjustable classification judgment threshold is the difference between the initial first adjustable classification judgment threshold and the first threshold update step size. If the misjudgment rate of Class A is less than or equal to the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of Class A is greater than or equal to 0, or if the misjudgment rate of Class A is greater than the preset upper limit of tolerance for misjudgment of Class A and the slope of the trend of Class A is less than 0, then the first adjustable classification judgment threshold remains unchanged.

7. The method for identifying durian ripeness levels based on deep learning as described in claim 6, characterized in that: The determination of the adjustment direction and adjustment amount of the first adjustable grading threshold also includes: Step four: Based on the comparison between the Level C false negative rate and the preset Level C false negative tolerance upper limit, and the sign of the slope of the Level B change trend, determine the adjustment direction and adjustment amount of the second adjustable grading judgment threshold, specifically as follows: If the C-level false negative rate is greater than the preset C-level false negative tolerance limit and the slope of the B-level change trend is greater than or equal to 0, then the adjustment is made in the direction of reducing the second adjustable classification judgment threshold according to the current second threshold update step size. That is, the adjusted second adjustable classification judgment threshold is the difference between the initial second adjustable classification judgment threshold and the second threshold update step size. If the C-level false negative rate is less than or equal to the preset C-level false negative tolerance limit and the slope of the B-level change trend is less than 0, then the second adjustable classification judgment threshold will be adjusted in the direction of increasing the second adjustable classification judgment threshold according to the current second threshold update step size. That is, the adjusted second adjustable classification judgment threshold is the sum of the initial second adjustable classification judgment threshold and the second threshold update step size. If the C-level false negative rate is less than or equal to the preset C-level false negative tolerance limit and the slope of the B-level change trend is greater than or equal to 0, or if the C-level false negative rate is greater than the preset C-level false negative tolerance limit and the slope of the B-level change trend is less than 0, then the second adjustable classification judgment threshold remains unchanged. Step 5: After executing steps 3 and 4, dynamically update the first threshold update step size and the second threshold update step size.

8. The method for identifying durian ripeness levels based on deep learning as described in claim 7, characterized in that: The specific process for dynamically updating the first threshold update step size and the second threshold update step size is as follows: The number of times the adjustment direction of the first adjustable grading judgment threshold and the second adjustable grading judgment threshold changes within multiple consecutive adjustment cycles is counted. If the adjustment direction does not change within consecutive adjustment cycles, it indicates that the threshold convergence speed is insufficient at the current step size. Then, the corresponding threshold update step size is increased according to the preset acceleration factor. That is, the adjusted first threshold update step size is the product of the first threshold update step size and the preset acceleration factor, or the adjusted second threshold update step size is the product of the second threshold update step size and the preset acceleration factor. And the adjusted step size does not exceed the preset maximum step size upper limit. If the number of times the adjustment direction reverses within a continuous adjustment period exceeds the preset oscillation tolerance number, the corresponding threshold update step size is reduced according to the preset attenuation factor. That is, the adjusted first threshold update step size is the product of the first threshold update step size and the preset attenuation factor, or the adjusted second threshold update step size is the product of the second threshold update step size and the preset attenuation factor, and the adjusted step size is not lower than the preset minimum step size lower limit. If the change in adjustment direction does not trigger any of the above conditions, the current threshold update step size remains unchanged; Step 6: Store the adjusted final adjustable grading threshold, the first adjustable grading threshold, and the second adjustable grading threshold into the cache and apply them to the real-time grading determination of the next sliding batch window.

9. The method for identifying durian ripeness levels based on deep learning as described in claim 1, characterized in that: The specific process for outputting the maturity level label corresponding to the durian to be tested is as follows: Retrieve the first adjustable grading threshold and the second adjustable grading threshold at the current moment. If the probability value of grade A is greater than or equal to the first adjustable grading threshold, directly determine that the durian to be tested is grade A, output the maturity level label A, and terminate the subsequent grading process. If the probability value of grade A is less than the first adjustable grading threshold, then the probability value of grade B is further compared with the second adjustable grading threshold: if the probability value of grade B is greater than or equal to the second adjustable grading threshold, then the durian to be tested is determined to be grade B, the maturity level label B is output, and the subsequent judgment process is terminated. If the probability value of grade A is less than the first adjustable grading threshold and the probability value of grade B is less than the second adjustable grading threshold, then the durian to be tested is determined to be grade C, and the maturity level label C is output.

10. A system applying the deep learning-based method for identifying durian ripeness levels as described in any one of claims 1-9, characterized in that, include: Multi-source sensor data synchronous acquisition module, dynamic threshold correction module, hierarchical classification decision quantization output module, and standardized report generation module; The multi-source sensor data synchronous acquisition module is used to acquire multi-band radio frequency dielectric imaging and hyperspectral images of the durian to be tested and perform noise reduction preprocessing, and synchronously acquire single fruit weight sensor data of the durian to be tested. The dynamic threshold correction module is used to construct a cross-modal deep learning feature extraction network. The multi-band radio frequency dielectric imaging map and hyperspectral image after noise reduction preprocessing are input into the cross-modal deep learning feature extraction network, and it is determined whether to perform dynamic threshold truncation correction. If yes, the fused durian quality depth feature vector of unified dimension is output after correction; otherwise, the fused durian quality depth feature vector of unified dimension is directly output. The hierarchical classification judgment and quantitative output module is used to pre-build a hierarchical deep learning classification model, input the durian quality deep feature vector into the hierarchical deep learning classification model, and determine whether to perform step size adaptive grading threshold dynamic adjustment. If yes, it retrieves the appropriate grading judgment threshold after adjustment; otherwise, it retrieves the appropriate grading judgment threshold and outputs the maturity level label corresponding to the durian to be tested, and simultaneously quantifies and outputs the internal health quantitative indicators of moldy area area, cavity volume, and flesh fullness. The standardized report generation module is used to integrate and correlate single fruit weight sensor data, maturity grade labels, and internal health quantitative indicators such as moldy area, fruit cavity volume, and fruit flesh fullness to generate structured and standardized data reports.

Images