A machine learning based fruit appearance defect detection and grading system

Abstract

This invention discloses a machine learning-based fruit appearance defect detection and grading system, relating to the fields of machine vision and agricultural product quality inspection. Existing technologies require re-labeling defect data for the target variety for cross-variety detection, or the transfer effect is unstable. To address this problem, this invention decouples hyperspectral images into variety background latent variables and defect perturbation latent variables using a causal decoupling autoencoder. Based on the source variety defect perturbation latent variables, a prototype library is constructed, enabling cross-variety detection of the target variety without requiring labeled samples of the target variety's defects, and outputting the quality grade. This invention can be applied to the field of automatic agricultural product quality inspection and grading.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and agricultural product quality inspection technology, and more specifically, to a fruit appearance defect detection and grading system based on machine learning. Background Technology

[0002] Fruit appearance defect detection is a crucial step in the commercial processing of agricultural products. Differences in the pigment composition of the peel, cuticle thickness, and microstructure of the waxy layer among different fruit varieties lead to a systematic shift in the spectral response of the same type of defect across different varieties.

[0003] Chinese patent application CN120997825A discloses a machine vision-based method and system for jujube appearance inspection and grading. It acquires spectral and morphological data of samples using a multi-view hyperspectral imaging system and a 3D reconstruction algorithm, and utilizes a dynamic attention network for multimodal feature fusion and grading. This approach involves collecting labeled data for each variety and training the detection model independently. When a new fruit variety needs to be detected, a large number of labeled defective samples need to be re-collected and the model retrained.

[0004] Chinese patent application CN112686885A discloses a method and system for detecting defects in fruit peels. It determines the presence of defects in the fruit peel by using depth image processing and segmentation of the fruit stalk and calyx regions, combined with the number of candidate defect regions. This method is also specific to a particular variety and does not address cross-variety migration mechanisms.

[0005] Some studies have attempted to align the data distributions of different varieties in the feature space using domain adaptation methods, but the training process is unstable, and treating variety differences as noise may result in the loss of effective variety information. Style transfer methods use generative adversarial networks to convert target variety images into the style of the source variety, but the generated hyperspectral images are difficult to guarantee in terms of spectral fidelity, and defective biochemical features may be modified.

[0006] Therefore, how to achieve cross-variety detection of fruit appearance defects in multi-variety scenarios without collecting a large amount of defect labeling data for the target variety is a technical problem that urgently needs to be solved in this field. Therefore, a machine learning-based fruit appearance defect detection and grading system is proposed to address the above problems. Summary of the Invention

[0007] This invention provides a fruit appearance defect detection and grading system based on machine learning, which aims to solve the problems in the prior art where defect data needs to be re-labeled for the target variety or the migration effect is unstable when detecting across varieties.

[0008] To achieve the above objectives, in a first aspect, the present invention provides a fruit appearance defect detection and grading system based on machine learning. The system decouples a hyperspectral image into variety background latent variables and defect perturbation latent variables using a causal decoupling autoencoder to achieve cross-variety detection, including: The data acquisition and labeling module is used to acquire hyperspectral images of source variety fruits (the source variety refers to a fruit variety for which defect labeling data has been accumulated), and to perform causal pairing labeling operations on the hyperspectral images. The causal pairing labeling operations include: labeling defective regions in the hyperspectral images pixel by pixel, and for each labeled defective region, selecting the local sub-region with the smallest spectral variance in the normal peel region adjacent to the defective region by searching through a sliding window, and using the selected local sub-region as the causal pairing reference region corresponding to the defective region. A causal decoupling autoencoder module is used to decompose the input hyperspectral image into a variety background latent variable representing fruit variety attributes and a defect perturbation latent variable representing spectral perturbations caused by defects. The causal decoupling autoencoder module includes a variety background encoder, an illumination encoder, a defect encoder, and an additive decoder composed of a background reconstruction submodule and a perturbation modulation submodule. The reconstructed image output by the additive decoder is Y=A+E, where A is the intrinsic spectral image output by the background reconstruction submodule, and E is the spectral perturbation image output by the perturbation modulation submodule. The causal decoupling autoencoder module is trained, and the training loss includes a reconstruction loss term, a Kolb-Leibler divergence regularization term, a causal independence loss term, and a defect semantic contrast loss term. The causal independence loss term includes a mutual information minimization regularization subterm and a causal pairing constraint loss subterm, where the causal pairing constraint loss subterm is the sum of the defect perturbation latent variables corresponding to all causal pairing reference region pixels in the current batch during training. The mean of the norm; The defect perturbation prototype library construction module is used to input the labeled defect regions of the source variety into the trained causal decoupled autoencoder, extract the defect perturbation latent variables of the defect pixels, and calculate the prototype center vector and the original intra-class sample covariance matrix of each defect category to construct the defect perturbation prototype library; the defect perturbation prototype library supports incremental updates based on newly added source variety defect labeling data that has been manually confirmed. The cross-variety detection module is used to input the hyperspectral image of the fruit to be detected of the target variety into the defect encoder, extract the defect perturbation latent variables pixel by pixel, and then... The norm is compared with a preset significance threshold to locate candidate defect regions; and for each pixel within the candidate defect region, the squared Mahalanobis distance between the latent variable of the defect perturbation of that pixel and each category of prototypes in the defect perturbation prototype library is calculated. and its normalized distance The defect category with the smallest normalized distance that is below the corresponding category statistical threshold is determined as the defect label for that pixel; if the normalized distance between that pixel and all known defect categories exceeds their respective category statistical thresholds (i.e., ... If the pixel is an unknown abnormal region, then mark it as such. The grading output module is used to calculate the proportion of defect area to the effective detection surface area of ​​the fruit based on the defect detection results, and output the quality grade of the fruit according to the grading decision rules.

[0009] Secondly, this invention provides a method for detecting and grading fruit appearance defects based on machine learning. The method utilizes labeled defect data of source varieties to train a causal decoupled autoencoder and constructs a defect perturbation prototype library to perform cross-variety detection on target varieties, including: Acquire a hyperspectral image of the source variety fruit and perform a causal pairing annotation operation on the hyperspectral image; the causal pairing annotation operation includes: annotating the defective regions in the hyperspectral image pixel by pixel, and for each annotated defective region, in the normal peel region adjacent to the defective region, searching through a sliding window to select the local sub-region with the smallest spectral variance, and using the selected local sub-region as the causal pairing reference region corresponding to the defective region; A causal decoupling autoencoder is trained using labeled source variety data. This autoencoder decomposes the input hyperspectral image into variety background latent variables and defect perturbation latent variables. The combined loss function used in its training includes a reconstruction loss term, a Koolbek-Leibler divergence regularization term, a causal independence loss term, and a defect semantic contrast loss term. The causal independence loss term includes a mutual information minimization regularization sub-term and a causal pairing constraint loss sub-term. The causal pairing constraint loss sub-term is calculated based on the defect perturbation latent variables corresponding to all causal pairing reference region pixels within the current batch during training. The mean of the norm; Based on the trained causal decoupled autoencoder, the defect perturbation latent variables of the defect pixels of the source variety are extracted, and the prototype center vector and the original intra-class sample covariance matrix of each defect are calculated according to the defect category to construct a defect perturbation prototype library. The hyperspectral image of the target variety's fruit to be tested is input into the defect encoder in the causal decoupled autoencoder. Pixel-by-pixel defect perturbation latent variables are extracted. The defect perturbation latent variables of each pixel are then processed... The norm is compared with a preset significance threshold to locate candidate regions for defects; For each pixel within the defect candidate region, calculate the squared Mahalanobis distance between the pixel's defect perturbation latent variable and each category of prototypes in the defect perturbation prototype library. and its normalized distance The defect category with the smallest normalized distance and below the corresponding category statistical threshold is determined as the defect label for that pixel; if the normalized distance between that pixel and all known defect categories exceeds their respective category statistical thresholds (i.e., ... If the pixel is an unknown abnormal region, then mark it as such. Based on the defect detection results, the proportion of defect area to the effective detection surface area of ​​the fruit is calculated, and the quality grade of the fruit is output according to the grading decision rules.

[0010] Optionally, the source varieties may consist of multiple fruit varieties, and each source variety may be labeled with causal pairings and then merged into a joint training set for training.

[0011] Thirdly, the present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory, wherein the computer program, when executed by the processor, implements the method described in the second aspect above.

[0012] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the second aspect above.

[0013] The technical effects and advantages of this invention are as follows: First, this invention utilizes a causal decoupled autoencoder, employing a large receptive field spatial average pooling structure of the variety background encoder and a small receptive field convolutional structure of the defect encoder to form an asymmetric coding pathway at the architectural level. This decouples the input hyperspectral image into variety background latent variables and defect perturbation latent variables. During training, the defect perturbation latent variables corresponding to the causal pairing reference region are used... The norm, as a causal pairing constraint loss term, constrains the latent variable response of defect perturbation corresponding to the normal peel area to maintain a level close to zero vector. At the same time, by minimizing the regularization term through mutual information, the statistical correlation between the latent variable of variety background and the latent variable of defect perturbation is reduced, thereby reducing the interference of variety background and light factors on the latent variable of defect perturbation.

[0014] Secondly, the defect perturbation latent variables extracted from the labeled defect pixels of the source variety in this invention record the perturbation pattern of defects at the tissue optical property level. This perturbation pattern is mainly determined by the local tissue optical property changes caused by the defect, exhibiting low correlation with the fruit variety background and demonstrating good cross-variety clustering characteristics in the trained feature space. For the target variety's fruit to be detected, by extracting its defect perturbation latent variables and performing distance matching with the prototype center vector in the defect perturbation prototype library, effective determination of the defect type can be achieved. Only hyperspectral images of normal fruit are needed for visualization purposes; no pixel-level defect-labeled samples of the target variety are required, thus enabling cross-variety detection.

[0015] Third, the graded output module of the present invention calculates the proportion of the total defect area to the effective detection surface area of ​​the fruit based on the pixel-level defect segmentation results, and outputs the quality grade by combining the area proportion threshold judgment and the critical defect elimination judgment. The area proportion threshold is used to divide different quality grades, and the critical defect elimination judgment is used to identify serious defects that need to be directly downgraded, providing an objective quality assessment basis for the commercial processing of fruit. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall framework of the system modules provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall workflow of the fruit appearance defect detection and grading system provided in this embodiment of the invention. Figure 3 A decision-making flowchart for the hierarchical output module provided in an embodiment of the present invention; Figure 4 This is a graph showing the change in training and monitoring metrics of a causal decoupling autoencoder provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of the selection of the causal pairing reference region provided in an embodiment of the present invention; Figure 6 This is a structural diagram of a causal decoupling autoencoder provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] To facilitate understanding of the technical solution of this invention, the following key terms are clearly defined: (1) "Source variety" and "target variety": The source variety refers to the fruit variety that has accumulated a large amount of pixel-level defect annotation data, which is used to train the causal decoupling autoencoder; the target variety refers to the fruit variety that needs to be detected but has not provided defect annotation data. The source variety and the target variety are different varieties under the same fruit type, such as Fuji apple and Gala apple.

[0019] (2) "Causal pairing labeling": For each labeled defective region, a sub-region is selected as the causal pairing reference region within the adjacent normal peel region by minimizing the spectral variance criterion. The reference region and the defective region are located in the same fruit, under the same light conditions, the same variety, and the same maturity. The only difference is whether or not it is affected by the defect. Therefore, the expected value of its defect perturbation latent variable is a zero vector.

[0020] (3) "Causal Independence": In this invention, "causal independence" refers to minimizing the regularization term and the causal pairing constraint loss term through mutual information during training, thereby significantly reducing the statistical correlation between the latent variable of variety background and the latent variable of defect perturbation. This definition aims to ensure that the latent variable of defect perturbation mainly reflects the optical properties of the defect itself, and is less affected by variety background and lighting factors. The "causal independence loss" in this invention is constructed based on this engineering definition.

[0021] (4) "Defect perturbation latent variable": refers to the high-dimensional feature vector extracted by the defect encoder that characterizes the spectral perturbation caused by the defect, denoted as w. After causal decoupling training, this latent variable mainly records the perturbation pattern of the defect at the level of tissue optical properties. The perturbation patterns of the same type of defect are similar in different varieties.

[0022] (5) "Defect perturbation prototype library": refers to the prototype center vector calculated from the defect perturbation latent variables of the source variety with labeled defect pixels, according to the defect category. and the original within-class sample covariance matrix The database was subsequently built. This prototype library is used to match and classify latent variables causing defect perturbations in the target variety.

[0023] (6) "Effective Detectable Surface Area of ​​Fruit": In this invention, it refers to the number of effective pixels obtained after segmenting the fruit region in the hyperspectral image. Since a single-view image cannot cover the entire surface of the fruit, this area is defined as the pixel area of ​​the visible fruit region in the image. The defect area is also counted by pixels, and the proportion of the total defect area is the ratio of the number of defect pixels to the number of effective pixels of the fruit. For scenarios requiring full surface detection, multi-view image stitching or taking the average proportion of multiple views can be used.

[0024] Example 1 The following is in conjunction with the appendix Figures 1 to 6 The technical solution of the present invention will be described in detail below.

[0025] This invention provides a machine learning-based fruit appearance defect detection and grading system. The system includes a data acquisition and annotation module, a causal decoupling autoencoder module, a defect perturbation prototype library construction module, a cross-variety detection module, and a grading output module. The specific implementation methods of each module are described below.

[0026] Data acquisition and annotation module: The data acquisition and annotation module is used to acquire hyperspectral images of the source and target varieties of fruit and to perform causal pairing annotation on the source variety images.

[0027] First, the acquisition conditions were set. A pushbroom hyperspectral imaging system was used to acquire hyperspectral images. The spectral range was 400 nm to 1000 nm, the spectral resolution was 5 nm, and each pixel contained reflectance values ​​across 121 consecutive spectral bands. The light source was arranged with symmetrical illumination at a 45-degree angle, and the acquisition environment was a darkroom. Before acquisition, standard white boards and dark current reference images were photographed for reflectance calibration. After acquisition, radiometric correction and reflectance normalization were performed on all hyperspectral images.

[0028] The source varieties are fruit varieties for which data has been accumulated and labeled. In one example, using apples, the source variety is Red Fuji apple. Samples are collected including five defect types: bruising, sunburn, russeting, bitter pit, and anthracnose, as well as normal fruits, with no fewer than 50 individual fruits for each defect type.

[0029] Causal pairing labeling includes the following two parts.

[0030] The first part involves pixel-by-pixel annotation of defect areas. Inspectors use a hyperspectral pseudo-color display mode to outline defects on the fruit surface at the pixel level, assigning a corresponding defect category label to each pixel within the defect area. During annotation, the defect outline is placed close to the actual boundary of the defect area, including all pixels whose optical properties are altered by the defect within the annotation range.

[0031] The second part involves selecting causal pairing reference areas. For each labeled defective area, a causal pairing reference area is selected within the area immediately surrounding the normal peel. This includes the following steps: Step A: Expand outwards by 3mm from the defect contour to form a ring-shaped candidate region. If the ring-shaped candidate region partially extends beyond the fruit surface boundary, only the portion within the fruit surface area is retained. In the image, 3mm is converted to pixel distance based on spatial resolution. For example, with a spatial resolution of 0.1mm / pixel, this is an outward expansion of 30 pixels.

[0032] Step B: Within the annular candidate region, a sliding window is used to scan pixel by pixel. The window size is 8 pixels × 8 pixels, and the sliding step size is 2 pixels. For each window position, first, the average spectral vector of all pixels within the window is calculated. Then, the squared Euclidean distance between the spectral vector of each pixel within the window and the average spectral vector is calculated. Finally, the mean of these squared distances is taken as the average spectral variance of the window.

[0033] Step C: After traversing all valid window positions within the annular candidate region, select the window with the smallest average spectral variance as the causal pairing reference region for the defect region, and record the spatial position mask of the reference region.

[0034] Boundary handling: If the annular candidate region contains other defects, fruit stems, shadows, or high-reflection areas, these areas are excluded from the candidate region to avoid interference with the reference area. If a complete 8-pixel × 8-pixel sliding window cannot be formed within the excluded candidate region (i.e., the effective area is insufficient), the width of the annular candidate region is dynamically increased (e.g., from 3mm to 5mm), or a normal peel area on the fruit surface far from the defective area is used as a substitute reference area. If an effective causal pairing reference area still cannot be found, the defective area does not participate in the subsequent calculation of the causal pairing constraint loss term, and only other loss terms are used for training.

[0035] The causal pairing reference region and the defect region originate from the same fruit surface, are under the same light conditions, are the same variety, and are at the same maturity level. The only difference is whether or not they are affected by defects. Therefore, the expected value of the latent variable of defect perturbation is a zero vector.

[0036] The target variety is a fruit variety different from the source variety. In one example, the target variety is Gala apple. Only fruit samples confirmed by manual visual inspection to be normal in appearance are required for the target variety, with a minimum of 100 individual fruits. These normal fruit samples are used to extract background priors and illumination priors after training and before cross-variety detection (see step 401 for details). The background priors and illumination priors are only used to assist in visualizing the detection results and do not participate in the localization of defect candidate regions or the determination of defect categories. The localization of defect candidate regions and the determination of defect categories are entirely based on matching the defect perturbation prototype library. Therefore, the target variety can complete cross-variety detection without any pixel-level defect annotation data. The collection conditions are consistent with those of the source variety.

[0037] Causal decoupling autoencoder module: The causal decoupling autoencoder module is used to decouple the input hyperspectral image into variety background latent variables and defect perturbation latent variables. This module includes a variety background encoder, an illumination encoder, a defect encoder, and an additive decoder.

[0038] The input hyperspectral image is denoted as X, with dimensions of H×W×C, where H and W are spatial dimensions, and C is the number of spectral bands (in this embodiment, C=121).

[0039] The structure and hidden variable dimensions of each encoder are as follows: Variety Background Encoder: Used to extract a globally smoothed spectral background reflecting the spectral baseline of the fruit peel. The encoder's structure is as follows: Global adaptive spatial average pooling is independently performed on each spectral band of X, outputting an 8×8 downsampled feature map; the downsampled feature map is flattened into a one-dimensional vector; this vector is then passed through three fully connected layers with output dimensions of 512, 256, and 128 respectively. The first two layers use linear rectified unit activation functions, and the third layer outputs the mean of the variety background latent variable u. Sum of logarithmic variance All dimensions are 128. During the training phase, u is obtained through reparameterized sampling; during the inference phase, the mean is directly output. As u.

[0040] Lighting encoder: Used to encode the global scaling factor of scene lighting intensity. The encoder's structure is as follows: The input X is averaged along the spectral dimension to obtain a single-channel spatial grayscale image; global average pooling is performed on this grayscale image to obtain a 1D global statistic; this is passed through two fully connected layers. The first layer outputs a 16-dimensional value (activated by linear rectified units), and the second layer outputs the mean of the lighting latent variable v. Sum of logarithmic variance All dimensions are 16. During the training phase, v is obtained through reparameterized sampling; during the inference phase, the mean is directly output. As v.

[0041] Defect Encoder: Used to extract local spectral abrupt changes caused by defects. The encoder's structure consists of three 2D convolutional layers, each with a 3×3 kernel and a stride of 1, using reflection padding to maintain spatial resolution. The three convolutional layers have C input channels and 64, 128, and 128 output channels respectively. Each convolutional layer is followed by a batch normalization layer and a Gaussian error linear unit activation function. Local Contrast Pooling Layer: This layer takes the output of the third convolutional layer as input and has a size of H×W×128. For each spatial location and each channel, it takes 9 pixel values ​​within a 3×3 spatial neighborhood centered at that location, calculates the neighborhood mean, then calculates the difference between each pixel value and the neighborhood mean, and finally calculates the square root of the sum of these squared differences (i.e., the sum of the squared differences within the neighborhood). The norm is used as the output at that position. This operation is performed independently for each channel, with the output space size remaining unchanged and the number of channels still being 128. Finally, the mean of the defect perturbation latent variable w is output after a 1×1 convolution. Sum of logarithmic variance The channel count is 128, and the spatial size remains H×W. During the training phase, w is obtained through reparameterized sampling; during the inference phase, the mean is directly output. As a latent variable w for defect perturbation, no random sampling is performed.

[0042] For pixels at image boundaries, their 3×3 neighborhood may extend beyond the image area. This embodiment uses reflection filling to expand the boundaries, ensuring that the complete neighborhood can be calculated for all pixels.

[0043] The three encoders run in parallel and do not share parameters with each other.

[0044] Additive decoder: includes background reconstruction submodule and perturbation modulation submodule.

[0045] The background reconstruction submodule takes u and v as input. u (128-dimensional) is mapped to 256-dimensionality via a fully connected layer, and v (16-dimensional) is mapped to 64-dimensionality via a fully connected layer. The two mapped vectors are then concatenated along the feature dimensions to obtain a 320-dimensional vector. This vector is then reshaped into an initial feature map with an initial spatial size of [insert initial size here]. (For example , The number of channels is 320. Then, an upsampling network consisting of four transposed convolutional layers is used. Each transposed convolutional layer has a kernel size of 3×3 and a stride of 2, with output channels of 128, 64, 32, and C respectively. Each layer is followed by a batch normalization layer and a linear rectified unit activation function. After upsampling, the initial size is adjusted or an interpolation layer is added after the last layer to make the output size consistent with the input size H×W, ultimately outputting an intrinsic spectral image A with a size of H×W×C.

[0046] The perturbation modulation submodule takes w as input and transforms the channels through two 1×1 convolution layers, with the number of channels being 256 and C respectively. Each layer is followed by a batch normalization layer and a linear rectified unit activation function, outputting a spectral perturbation map caused by defects. The dimensions are H×W×C.

[0047] The final reconstructed image Y is obtained by adding A and E pixel by pixel, i.e., Y = A + E. Here, A represents the background spectrum of the variety when there are no defects, and E represents the spectral perturbations caused by defects. The dimensions of A and E are the same as the input X, both being H × W × C.

[0048] The specific training methods are as follows: The causal decoupling autoencoder module is trained using labeled source variety data. The training data consists of hyperspectral images of the source varieties with causal pairing annotations. Each training sample includes a hyperspectral image X, pixel-level annotations of defect regions, and a spatial location mask for the causal pairing reference area. Before training, the network parameters are randomly initialized.

[0049] The total training loss function consists of the reconstruction loss term. Korbeck-Leibler divergence regularization term causal independence loss item Loss terms compared with defect semantics The weighted summation consists of:

[0050] in, The definitions are as follows: (1) Reconstruction loss : Calculate the mean square error between the input hyperspectral image X and the reconstructed image Y, i.e.

[0051] in This represents the Frobenius norm.

[0052] Kulbeck-Leibler divergence regularization term The approximate posterior distribution of each latent variable is constrained to approximate the standard Gaussian prior. In this embodiment, the variety background encoder, illumination encoder, and defect encoder all output the mean value of their respective latent variables. Sum of logarithmic variance Sampling is performed using a reparameterization technique. The Kulbeck-Leibler divergence is calculated using the following formula: in, For standard Gaussian priors, Let be the hidden vector of the defect perturbation at spatial location (i, j).

[0053] (3) Mutual information minimization regularization sub-item This is achieved through an independent mutual information estimation network, used to minimize the mutual information between the latent variable u of the variety background and the latent variable w of the defect perturbation. The mutual information minimization regularization term and the causal pairing constraint loss term together constitute the causal independence loss term, used to achieve the aforementioned definition of "causal independence," that is, to reduce the statistical correlation between the latent variable of the variety background and the latent variable of the defect perturbation. The specific implementation method is described below.

[0054] (4) Causal pairing constraint loss sub-item : Calculate the latent variable w of the defect perturbation corresponding to all causal paired reference region pixels in the current training batch. The mean of the norm, i.e.

[0055] in This is the set of all causal pairing reference region pixels in the current batch. The dimension is d (d=128 in this embodiment). For vectors of Norm. This loss term forces the model to output a near-zero latent variable w for defect perturbations in normal peel regions. It should be noted that... Only pixels within the causal pairing reference region are calculated, not the entire normal peel region. During training, if the current batch does not contain any causal pairing reference region pixels (e.g., the batch consists entirely of normal samples), then... Take 0.

[0056] (5) Defect semantic comparison loss term Supervised comparative learning is used to analyze the latent variables of defect perturbations within the defect region, enabling similar defects to... They are close to each other in the feature space, and are different types of defects. They should move away from each other. See below for the specific formula.

[0057] In this embodiment, the weighting coefficient is taken as: =1.0, =0.01, =0.1, =0.5, =1.0, =10.0.

[0058] The above coefficients can be adjusted appropriately according to the magnitude of the loss during the training process.

[0059] Mutual information minimization regularization sub-item The implementation method is as follows: An independent mutual information estimation network is constructed. This network is a three-layer fully connected network with output dimensions of 256, 128, and 1 for each layer, respectively. The first two layers use linear rectified unit activation functions. The latent variable w of defect perturbation is meand along the spatial dimension (resulting in a vector with the same dimension as u), and then concatenated with the latent variable u of variety background along the feature dimension, serving as the input to the mutual information estimation network. The mutual information estimation network outputs a lower bound estimate of the mutual information between u and w.

[0060] The causal decoupling autoencoder and the mutual information estimation network employ an alternating training strategy: Step A: Freeze the parameters of the causal decoupled autoencoder and update the mutual information estimation network parameters with the goal of maximizing the mutual information estimate; Step B: Freeze the mutual information estimation network parameters and update the parameters of the variety background encoder and defect encoder in the causal decoupled autoencoder with the objective of minimizing the mutual information estimate.

[0061] After updating the causal decoupled autoencoder parameters every 5 times, the mutual information estimation network parameters are updated once. The above ratio is an example setting and can be adjusted according to training stability. Through this adversarial training, the mutual information between u and w is forced to approach zero, thereby achieving statistical independence.

[0062] Defect semantic contrast loss term A supervised contrastive loss method is used. For each anchor pixel, the defect perturbation latent variable... (Within the defect area), other pixels within the same defect category are used as the positive sample set. All pixels of different defect categories are used as the negative sample set (including other defect pixels in the same frame as the anchor point and defect pixels in other samples within the batch). The loss function is defined as:

[0063] in: This represents the total number of valid anchor pixel values ​​in the current batch. anchor point The set of positive samples, that is, other defect pixels that belong to the same defect category as anchor point i; Let i be the set of anchor point i and all positive and negative samples (excluding anchor point itself), used for denominator normalization calculation; This is a temperature coefficient used to adjust the sharpness of the similarity distribution. A smaller value indicates a more concentrated distribution. In this embodiment, we take [value missing]. ; Represents the dot product of two vectors, where the vectors are already... Under normalization conditions, it is equivalent to cosine similarity.

[0064] When the number of positive samples for a certain type of defect within a batch is insufficient, a memory can be used to cache historical feature vectors to expand the positive samples. The memory stores the latent variable feature vectors of defect pixels from historical training batches to expand the positive and negative samples for contrastive learning. The memory size is set to 4096, and it is updated randomly after each training round. When the memory is full, an old sample is randomly discarded and the current sample is stored. If the number of positive samples for a certain type of defect in the current batch is less than 2, contrastive learning mainly relies on the features in the memory.

[0065] Training employs an adaptive moment estimation optimizer. An exemplary hyperparameter setting is as follows: Learning rate: ; Batch size: 32 (each batch contains 32 hyperspectral images and their annotations); Maximum number of training rounds: 200 rounds; Early stopping condition: Training stops if the total loss of the validation set does not decrease for 10 consecutive rounds. This value is an example setting and can be adjusted according to the actual training situation. The validation set is obtained by dividing the source variety training set in a 9:1 ratio. It does not participate in model parameter updates and is only used to monitor the training process.

[0066] Figure 4 The graph shows the changes in training monitoring metrics during the training process in this embodiment: the average perturbation norm of the causal pairing reference region, i.e., the latent variable w of the defect perturbation corresponding to the pixel in the causal pairing reference region. The mean norm; and the average perturbation norm difference between the defect region and the causal pairing reference region are defined as the latent variable of pixel defect perturbation within the defect region. The mean of the norm and the latent variable of pixel defect perturbation in the causal pairing reference area The difference between the mean norms. These metrics are used to monitor the model's ability to distinguish between defective and normal regions, and are not an independent component of the training loss. The decreasing trend of these two metrics with increasing training epochs reflects the gradual improvement in the model's ability to distinguish between defective and normal regions. When both metrics stabilize, the model training has converged.

[0067] After training, samples with causal pairing labels are taken from the source variety test set, and the w corresponding to the causal pairing reference region is calculated. Norm mean. When this mean is below a preset threshold, decoupling is considered satisfactory; if the mean is above the preset threshold, it is increased. The coefficients were retrained.

[0068] Defect Disturbance Prototype Library Building Module: The defect perturbation prototype library construction module is used to input the labeled defect regions of the source variety into the trained causal decoupled autoencoder, extract the defect perturbation latent variables of the defect pixels, calculate the prototype center vector and the original intra-class sample covariance matrix according to the defect category, and construct the defect perturbation prototype library.

[0069] Specifically, the following steps are included: Step 301: Input all labeled defect samples from the source variety training set into the causal decoupled autoencoder one by one, and run the defect encoder to extract the w vector of each spatial pixel (during the inference stage, the defect encoder directly outputs the mean). (No random sampling is performed).

[0070] Step 302: Based on the defect annotation information, retain only the w vectors corresponding to pixels located within the defect area. Store the w vector of each defect pixel in association with its defect category label.

[0071] Step 303: The defect categories include five types: bruising, sunburn, fruit rust, bitter pit, and anthracnose. For each defect category k, the w vectors of all defect pixels in all training samples of that category are aggregated. The mean of all w vectors for that category is calculated and used as the prototype center vector. Calculate the values ​​of all w vectors of this class relative to... The covariance matrix is ​​used as the original within-class sample covariance matrix. .

[0072] Step 304: Store each type of defect in the defect perturbation prototype library. ,in This is the original in-class sample covariance matrix.

[0073] Step 305, the core storage fields of the defect perturbation prototype library are configured as follows: Each type of defect is stored in the prototype library as a record entry, and each entry includes the following fields: (1) Defect category number, used to uniquely identify the defect type; (2) Source variety name or source identifier, used to record the fruit variety from which the prototype data originated; (3) The number of defective pixel samples, denoted as This is used to record the total number of defective pixels used when constructing the prototype of this class; (4) The prototype center vector, denoted as The dimension is the same as the latent variable w of the defect perturbation; (5) The original within-class sample covariance matrix, denoted as , is a d×d dimensional matrix, where d is the dimension of the latent variable of the defect perturbation; (6) Category statistical threshold, denoted as It is used to determine whether a pixel belongs to this type of defect when inspecting across varieties; (7) Regularization parameter, which records the shrinkage coefficient actually used when constructing the prototype of this class. The possible values ​​of ; (8) Feature dimension, denoted as d, is used to record the dimension of the latent variable of defect disturbance, to ensure that the dimension matches when reading prototype data from different sources; (9) Update the timestamp or version number to record the most recent update time of the entry.

[0074] The aforementioned storage fields enable the defect perturbation prototype library to have cross-variety expansion capabilities. When new source variety data is added, incremental updates can be performed based on existing prototypes.

[0075] Step 306, the incremental update rules for the defect disturbance prototype library are as follows: When the defect annotation data for newly added source varieties, confirmed by manual verification, is available, an incremental update is performed on the defect perturbation prototype library. The newly added source varieties refer to fruit varieties with newly accumulated pixel-level defect annotation data, excluding those already used to construct the defect perturbation prototype library. The specific update method is as follows: First, the labeled defect regions of the newly added source varieties, which have been manually confirmed, are input into the causal decoupled autoencoder after training to extract the latent variable of defect perturbation of defect pixels. Then, for each type of defect, let the number of samples used in the existing prototype construction be... The number of new samples is The existing prototype center vector is The mean of the new samples is Calculate the updated prototype center vector using the following weighted update formula. :

[0076] For the original within-class sample covariance matrix Calculate the updated within-class sample covariance matrix using the following weighted update formula. :

[0077] in, This is the original within-class sample covariance matrix before the update. This is the within-class sample covariance matrix for the newly added samples.

[0078] The updated total number of samples is ; The updated category statistical thresholds are recalculated based on all updated samples, taking the latent defect perturbation variables of all defect pixels in that category after the update and the updated prototype center vector. The quantiles of the squared Mahalanobis distance between them at the preset confidence level; After the update is complete, the update timestamp field in the entry will be updated synchronously.

[0079] When the number of defect samples for a new source variety is insufficient (for example, the number of new sample pixels for a certain type of defect is less than 10), incremental updates are not performed for that type of defect. Instead, the data of the new sample is recorded and updated only after sufficient samples have been accumulated.

[0080] It is strictly prohibited to use cross-variety test results that have not been manually verified to update the defect perturbation prototype library. All defect annotation data used for incremental updates must come from source variety samples that have been manually verified pixel by pixel.

[0081] Cross-variety detection module: The cross-variety detection module is used to perform cross-variety defect detection on target variety fruits using a defect perturbation prototype library (without requiring target variety defect-labeled samples). The detection process consists of three steps: background prior extraction, significance testing, and prototype matching.

[0082] Step 401: Extract background priors and illumination priors for the target variety. The hyperspectral images of all normal fruit samples of the target variety were input one by one into the causal decoupled autoencoder. The variety background encoder was run on each normal sample to extract u (the mean value was taken during the inference stage). ), run the light encoder to extract v (average value taken during inference phase) The background prior is obtained by taking the mean value of u for all normal samples, and the illumination prior is obtained by taking the mean value of v for all samples. The background prior and illumination prior are only used to assist in the visualization of the detection results and do not participate in the localization of defect candidate regions or the determination of defect categories. The localization of defect candidate regions and the determination of defect categories are entirely based on matching with the defect perturbation prototype library. In this embodiment, the background prior and illumination prior do not participate in the core process of defect detection.

[0083] Step 402: Significance test to locate candidate defect regions The hyperspectral image of the target variety's fruit to be tested is input into the defect encoder, and the pixel-by-pixel defect perturbation latent variable w is extracted (the mean value is taken during the inference stage). For each spatial location, calculate the vector w. Norm.

[0084] The preset significance threshold is based on the normal peel region w of the source variety. Norm statistical distribution setting. In the source variety training set, collect all pixels in the causal pairing reference region (normal peel). Norm values ​​yield the set The preset significance threshold is taken as the upper quantile of the distribution corresponding to the set at a preset confidence level. In one example, the 95th upper quantile of the distribution is taken as the preset significance threshold. The preset confidence level ranges from 90% to 99%. Each pixel's... The norm is compared with a preset saliency threshold. If it is greater than the threshold, the pixel is classified as a defect candidate region.

[0085] Step 403: Determine the defect category by distance matching During the source variety training phase, for each type of defect, the latent defect perturbation variable w and its prototype center vector for all defect pixels of that type are calculated. The squared Mahalanobis distance yields the distance set. In one example, the upper 99th quantile of this set is taken as the statistical threshold for the category of defects. The confidence level corresponding to this quantile ranges from 95% to 99.5%. The categorical statistical threshold... Used for subsequent normalization distance The calculation is performed to eliminate the influence of differences in feature distribution scales between different defect categories.

[0086] For each pixel within the defect candidate region, calculate the squared Mahalanobis distance between the pixel's latent defect perturbation variable w and the prototypes of various defects in the defect perturbation prototype library. Calculate one by one:

[0087] Squared Mahalanobis distance of type k defect Equal to the latent variable w of the candidate pixel defect perturbation and the center of the k-th prototype. The difference vector is first multiplied by the inverse of the original within-class sample covariance matrix. Then multiply by the transpose of the difference vector to get a scalar value. The value is a scalar; the smaller the value, the higher the degree of matching between the latent variable of the defect perturbation of the candidate pixel and the prototype of the k-th type of defect. The squared Mahalanobis distance takes into account the covariance structure of the feature vector of this type of defect.

[0088] The calculated Statistical thresholds for corresponding categories Compare them one by one. If there exists one that satisfies this condition... If a defect category meets the condition, then all defect categories that satisfy this condition are grouped into a candidate set; if no such defect category exists... If the defect category is not specified, then the candidate set is empty.

[0089] The judgment rules are as follows: If the candidate set is not empty, then a normalized distance is selected from the candidate set. The smallest defect category is used as the defect label for that pixel; If the candidate set is empty, the pixel is marked as an unknown abnormal region.

[0090] After traversing all pixels within the defect candidate region, pixel-level defect segmentation results of the target variety fruit are obtained. The segmentation results include known defect category labels, unknown abnormal region labels, and normal region labels.

[0091] Shrinkage regularization (optional): When the original within-class sample covariance matrix of a certain type of defect... Singular or its condition number exceeds 1× At this time, a shrinkage regularization is applied to the original within-class sample covariance matrix to ensure numerical stability. The condition number refers to the ratio of the largest eigenvalue to the smallest eigenvalue of the matrix. The shrinkage regularization formula is:

[0092] Where d is the dimension of the latent variable of the defect perturbation. Values , This represents the number of sample pixels for this type of defect. (Using regularization...) The squared Mahalanobis distance is calculated by replacing the original matrix.

[0093] Shrinking the regularized within-class sample covariance matrix It is the original within-class sample covariance matrix A weighted mixture with the identity matrix I. The shrinkage coefficient, Let be the trace of the matrix (the sum of its diagonal elements), and d be the feature dimension. This process ensures that the covariance matrix is ​​invertible and numerically stable, avoiding matrix singularities during Mahalanobis distance squared calculation.

[0094] Hierarchical output module: The graded output module is used to output fruit quality grades based on pixel-level defect segmentation results.

[0095] First, the pixel range of the effective detection surface of the fruit is determined. The hyperspectral image of the target variety's fruit is segmented into fruit regions. Specifically, a feature band with high contrast between the fruit and background in the hyperspectral image (e.g., a band within the range of 760nm to 800nm) is selected, and adaptive thresholding is performed on this band to obtain an initial binary mask. Morphological opening is then performed on the initial binary mask to remove background noise points, followed by morphological closing to fill small voids on the fruit surface caused by highlights or shadows, thus obtaining a binary mask for the fruit region.

[0096] Based on the binary mask of the fruit area, the fruit stalk, calyx, shadow, and high-reflectivity areas are removed. The removal of the fruit stalk and calyx areas is achieved by threshold segmentation and morphological analysis of the 824nm wavelength feature image to identify their spatial location and exclude them from the fruit mask. The removal of shadow areas involves calculating the average reflectance of each pixel across all spectral bands, identifying pixels with an average reflectance below a preset shadow threshold, and removing them. The removal of high-reflectivity areas involves detecting the presence of saturated bands (reflectance close to 100%) in the spectral reflectance curve of each pixel; if present, the pixel is identified as a high-reflectivity area and removed. After these removals, the remaining set of valid pixels constitutes the effective detection surface of the fruit.

[0097] Let c be the total number of pixels contained in the effective detection surface of the fruit after the above processing. The category label of each pixel is read from the pixel-level defect segmentation results. Pixels labeled as having known defect categories or being in unknown anomaly regions are counted as defect pixels, with a total of [number missing]. The percentage of total defect area R is calculated using the following formula:

[0098] That is, the total number of defective pixels Divide by the total number of pixels on the effective detection surface of the fruit Then multiply by 100%.

[0099] When it is necessary to obtain the defect area in physical area units, the calculation is performed based on the spatial resolution of the hyperspectral imaging system. Let the spatial resolution of the hyperspectral imaging system be s mm / pixel, then the actual physical area corresponding to a single pixel is... mm 2 The actual physical area of ​​the defect is mm 2 The actual physical area of ​​the fruit's effective detection surface is mm 2 The spatial resolution s is determined by the lens focal length, camera pixel size, and shooting distance of the hyperspectral imaging system.

[0100] For scenarios requiring full-surface inspection, a multi-view hyperspectral imaging system is used to acquire images of the fruit from multiple perspectives. For each perspective, the percentage of defect area is calculated independently using the method described above. Where i is the viewpoint number. The final percentage of fruit defect area is determined using one of the following methods: take the maximum percentage of defect area across all viewpoints, i.e. Alternatively, take the weighted average of the defect area percentages from all viewpoints, i.e. The weighting coefficient Determined based on the proportion of the fruit surface area covered by each viewpoint. The default setting is to take the maximum value to ensure that the most serious defects on the fruit surface are fully detected; when the coverage area of ​​different viewpoints varies greatly, a weighted average method can be used to more accurately reflect the overall quality.

[0101] The area percentage threshold and critical defect size threshold used in the above-mentioned grading decision rules can be adjusted according to the biological characteristics of different fruit varieties, enterprise grading standards or industry standards. The area percentage threshold and size threshold are configured as adjustable parameters.

[0102] It should be noted that the "effective detection surface area of ​​fruit" mentioned in this invention refers to the pixel area of ​​the visible fruit region in a single hyperspectral image, not the total surface area of ​​the fruit. In practical applications, the entire surface can be approximated by acquiring images from multiple perspectives and taking the maximum or average value of the defect percentage from each perspective.

[0103] Secondly, the quality level is determined through a hierarchical decision-making rule. This embodiment uses apples as an example, employing a rule that combines area percentage threshold determination with critical defect elimination determination.

[0104] Area percentage threshold determination: Three configurable thresholds are set; in one example, they are 1%, 3%, and 5%. The determination method is as follows: like It was determined to be a top-grade product; like It was determined to be a first-class product; like It was determined to be a second-grade product; like It was determined to be a defective product.

[0105] Critical Defect Elimination Criteria: When an anthrax defect label is detected, a connected component analysis is performed on the anthrax-affected area to calculate the minimum circumscribed circle diameter of each connected component. If the diameter of any connected component exceeds 5mm, it is directly judged as a defective product, regardless of its area percentage.

[0106] The above-mentioned area ratio threshold and anthracnose diameter threshold can be adjusted according to different fruit varieties, enterprise grading standards or industry standards. This embodiment is only an example.

[0107] Finally, output the quality grade of each fruit, the total defect area percentage R, and the pixel statistics of each defect type (such as the number of pixels with bruises and dents, the number of pixels with sunburn, etc.).

[0108] Multi-source variety joint training (alternative implementation method): As an alternative implementation method, the source variety can be composed of multiple fruit varieties.

[0109] In this scenario, the data acquisition and labeling module performs causal pairing labeling for each source variety, following the same labeling process as for a single source variety. During training, the causal decoupling autoencoder module merges the labeled data from each source variety into a joint training set. Each batch is randomly sampled from the joint training set, and each batch may contain samples from different source varieties.

[0110] The network structure and training loss calculation method remain unchanged. The causal pairing constraint loss term only applies to the causal pairing reference region of each sample. The defect semantic contrast loss term is executed on the joint training set. Pixels of different source varieties of the same defect category are all assigned to the same set of positive sample pairs.

[0111] After training, the defect perturbation prototype library construction module extracts defect pixels from all source varieties on the joint training set. The vectors are mixed according to defect categories, and the prototype center vector and the original intra-class sample covariance matrix are calculated. If the number of sample pixels of a certain type of defect in a certain variety is less than 10 (this number is an example threshold), then the data of this type of defect in that variety will not participate in the calculation, or a weighted fusion strategy will be adopted (the weight is proportional to the sample size). The processing flow of the cross-variety detection module and the graded output module remains unchanged.

[0112] It should be noted that the feasibility of this alternative implementation method is based on the extended applicability of the aforementioned causal decoupling autoencoder to multi-source data, and its specific performance depends on the actual application scenario and data distribution.

[0113] Feasibility description of the implementation method: The specific implementation methods of the above modules constitute the complete implementable technical solution of the present invention. The values ​​of each hyperparameter are exemplary settings and can be adjusted according to factors such as data distribution, imaging equipment parameters, and defect types in actual application scenarios, and do not constitute a limitation of the present invention.

[0114] This invention, in cross-variety detection scenarios, utilizes pre-labeled defect data from source varieties to train a causal decoupled autoencoder and constructs a defect perturbation prototype library. Target varieties can then be detected and classified without pixel-level defect labeling. The effectiveness of this technical solution relies on the causal decoupled autoencoder's ability to decouple latent variables of variety background and defect perturbation, as well as the clustering characteristics of the defect perturbation latent variables in cross-variety scenarios. Specific detection performance depends on factors such as the scale and quality of the training data, the degree of difference between varieties, and the consistency of imaging conditions.

[0115] Fruit variety suitability instructions: The above embodiments of the present invention have been described in detail using apples (Red Fuji apples as the source variety and Gala apples as the target variety) as an example, but the technical solution of the present invention is not limited to apple varieties.

[0116] The hyperspectral causal decoupling method employed in this invention is based on separating latent variables of variety background from latent variables of defect perturbation through a causal decoupling autoencoder. This allows the latent variables of defect perturbation to primarily characterize the optical property perturbation of the defect itself, rather than the variety background characteristics. This technical principle does not rely on the specific varietal characteristics of a particular fruit, but is based on the difference in reflectance between the defective region and the normal peel region in the spectral range of 400 nm to 1000 nm. This difference in spectral response is universal in various fruit epidermal defects.

[0117] For example, the method of this invention can be used for cross-variety defect detection when the epidermal defects of citrus fruits (such as oranges and grapefruits), stone fruits (such as peaches and plums), pome fruits (such as pears), and some vegetable agricultural products (such as tomatoes and cucumbers) exhibit detectable spectral feature differences in the 400nm to 1000nm spectral range.

[0118] The spectral detection principle for the above-mentioned varieties is the same as that in this embodiment. Those skilled in the art only need to adjust the spatial resolution and spectral range of the imaging system according to the specific variety to implement it, without any creative effort. The core algorithm framework (causal decoupling autoencoder training, defect perturbation latent variable extraction, defect perturbation prototype library construction and matching) remains unchanged.

[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fruit appearance defect detection and grading system based on machine learning, characterized in that, include: The data acquisition and annotation module is configured as follows: Obtain hyperspectral images of fruits from a source variety, where the source variety refers to a fruit variety for which defect labeling data has been accumulated; The defect regions in the hyperspectral image are labeled pixel by pixel; Furthermore, for each labeled defective region, in the normal peel region adjacent to the defective region, a local sub-region with the smallest spectral variance is selected by sliding window search, and the selected local sub-region is used as the causal pairing reference region corresponding to the defective region. The causal decoupling autoencoder module is configured as follows: The input hyperspectral image is decomposed into variety background latent variables to characterize fruit variety attributes and defect perturbation latent variables to characterize spectral perturbations caused by defects. The causal decoupling autoencoder module is trained using a combined loss function including a reconstruction loss term, a Kolb-Leibler divergence regularization term, a causal independence loss term, and a defect semantic contrast loss term. The causal independence loss term includes a mutual information minimization regularization sub-term and a causal pairing constraint loss sub-term. The causal pairing constraint loss sub-term is taken as the value of the latent variable of the defect perturbation corresponding to all causal pairing reference region pixels in the current batch during training. The mean of the norm; The defect perturbation prototype library building module is configured as follows: The labeled defect regions of the source variety are input into the causal decoupled autoencoder module after training to extract the latent variable of defect perturbation of defect pixels. Based on the defect category, the prototype center vector and the original in-class sample covariance matrix of each defect are calculated to construct a defect perturbation prototype library. The cross-variety detection module is configured as follows: The hyperspectral image of the target variety's fruit to be tested is input into the defect encoder in the causal decoupling autoencoder module to extract the pixel-by-pixel defect perturbation latent variable. By perturbing the latent variables of each pixel's defects The norm is compared with a preset significance threshold to locate candidate regions for defects; And, for each pixel within the defect candidate region: Calculate the statistical distance between the latent variable of the defect perturbation of the pixel and the prototypes of each defect category in the defect perturbation prototype library; The defect category with the smallest statistical distance and below the corresponding category statistical threshold is determined as the defect label for that pixel; If the statistical distance between a pixel and all known defect categories exceeds the statistical threshold of their respective categories, then the pixel is marked as an unknown abnormal region. The hierarchical output module is configured as follows: Based on the defect detection results, the proportion of defect area to the effective detection surface area of ​​the fruit is calculated, and the quality grade of the fruit is output according to the grading decision rules. The output of the data acquisition and labeling module is connected to the input of the causal decoupling self-encoder module; The output terminal of the causal decoupling autoencoder module is connected to the input terminal of the defect perturbation prototype library construction module and the input terminal of the cross-variety detection module, respectively. The output of the cross-variety detection module is connected to the input of the grading output module.

2. The fruit appearance defect detection and grading system based on machine learning according to claim 1, characterized in that, The causal decoupled autoencoder module includes a variety background encoder, an illumination encoder, a defect encoder, and an additive decoder consisting of a background reconstruction submodule and a perturbation modulation submodule. The reconstructed image Y output by the additive decoder satisfies Y=A+E, where A is the intrinsic spectral image output by the background reconstruction submodule and E is the spectral perturbation image output by the perturbation modulation submodule.

3. The fruit appearance defect detection and grading system based on machine learning according to claim 2, characterized in that, The variety background encoder is composed of spatial average pooling and fully connected layer stacking, and the receptive field covers the entire surface area of ​​the fruit; the defect encoder is composed of 3×3 convolution kernel and local contrast pooling, and the receptive field is limited to the local neighborhood of the defect.

4. The fruit appearance defect detection and grading system based on machine learning according to claim 1, characterized in that, The statistical distance is the squared Mahalanobis distance. For the k-th type of defect, the squared Mahalanobis distance... Calculate using the following formula: in, is the squared Mahalanobis distance of the k-th type of defect, and its value is a non-negative scalar. The smaller the value, the higher the degree of matching between the latent variable of the defect perturbation of the candidate pixel and the prototype of the k-th type of defect. w is the latent variable for defect perturbation of candidate pixels, with dimension d and values ​​being a real number vector; Let be the prototype center vector of the k-th type of defect, with dimension d and values ​​being a real number vector; Let be the original within-class sample covariance matrix of the k-th type of defect, which is a d×d-dimensional positive semi-definite matrix estimated from the training samples of the source variety; When the original within-class sample covariance matrix Singular or its condition number exceeds 1× At that time, for the original within-class sample covariance matrix Apply shrinkage regularization to reduce the original within-class sample covariance matrix. Reversible, the contraction regularization is calculated according to the following formula: in, Let be the in-class sample covariance matrix after shrinking and regularization, which is a d×d positive definite matrix; The shrinkage coefficient has a value of [value missing]. The value range is (0, 0.01]; The original within-class sample covariance matrix The trace of is a non-negative scalar; d represents the dimension of the latent variable causing the defect disturbance, and its value is a positive integer; I is a d×d dimensional identity matrix; The number of sample pixels for the k-th type of defect, taking a positive integer value; The condition number refers to the ratio of the largest eigenvalue to the smallest eigenvalue of a matrix; Using the shrinkage regularized within-class sample covariance matrix Replace the original within-class sample covariance matrix Calculate the squared Mahalanobis distance.

5. A method for detecting and grading fruit appearance defects based on machine learning, used to implement the system according to any one of claims 1 to 4, characterized in that, Includes the following steps: Step 1: Obtain hyperspectral images of the source variety's fruit and perform causal pairing annotation on the hyperspectral images; The causal pairing labeling operation includes: The defect regions in the hyperspectral image are labeled pixel by pixel; Furthermore, for each labeled defective region, in the normal peel region adjacent to the defective region, a local sub-region with the smallest spectral variance is selected by sliding window search, and the selected local sub-region is used as the causal pairing reference region corresponding to the defective region. Step 2: Train a causal decoupling autoencoder using the labeled source variety data; The causal decoupling autoencoder is used to decompose the input hyperspectral image into variety background latent variables and defect perturbation latent variables. The combined loss function used for its training includes a reconstruction loss term, a Kolb-Leibler divergence regularization term, a causal independence loss term, and a defect semantic contrast loss term. The causal independence loss term includes a mutual information minimization regularization sub-term and a causal pairing constraint loss sub-term. The causal pairing constraint loss sub-term is taken as the value of the latent variable of the defect perturbation corresponding to all causal pairing reference region pixels in the current batch during training. The mean of the norm; Step 3: Based on the trained causal decoupled autoencoder, extract the latent variables of defect perturbation of defect pixels of the source variety, and calculate the prototype center vector and the original in-class sample covariance matrix of each defect according to the defect category to construct a defect perturbation prototype library. Step 4: Input the hyperspectral image of the target variety's fruit to be tested into the defect encoder in the causal decoupled autoencoder, extract the pixel-by-pixel defect perturbation latent variable, and then analyze the defect perturbation latent variable of each pixel. The norm is compared with a preset significance threshold to locate candidate regions for defects; Step 5: For each pixel within the candidate defect region: The squared Mahalanobis distance between the latent variable of the defect perturbation of this pixel and the prototype center vector of each defect category in the defect perturbation prototype library. Calculate one by one; And calculate the normalized distance. The defect category with the smallest normalized distance and below the corresponding category statistical threshold is determined as the defect label for that pixel; If the normalized distance of this pixel to all known defect categories exceeds the corresponding category statistical threshold (i.e.) If the pixel is an unknown abnormal region, then mark it as such. Step 6: Based on the defect detection results, calculate the proportion of the defect area to the effective detection surface area of ​​the fruit, and output the quality grade of the fruit according to the grading decision rules.

6. The method according to claim 5, characterized in that, The selection of the causal pairing reference region includes: A ring-shaped candidate region is formed by expanding outwards by 3mm from the defect contour. Within the annular candidate area, in 8 pixels The spectral variance is calculated window by window with an 8-pixel sliding window and a step size of 2 pixels. The window with the smallest spectral variance is selected as the causal pairing reference area. If the ring-shaped candidate area extends beyond the fruit boundary or contains other defects, fruit stalks, or shaded areas, it will be excluded. If a complete sliding window cannot be formed within the candidate area, the candidate ring is expanded or a nearby normal peel area is used as an alternative reference area.

7. The method according to claim 5, characterized in that: The preset confidence level for the preset significance threshold ranges from 90% to 99%. The preset confidence level of the category statistical threshold ranges from 95% to 99.5%.

8. The method according to claim 5, characterized in that, The hierarchical decision-making rules include: The quality grade is determined by comparing the ratio of the total defect area to the effective detection surface area of ​​the fruit with at least one preset area ratio threshold. The preset area ratio threshold can be configured as one or more adjustable parameters according to the fruit variety or grading standard. Furthermore, when a preset critical defect type is detected, a connected component analysis is performed on the defect region of that critical defect type. If the size of any connected component exceeds a preset size threshold, it is directly determined to be a defective product. The preset size threshold is configured as an adjustable parameter according to the fruit variety or grading standard.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 5 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 5 to 8.

Images