Carya cathayensis kernel defect identification method based on improved ConvNeXtV2

Abstract

The invention provides a carya cathayensis kernel defect identification method based on improved ConvNeXtV2, and belongs to the field of deep learning and computer vision technologies and intelligent agricultural product processing. The method mainly comprises the steps of establishment and preprocessing of a data set, and acquisition of pecan surface defect image features. Performing data enhancement and normalization processing on the acquired image features; the basic model is improved by combining an attention module and a multi-scale fusion module, a channel attention module is embedded in a packaging mode, and a PANet bidirectional feature fusion network is constructed; training parameters are set, and the improved ConvNeXtV2 network structure is trained in combination with a training set; and testing and evaluating the trained model by using a verification set in the data set to realize high-precision detection of six common hickory nut kernel defects. According to the invention, the improved ConvNeXtV2 model effectively solves the problem of easy missing detection of the tiny texture defects of the pecan kernels in the application of pecan kernel defect classification detection, and the detection precision and robustness are significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning, computer vision technology and intelligent agricultural product processing, and in particular to a method for identifying defects in pecan kernels based on an improved ConvNeXtV2 network. Background Technology

[0002] In recent years, deep learning technology has made groundbreaking progress in the field of image recognition. Although major object detection networks (such as ResNet and EfficientNet) perform well on general classification tasks, these models often struggle to effectively distinguish subtle defect features from normal kernel wrinkles when dealing with agricultural products like pecan kernels, which have complex surface textures, leading to a high false detection rate. ConvNeXtV2, as a modern convolutional neural network, while possessing powerful feature extraction capabilities, is prone to losing small target features in deep semantic information when processing fine features such as mold and spots in extremely small areas due to its feature fusion mechanism. Therefore, developing a pecan kernel defect identification algorithm that can deeply mine fine-grained features and enhance multi-scale information fusion capabilities, thereby significantly improving the accuracy of identifying small and easily confused defects, has significant application value for ensuring food safety and improving sorting quality.

[0003] The use of machine vision technology to study the quality characteristics of agricultural products is a hot topic in agricultural product quality testing research both domestically and internationally, and it has already been widely applied in apples, pears, and grains. This paper proposes a method for identifying and detecting typical defects in pecan kernels based on machine vision and deep learning, and establishes an identification model for typical defects in pecan kernels, providing a foundation for automated sorting of pecan kernels, which has strong scientific significance and practical value. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a defect identification method for pecan kernels based on an improved ConvNeXtV2. By introducing a dynamic convolution mechanism, an SE attention module, and a PANet multi-scale feature fusion network into the ConvNeXtV2 architecture, a network model named SHT-CNv2 is constructed, which effectively solves the problem of easy missed detection of small defects and improves detection accuracy and robustness.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] A method for identifying defects in pecan kernels based on an improved ConvNeXtV2 network, comprising the following steps:

[0007] S1. Construct a pecan kernel classification dataset, including taking image data using a self-built shooting system, classifying and labeling the images, and dividing the data into a training set, a validation set, and a test set according to a preset ratio;

[0008] S2. Construct an improved ConvNeXtV2 network structure, named SHT-CNv2. Specific improvements include:

[0009] S2-1. Embed a channel attention mechanism in the backbone of the ConvNeXtV2 network, and connect an SE channel attention module after the deep convolutional layer of each convolutional block.

[0010] S2-2. Introduce a dynamic convolution mechanism in the backbone part of the ConvNeXtV2 network. In Stage 3 and Stage 4 of the backbone network, use dynamic depth convolution layers to replace the original static depth convolution layers.

[0011] S2-3. In the Neck part of the ConvNeXtV2 network, a PANet multi-scale feature fusion module is constructed. First, the number of channels of the feature maps of the last three stages output by the Backbone is unified through the channel alignment layer. A top-down feature pyramid path is constructed. The deep features are upsampled and added to the shallow features. On this basis, a bottom-up path aggregation network is constructed. The feature maps are downsampled and added to the features of the previous level.

[0012] S2-4. In the Neck and Head parts of the ConvNeXtV2 network, feature enhancement and classification structures are constructed. Enhancement layers consisting of standard convolution, batch normalization and SiLU activation functions are used to process the features at each level. The processed features are adjusted to the same size and then superimposed and fused. The fused features are weighted by the global SE attention module and then input into the classification head containing convolutional layers, global average pooling layers and fully connected layers to output the results.

[0013] S2-5. Set training parameters and adopt a phased freezing strategy. In the early stage of training, freeze the parameters of the Stem layer of the Backbone and the first two stages, and only update the weights of the deeper layers of the network and the newly added modules. After reaching the preset number of rounds, unfreeze all parameters for fine-tuning, and combine the Mixup and random erasure data augmentation strategies to train the SHT-CNv2 network.

[0014] S3. Test and evaluate the trained SHT-CNv2 model using the test set:

[0015] To achieve the above technical solution, this method first collects a large number of pecan kernel images through an imaging system and performs classification and annotation to construct a standardized training dataset. Subsequently, the ConvNeXtV2 network is specifically improved: an SE module is embedded in the backbone network to enhance channel feature filtering capabilities; dynamic convolution is introduced into the deep network to improve adaptive perception of complex defect morphologies; and a PANet fusion module is constructed to integrate multi-scale information. These improvements collectively construct the SHT-CNv2 model. Finally, training is performed using a phased freezing and data augmentation strategy, followed by testing and evaluation. This method overcomes the problems of low efficiency and inconsistent standards in traditional manual sorting, as well as the insufficient extraction of minute defect features by existing models. Thanks to the introduction of dynamic convolution and multi-scale fusion, this algorithm can significantly improve the recognition accuracy of minute defects such as spots and mold while maintaining high inference speed, meeting the needs of the pecan kernel processing industry for high-precision and automated sorting.

[0016] As a preferred embodiment of the present invention, the SE channel attention mechanism module embedded in S2-1 includes the following steps:

[0017] S2-1-1. Insert the SE attention module after the output of the deep convolutional layer of the ConvNeXtV2 Block;

[0018] S2-1-2. In the SE attention module, the input feature map is first subjected to global average pooling to compress it into a 1×1 spatial feature.

[0019] S2-1-3, The number of channels is reduced to 1 / 16 of the number of input channels through the first 1×1 convolutional layer, and then activated by the SiLU activation function;

[0020] S2-1-4. The number of channels is restored to the original dimension through the second 1×1 convolutional layer, and the channel weight coefficients are generated by the Sigmoid function.

[0021] S2-1-5. Multiply the generated weight coefficients element-wise with the original input feature map to complete the channel feature recalibration. This module, through a "compression-activation-recalibration" process, can automatically learn and strengthen the channel weights containing key defect information, while suppressing interference from irrelevant information such as background noise. This explicit channel dependency modeling allows the network to more accurately focus on defect features when processing the complex texture of pecan kernel surfaces, thus improving the model's discriminative ability.

[0022] As a preferred embodiment of the present invention, the dynamic convolution mechanism introduced in S2-2 specifically includes the following steps:

[0023] S2-2-1. Construct a routing layer, perform global average pooling on the input feature map, then reduce the dimension through the first convolutional layer, activate it with the SiLU activation function, increase the dimension to the product of the number of experts and the channel dimension through the second convolutional layer, and finally generate normalized routing weights through the Sigmoid function.

[0024] S2-2-2, Define the convolution kernel expert group and initialize multiple convolution kernel parameters with the same size;

[0025] S2-2-3. Based on the calculated routing weights, dynamically weight and sum the convolutional kernel expert groups to generate an aggregated convolutional kernel adapted to the current input;

[0026] S2-2-4. Perform grouped convolutions on the input feature map using aggregated convolution kernels, and restore the output to the original feature map shape. By calculating weights through the routing layer and aggregating multiple expert kernels, the network significantly increases the flexibility and expressive power of feature extraction, thus more effectively adapting to the diverse and subtle differences in the defects of pecan kernels, and improving model performance without significantly increasing the amount of inference computation.

[0027] As a preferred embodiment of the present invention, the PANet multi-scale feature fusion module in S2-3 specifically includes the following steps:

[0028] S2-3-1. Extract the output feature maps of the last three stages (Stage 2, Stage 3, Stage 4) of the Backbone network;

[0029] S2-3-2. Use a 1×1 convolution as an alignment layer to align the number of channels of the extracted feature maps of the three stages to 256 dimensions.

[0030] S2-3-3. Construct a top-down Feature Pyramid (FPN) path, perform bilinear interpolation upsampling on deep features, and add them element-wise with the aligned shallow features.

[0031] S2-3-4. Construct a bottom-up path aggregation (PANet) path, perform 3×3 convolution downsampling with a stride of 2 on the feature map, and add it element-wise with the corresponding layer features in the FPN path. To implement the above technical solution, this module uses a bidirectional path aggregation strategy. On the one hand, it uses FPN to pass deep semantic information to the shallow layer, and on the other hand, it uses PANet to feed back shallow texture details to the deep layer. This multi-level feature fusion mechanism ensures that the model can recognize both large-area morphological defects and pinpoint-sized spot defects, solving the problem of insufficient single-scale feature representation ability.

[0032] As a preferred embodiment of the present invention, the feature enhancement and classification structure in S2-4 specifically includes the following steps:

[0033] S2-4-1. For the feature maps of each level output by PANet in S2-3, feature refinement is performed by an enhancement layer consisting of 3×3 convolution, batch normalization and SiLU activation function.

[0034] S2-4-2. Adjust the enhanced multi-scale feature maps to the same size and stack and fuse them. Then, perform overall feature optimization through the final convolutional layer and the global SE attention module.

[0035] S2-4-3. The optimized features are input into the classification head, passing sequentially through convolutional layers, batch normalization, SiLU activation function, and global average pooling layer. Finally, a fully connected layer outputs the prediction results for six types of defects. To implement the above technical solution, this structure further refines and globally optimizes the fused features before final classification. Through the cooperation of the enhancement layer and the global SE module, salient features are further strengthened, making the feature map input to the classifier more separable, thereby improving the final classification accuracy.

[0036] As a preferred embodiment of the present invention, the training process in S2-5 adopts a mixed precision and gradient accumulation strategy, and the specific steps include:

[0037] S2-5-1. Enable Automatic Mixed Precision (AMP) and use torch.amp.GradScaler to dynamically scale the loss value to prevent gradient underflow;

[0038] S2-5-2. Set the gradient accumulation step to 2, accumulate the gradients of the two batches, and then perform an optimizer parameter update and gradient zeroing operation. Mixed-precision training effectively reduces memory usage and accelerates the computation process, while the gradient accumulation strategy simulates a larger batch size under limited hardware conditions, which helps the model training to converge stably, prevents oscillations caused by too small a batch size, and ensures the effect of model optimization.

[0039] In a preferred embodiment of the present invention, step S1 involves collecting several typical hickory kernel samples selected during the production and processing process. These samples are categorized into six types: normal, spotted, shriveled, dark-colored, moldy, and with shell. Image data for each type of typical sample is captured using a self-built imaging system. Data augmentation is applied to this data, including adjusting image contrast and saturation, and performing image cropping, scaling, random rotation, and mirroring. By implementing the above technical solution and through detailed classification and data augmentation of the six typical sample types, the model can learn the distribution of defect features under different lighting, angles, and noise environments. This greatly enriches the diversity of training samples, thereby improving the model's generalization ability and anti-interference capability in actual industrial environments.

[0040] As a preferred embodiment of the present invention, the training parameters are set as follows: a total of 30 training rounds and a batch size of 32; a phased learning rate adjustment strategy is adopted, with an initial learning rate of 0.0003 used during the first 5 rounds of freeze period, and the learning rate is reset to 0.000003 during the unfreezing and fine-tuning phase, followed by 3 rounds of linear warm-up of the learning rate, and then decay using a cosine annealing strategy; the AdamW optimizer is used for parameter updates, with a weight decay coefficient of 0.0003; the loss function is a cross-entropy loss function with a label smoothing coefficient of 0.05.

[0041] The above technical solution effectively balances the rapid convergence of the model in the early stage of training with the fine-tuning in the later stage through refined hyperparameter settings and a phased training strategy, avoiding overfitting and ensuring that the final model achieves optimal detection performance. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall process of the technical solution of the present invention;

[0043] Figure 2 This is a schematic diagram of six typical samples in the pecan kernel classification dataset constructed by this invention;

[0044] Figure 3 This is the overall network structure diagram of SHT-CNv2 proposed in this invention;

[0045] Figure 4 This is a schematic diagram of the SE channel attention mechanism module embedded in this invention;

[0046] Figure 5 This is a schematic diagram of the confusion matrix of the SHT-CNv2 model on the test set in an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the confusion matrix of the baseline model (ConvNeXtV2-Base) on the same validation set in the comparison.

[0048] Figure 7 It is a comparison of performance indicators at different stages of model testing;

[0049] Figure 8 This is the accuracy curve of the SHT-CNv2 model;

[0050] Figure 9 This is the loss curve of the SHT-CNv2 model;

[0051] Figure 10 This is the Grad-CAM visualization result of the SHT-CNv2 model. Detailed Implementation

[0052] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, so that the technical solution of the present invention can be more easily understood and mastered.

[0053] A method for identifying defects in pecan kernels based on an improved ConvNeXtV2 algorithm includes the following steps:

[0054] Step S1: Collect pecan kernel samples selected during the production and processing process. The samples cover six categories: normal, spotted, shriveled, dark, moldy, and with shell. Image acquisition systems are used to capture images of the samples, and the images are then classified and labeled. The labeled dataset is divided into training, validation, and test sets according to a preset ratio.

[0055] Step S2: In the Backbone part of the ConvNeXtV2 network, a dynamic convolution mechanism is introduced. In Stage 3 and Stage 4 of the backbone network, dynamic deep convolutional layers are used to replace the original static deep convolutional layers. In the Neck part, a PANet multi-scale feature fusion module is constructed. First, the number of channels of the feature maps of the last three stages output by the Backbone is unified through a channel alignment layer. A top-down feature pyramid path is constructed. Deep features are upsampled and added to shallow features. On this basis, a bottom-up path aggregation network is constructed. The feature maps are downsampled and added to the features of the previous level. In the Neck and Head parts, a feature enhancement and classification structure is constructed. Enhancement layers consisting of standard convolution, batch normalization, and SiLU activation functions are used to process the features of each level. The processed features are adjusted to the same size and superimposed and fused. The fused features are weighted by the global SE attention module and then input to the classification head output result containing convolutional layers, global average pooling layers, and fully connected layers. The SHT-CNv2 network structure is constructed through the above optimization methods.

[0056] Step S3: Set training parameters, adopt a phased freezing strategy, and use the dataset obtained in step S1 to train the SHT-CNv2 network constructed in step S2 to obtain the SHT-CNv2.pth model.

[0057] Step S4: Test and evaluate the SHT-CNv2.pth model trained in Step S3 using the test set from the dataset in Step S1.

[0058] The specific implementation of the dataset construction and preprocessing in step S1 is as follows;

[0059] We collected six typical types of pecan kernels selected during the production and processing process: normal, spotted, shriveled, dark-colored, moldy, and shelled. An image dataset was constructed using a self-built imaging system (MER-500-14GC, China Daheng Group Co., Ltd.), with a resolution of 2592 pixels × 1944 pixels, a frame rate of 14 fps, and an exposure time of 36 μs to 1 s. The collected samples cover the six common categories in production and processing: normal kernels, spotted kernels, shriveled kernels, dark-colored kernels, moldy kernels, and shelled kernels. The dataset contains a total of 14,400 images, with 2,400 images for each category.

[0060] Images were categorized and labeled using annotation tools, and the directory structure was organized according to the ImageFolder format. The dataset was randomly divided into training, validation, and test sets in a 3:1:1 ratio.

[0061] Data augmentation operations are performed on the training data to enhance the robustness of the model. Augmentation methods include: randomly changing the contrast and saturation of images; random resized cropping, scaling, rotation, and mirroring; and using random erasing to simulate occlusion.

[0062] The specific implementation method for constructing the SHT-CNv2 network structure in step S2 is as follows:

[0063] S2-1: Embedded SE Channel Attention Mechanism

[0064] After the output of the deep convolutional layer of each ConvNeXtV2 Block in the Backbone, an SE (Squeeze-and-Excitation) module is embedded. This module recalibrates the feature weights by explicitly modeling the dependencies between channels.

[0065] For the input feature map ,in For the number of channels, Space dimensions:

[0066] Squeeze: Compresses the spatial dimension to a smaller value using global average pooling. The global feature descriptor is obtained. ,in Let Xc(i,j) be the global feature descriptor (scalar) for the c-th channel, and let Xc(i,j) be the input feature. Figure X The value at spatial location (i,j) in the c-th channel, where H and W are the spatial height and width of the feature map. The specific calculation formula for each channel is as follows:

[0067]

[0068] Excitation: Captures channel dependencies using two fully connected layers. First, the channel dimensionality is reduced to... (In this embodiment) After activation by SiLU, it is then upgraded back to a higher dimension. Finally, the weights are generated through Sigmoid activation. Weight The specific calculation formula is as follows:

[0069]

[0070] in, The SiLU activation function is used. It is the Sigmoid activation function. , .

[0071] Recalibration (Scale): Resize the weights Compared with the original feature map Perform channel-by-channel multiplication to obtain the output. , The specific calculation formula is as follows:

[0072]

[0073] S2-2: Introducing Dynamic Convolution Mechanism

[0074] To improve the network's ability to adaptively extract complex defect features, dynamic deep convolutional layers (DynamicDWConv2d) are used to replace the original static deep convolutional layers in Stages 3 and 4 of Backbone.

[0075] This module includes the routing layer and Each convolutional kernel expert (in this embodiment) ).

[0076] Route weight calculation: For input Perform global average pooling (GAP) and two convolutional mappings to compute the target... Normalized weights of individual experts , The specific calculation formula is as follows:

[0077]

[0078] in , This is the SiLU activation function.

[0079] Dynamic aggregation convolution kernel: Based on the calculated routing weights, apply the results to a predefined group of static convolution kernels. Perform weighted summation to generate a dynamic convolution kernel for the current input. , The specific calculation formula is as follows:

[0080]

[0081] Dynamic convolution: using aggregated convolution kernels For input The formula for performing grouped convolution is as follows:

[0082]

[0083] in This is the output feature map after dynamic convolution.

[0084] S2-3: Constructing the PANet multi-scale feature fusion module

[0085] Extracting feature maps from the last three stages of Backbone First use Convolution aligns its channel count to 256, resulting in .

[0086] FPN Top-Down Path: This path transfers strong semantic features from deep layers to shallow layers. The calculation formula is as follows:

[0087]

[0088] in, This represents the feature map of the i-th layer obtained after fusion via the FPN path. This represents a 2x bilinear interpolation upsampling operation. The calculation order is from deep to shallow, that is, first let... = Then calculate sequentially. , .

[0089] PANet bottom-up path: This path feeds back fine-grained texture features from shallow layers to deeper layers. The calculation formula is as follows:

[0090]

[0091] in, This represents the feature map of the i-th layer obtained after secondary fusion via the PANet path. This indicates that a 2x downsampling is achieved using a 3×3 convolutional layer with a stride of 2. The computation order is from shallow to deep, starting at [the specified point]. = Then calculate sequentially. , , .

[0092] S2-4: Feature Enhancement and Classification

[0093] For the features of each level output by PANet { , , The features are refined using enhancement layers (composed of 3×3 convolutions, batch normalization, and SiLU activation functions). Then, the refined feature maps are uniformly adjusted to match the desired texture size using upsampling. Using the same spatial dimensions (H, W), and performing element-wise summation, the fused features are obtained. . The specific calculation formula is as follows:

[0094]

[0095] in, This indicates that the spatial dimensions of the feature map are upsampled to the same level as...

[0096] After the fused features are weighted by the global SE module, the Logits values ​​for the six categories are output through the classification header. The calculation formula is as follows:

[0097]

[0098] in, The final output of the model consists of the predicted values ​​(Logits) for the six categories. It is a fully connected layer (classifier). It is a global average pooling layer. The SiLU activation function is used. For batch normalization layer, It is a convolutional layer. The fusion feature is obtained from formula (9).

[0099] In step S3, the parameters are trained using a phased freezing strategy, the specific implementation of which is as follows:

[0100] The total number of training epochs was 30, with a batch size of 32. A phased learning rate adjustment strategy was adopted, using an initial learning rate of 0.0003 during the first 5 epochs of freeze-up, and resetting the learning rate to 0.000003 during the unfreezing and fine-tuning phase. The learning rate was first linearly warmed up for 3 epochs, followed by a cosine annealing strategy for decay. The AdamW optimizer was used for parameter updates, with a weight decay coefficient of 0.0003. The loss function used was the cross-entropy loss function with a smoothing coefficient of 0.05, and the specific calculation formula is as follows:

[0101] Loss function: Label smoothing is employed. The cross-entropy loss function is used to prevent overfitting.

[0102]

[0103] in, This represents the total training loss value. The label smoothing coefficient is 0.05 in this example, and C is the total number of categories, which is 6 here. The one-hot encoding of the real label (1 for class i, 0 for the rest). Predict the probability of the model belonging to class i.

[0104] Learning rate adjustment: Cosine annealing is employed. During the unfreezing and fine-tuning phase, the learning rate... With the number of iterations The change is calculated using the following formula:

[0105]

[0106] in, This represents the current iteration round number. The preset total number of iterations. For maximum learning rate This is the minimum learning rate.

[0107] The specific implementation method for model testing and evaluation in step S4 is as follows:

[0108] The trained model is evaluated using a test set. Accuracy is used as the primary evaluation metric, calculated using the following formula:

[0109]

[0110] in, (True Positive) represents the number of true positive samples. (True Negative) represents the number of true negative samples. (False Positive) represents the number of false positive samples. (False Negative) represents the number of false negative samples.

[0111] To further improve the robustness of the model evaluation, a Test-Time Augmentation (TTA) strategy was adopted during the testing phase. Specifically, for the same test image, the model prediction probabilities for both the original image and the horizontally flipped image were calculated, and the average of the two was used as the final prediction probability. The calculation formula is as follows:

[0112]

[0113] in, The model's predicted probability represents the original input image. This represents the model's predicted probability for the image obtained after horizontally flipping the original image. This represents the final predicted probability after Test-Time Enhancement (TTA) processing.

[0114] The improved ConvNeXtV2 model obtained in this invention was compared with other models to verify the effectiveness of the optimization method. The experimental results are shown in Table 1:

[0115] Table 1. Comparison of the ConvNeXtV2 model with other models.

[0116]

[0117] Ablation experiments were conducted on the improved ConvNeXtV2 method for detecting defects in pecan kernels according to this invention. The experimental results are shown in Table 2:

[0118] Table 2. Comparison of ablation experiments using the improved ConvNeXtV2 model.

[0119]

[0120] A comparison of model performance at different improvement stages of the improved ConvNeXtV2 pecan kernel defect detection method of this invention. Experimental results are shown in Table 3 and Appendix. Figure 7 As shown:

[0121] Table 3. Experimental results comparing model performance at different improvement stages.

[0122]

[0123] Appendix Figure 7 This is a comparison of performance indicators at different stages of model testing (see the instruction manual for details). Figure 7 );

[0124] Among them, the appendix Figure 7 The calculation methods for the relevant indicators are as follows:

[0125]

[0126]

[0127]

[0128] Where P represents precision and R represents recall. The F1 score is the harmonic mean of precision and recall.

[0129] The effectiveness of the improved ConvNeXtV2 model and the two-stage training strategy obtained in this invention was investigated by recording the changes in the model's performance metrics over 30 epochs. The experimental results are attached. Figure 8 With appendix Figure 9 As shown;

[0130] Appendix Figure 8 This is the accuracy curve of the SHT-CNv2 model (see the instruction manual for details). Figure 8 );

[0131] Appendix Figure 9 This is the loss curve of the SHT-CNv2 model during training (see the instruction manual for details). Figure 9 ).

[0132] To intuitively verify the effectiveness of the proposed SHT-CNv2 model in extracting and locating defect features in pecan kernels, and to delve into the decision-making mechanism within the network, this embodiment employs Grad-CAM (Gradient-weighted Class Activation Mapping) heatmap visualization technology for qualitative analysis of the model. Figure 10 This is a detection image example tested using the trained model after adopting the improved scheme of this invention.

[0133] Through the above implementation methods, the SHT-CNv2 model constructed in this invention significantly improves the accuracy of identifying minute defects in pecan kernels. As shown in the ablation experiments in Tables 1 and 2, the complete model SHT-CNv2 achieves an accuracy of 98.72% on the validation set, outperforming the benchmark ConvNeXtV2 model and other comparative models. This model effectively solves the problem of easily missing minute texture defects while maintaining high inference speed, meeting the requirements of high precision and real-time performance for automated pecan kernel sorting.

[0134] Of course, the above are just typical examples of the present invention. In addition, the present invention may have many other specific embodiments. All technical solutions formed by equivalent substitution or equivalent transformation fall within the scope of protection claimed by the present invention.

Claims

1. A method for identifying defects in pecan kernels based on an improved ConvNeXtV2 network, comprising a ConvNeXtV2 network, characterized in that, Includes the following steps: S1. Construct a pecan kernel classification dataset, including taking image data using a self-built shooting system, classifying and labeling the images, and dividing the data into a training set, a validation set, and a test set according to a preset ratio; S2. Construct an improved ConvNeXtV2 network structure, named SHT-CNv2. Specific improvements include: S2-1. An SE channel attention mechanism is embedded in the backbone of the ConvNeXtV2 network; S2-2, Introduce a dynamic convolution mechanism in the Backbone part of the ConvNeXtV2 network; S2-3. Construct a PANet multi-scale feature fusion module in the Neck part of the ConvNeXtV2 network; S2-4. Construct feature enhancement and classification structures in the Neck and Head parts of the ConvNeXtV2 network; S2-5. Set training parameters and adopt a phased freezing strategy. In the early stage of training, freeze the parameters of the Backbone Stem layer and the first two stages. S3. Use the test set to test and evaluate the trained SHT-CNv2 model.

2. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The SE channel attention mechanism module embedded in S2-1 sequentially includes a global average pooling layer, a first convolutional layer for dimensionality reduction, a SiLU activation function, a second convolutional layer for dimensionality increase, and a Sigmoid activation function. It recalibrates the input features using the generated channel weights, specifically including the following steps: S2-1-1. Insert the SE attention module after the output of the deep convolutional layer of the ConvNeXtV2 Block; S2-1-2. In the SE attention module, the input feature map is first subjected to global average pooling to compress it into a 1×1 spatial feature. S2-1-3, The number of channels is reduced to 1 / 16 of the number of input channels through the first 1×1 convolutional layer, and then activated by the SiLU activation function; S2-1-4. The number of channels is restored to the original dimension through the second 1×1 convolutional layer, and the channel weight coefficients are generated by the Sigmoid function. S2-1-5. Multiply the generated weight coefficients element-wise with the original input feature map to complete the channel feature recalibration.

3. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The dynamic convolution mechanism introduced in S2-2 replaces the original static deep convolution layers with dynamic deep convolution layers in Stages 3 and 4 of the backbone network. These dynamic deep convolution layers include a routing layer and multiple expert convolution kernels. The routing layer dynamically calculates weights based on the input features and aggregates expert convolution kernels to perform grouped convolution operations on the feature maps. Specifically, this includes the following steps: S2-2-1. Construct a routing layer, perform global average pooling on the input feature map, then reduce the dimension through the first convolutional layer, activate it with the SiLU activation function, increase the dimension to the product of the number of experts and the channel dimension through the second convolutional layer, and finally generate normalized routing weights through the Sigmoid function. S2-2-2, Define the convolution kernel expert group and initialize multiple convolution kernel parameters with the same size; S2-2-3. Based on the calculated routing weights, dynamically weight and sum the convolutional kernel expert groups to generate an aggregated convolutional kernel adapted to the current input; S2-2-4. Perform grouped convolution on the input feature map using aggregated convolution kernels and restore the output to the original feature map shape.

4. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The PANet multi-scale feature fusion module in S2-3 first unifies the number of channels in the feature maps of the last three stages output by the Backbone through a channel alignment layer, constructs a top-down feature pyramid path, upsamples the deep features and adds them to the shallow features, and then constructs a bottom-up path aggregation network based on this, downsamples the feature maps and adds them to the features of the previous level. Specifically, it includes the following steps: S2-3-1. Extract the output feature maps of the last three stages of the Backbone network: Stage 2, Stage 3, and Stage 4. S2-3-2. Use 1×1 convolutions as alignment layers to uniformly align the number of channels of the extracted feature maps of the three stages to 256 dimensions. S2-3-3. Construct a top-down Feature Pyramid (FPN) path, perform bilinear interpolation upsampling on deep features, and add them element-wise to the aligned shallow features. S2-3-4. Construct a bottom-up path aggregation PANet path, perform 3×3 convolution downsampling on the feature map with a stride of 2, and add it element-wise with the corresponding layer features in the FPN path.

5. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The construction of the feature enhancement and classification structure in S2-4 uses an enhancement layer consisting of standard convolution, batch normalization, and SiLU activation function to process features at each level. The processed features are adjusted to the same size and then stacked and fused. The fused features are weighted by the global SE attention module and then input to a classification head containing convolutional layers, global average pooling layers, and fully connected layers to output the results. Specifically, the steps include: S2-4-1. For the feature maps of each level output by PANet in S2-3, feature refinement is performed by enhancement layers consisting of 3×3 convolution, batch normalization and SiLU activation function. S2-4-2. Adjust the enhanced multi-scale feature maps to the same size and stack and fuse them. Then, perform overall feature optimization through the final convolutional layer Final Conv and the global SE attention module Global SEBlock. S2-4-3. The optimized features are input into the classification head, which then passes through a convolutional layer, batch normalization, SiLU activation function, and global average pooling layer in sequence. Finally, the prediction results for 6 types of defects are output through a fully connected layer.

6. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The training process in S2-5 employs a hybrid precision and gradient accumulation strategy: S2-5-1. Enable Automatic Mixed Precision AMP and use torch.amp.GradScaler to dynamically scale the loss value to prevent gradient underflow. S2-5-2. Set the gradient accumulation step to 2, accumulate the gradients of the two batches, and then perform an optimizer parameter update and gradient clearing operation.

7. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, Several typical hickory kernel samples selected during the production and processing were collected. The samples were divided into 6 categories. Image data of each typical sample was captured using a self-built imaging system. The images were classified and labeled to construct a hickory kernel defect image dataset, which was then randomly divided into training set and validation set according to a preset ratio.

8. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, Data augmentation is performed on the data, including changing the contrast and saturation of the image, and cropping, scaling, randomly rotating, and mirroring the image.

9. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The division of the training set and validation set in the preset ratio is determined based on the actual dataset directory structure and is set to ImageFolder format.

10. The method for identifying defects in pecan kernels based on the improved ConvNeXtV2 according to claim 1, characterized in that, The training parameters include a phased learning rate adjustment strategy, the AdamW optimizer, and a labeled smooth cross-entropy loss function.

Images