Walnut image processing method and model based on walnut occlusion phenomenon detection model, and readable storage medium

Abstract

The invention relates to the technical field of deep learning, in particular to a walnut image processing method and model based on a walnut occlusion phenomenon detection model, and a readable storage medium. A multi-attention feature fusion module is provided to balance shallow local information and deep global information; on the basis of an original C2f module architecture, a C2f module with a function of enhancing edge features in the walnut image is provided to improve the detection precision of the model; reserving a target convolutional layer of which the layer adaptive amplitude score is greater than a preset pruning ratio, so as to adaptively select the sparseness of each layer in the pruning process; a knowledge distillation mode specially aiming at an intensive target detector, namely dichotomy distillation loss and intersection-to-parallel ratio-based positioning loss, is provided for fine tuning training, and accurate detection of the walnut occlusion phenomenon in the walnut image acquired by the unmanned aerial vehicle based on a deep learning model is realized. The objective of the invention is to solve the problem of how to perform walnut shielding phenomenon detection through a deep learning model.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a walnut image processing method, model and readable storage medium based on a walnut occlusion phenomenon detection model. Background Art

[0002] The walnut picking cycle is short, and untimely picking can easily lead to problems such as corruption and deterioration. Currently, using intelligent picking robots to replace manual labor to improve walnut harvesting efficiency is a current research direction.

[0003] Current intelligent harvesting robots often use deep learning object detection algorithms to detect and locate walnuts. However, walnuts in unstructured orchard environments often face varying degrees of occlusion. Traditional object detection algorithms ignore occlusion information between walnuts, resulting in positioning errors when locating walnut picking points for walnuts with occlusion, which in turn affects the success rate of harvesting. Summary of the Invention

[0004] The main purpose of this application is to provide a walnut image processing method based on a walnut occlusion phenomenon detection model, aiming to solve the problem of how to detect walnut occlusion phenomenon through a deep learning model.

[0005] To achieve the above objectives, the present application provides a walnut image processing method based on a walnut occlusion phenomenon detection model, the method comprising:

[0006] Inputting the collected walnut image into a trained walnut occlusion phenomenon detection model to determine a walnut occlusion phenomenon detection result in the walnut image through the walnut occlusion phenomenon detection model;

[0007] The walnut occlusion phenomenon detection model includes a multi-attention feature fusion module for fusing deep information and shallow information in the walnut image, and a C2f module with a function of enhancing edge features in the walnut image;

[0008] wherein the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio;

[0009] The walnut occlusion detection model is fine-tuned using binary classification distillation loss and intersection-over-union (IoU)-based positioning distillation loss.

[0010] Optionally, the calculation expression of the layer adaptive amplitude score is:

[0011]

[0012] Where, Represents the weight tensor The index is The LAMP score of the element, Represents the weight tensor The position index of a specific weight in , Represents the weight tensor of the current convolutional layer, and Indicates a specific weight value.

[0013] Optionally, the step of fine-tuning training includes:

[0014] The weighted sum of the binary classification distillation loss and the intersection-over-union (IoU)-based positioning distillation loss is used as the loss function of the walnut occlusion detection model. :

[0015]

[0016] Where, To adjust the classification loss weight, is the binary distillation loss, is the weight of the positioning loss, To locate distillation losses.

[0017] Optionally, the step of calculating the binary distillation loss includes:

[0018] The numerical image in the classification task is regarded as multiple binary classification images, and each image is calculated by The binary classification score , , according to the binary classification score , Calculate the binary distillation loss :

[0019]

[0020] in:

[0021]

[0022]

[0023] Where, is the binary cross entropy loss, , Respectively , In the Position No. The binary classification score of the class, Loss weighting strategy.

[0024] Optionally, the step of calculating the positioning distillation loss based on the intersection-over-union ratio includes:

[0025] Determine the bounding box of the preset walnut image sample in the student model , and the corresponding bounding box in the teacher model :

[0026]

[0027]

[0028] in, and Represents the teacher model and student model in the Position prediction for each location, Indicates the Anchor points;

[0029] calculate and The intersection-over-union ratio of

[0030]

[0031] Introducing the loss weighting strategy to obtain the positioning distillation loss for:

[0032]

[0033] Where, is the weight value, Indicates a fixed category Take the maximum value of this category for all anchor boxes and use it as the first The weighting factor for each position, Indicates the number of candidate boxes corresponding to each input sample.

[0034] In addition, to achieve the above objectives, the present application also provides a walnut occlusion phenomenon detection model, in which the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio, and fine-tuning training is performed using a binary classification distillation loss and an intersection-over-union (IoU)-based positioning distillation loss. The walnut occlusion phenomenon detection model also includes:

[0035] YOLOv8s architecture;

[0036] a multi-attention feature fusion module, configured to fuse deep information and shallow information in the walnut image;

[0037] The C2f module is used to enhance edge features in the walnut image.

[0038] Optionally, the preset pruning ratio is 2.2.

[0039] Optionally, the multi-attention feature fusion module includes a coordinate attention module (CA) and a large kernel attention module (GLKA);

[0040] The large-core attention module includes three depth-wise separable convolutions of different scales: 3×3, 5×5, and 7×7, which are used to extract deep walnut feature information of different scales.

[0041] The attention module obtains global features by performing global average pooling operations in both horizontal and vertical directions. The first level learns the relationship between different channels through a fully connected layer to generate a channel attention vector.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the walnut image processing method based on the walnut occlusion phenomenon detection model as described in any of the above items are implemented.

[0043] This application has at least the following beneficial effects:

[0044] 1. A multi-attention feature fusion module is proposed to balance shallow local information and deep global information to achieve walnut occlusion detection;

[0045] 2. Based on the original C2f module architecture, a C2f module with the function of enhancing the edge features in the walnut image is proposed to improve the detection accuracy of the model;

[0046] 3. Remove the convolutional layers whose layer adaptation amplitude score is less than the preset pruning ratio, and retain the target convolutional layers whose layer adaptation amplitude score is greater than the preset pruning ratio, so as to adaptively select the sparsity of each layer during the pruning process;

[0047] 4. To overcome the imbalance between foreground and background when predicting classification scores on dense feature maps, a knowledge distillation method specifically for dense object detectors is proposed. This method uses a binary classification distillation loss and an intersection-over-union (IoU)-based localization loss for fine-tuning training. This allows for accurate detection of walnut occlusion in walnut images collected by drones based on a deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a first embodiment of a walnut image processing method based on a walnut occlusion phenomenon detection model of the present application;

[0049] Figure 2 This is the multi-attention feature fusion module involved in the embodiments of this application;

[0050] Figure 3 The MEEC2f architecture involved in the embodiment of this application

[0051] Figure 4 Schematic diagram of the process of the LAMP pruning method involved in the embodiment of the present application;

[0052] Figure 5 Schematic diagram of the BCKD knowledge distillation method involved in the embodiment of the present application;

[0053] Figure 6 This is a schematic diagram of the test visualization results involved in the embodiments of this application.

[0054] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] To better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] First embodiment

[0057] Reference Figure 1 In this embodiment, a walnut image processing method based on a walnut occlusion phenomenon detection model is characterized in that the method comprises the following steps:

[0058] S10, inputting the collected walnut image into the trained walnut occlusion phenomenon detection model to determine the walnut occlusion phenomenon detection result in the walnut image through the walnut occlusion phenomenon detection model.

[0059] The walnut occlusion phenomenon detection model includes a multi-attention feature fusion module for fusing deep information and shallow information in the walnut image, and a C2f module with a function of enhancing edge features in the walnut image;

[0060] In this embodiment, in order to better balance shallow local information and deep global information, a multi-attention feature fusion module (MAFF) is proposed. Figure 2As shown in the figure. When walnut data is input into the model, simply using a simple concat operation to fuse shallow and deep features often fails to effectively capture detailed features such as the edge texture of the walnut. Furthermore, since walnuts are typically small objects, and the deep feature maps have low resolution after multiple convolutions and pooling, it is difficult to accurately capture the details of the walnut. MAFF uses the GLKA and CA modules to enhance the integration of deep and shallow information. GLKA effectively captures a wide range of contextual information by expanding the receptive field of the convolution kernel, helping the model better understand the relationship between the walnut and the background. CA, on the other hand, aggregates features from two different directions, capturing long-range dependencies while maintaining sensitivity to position. By outputting perceptual attention maps along the horizontal and vertical directions, which represent the weighted features in different spatial directions, the model can integrate attention from different directions and further strengthen the representation of the region of interest. Through this multi-scale feature information fusion, the model can capture detailed information at different levels, further enhancing its expressive power.

[0061] Optionally, in some specific embodiments, MAFF can be composed of two parts: a large kernel convolution module (GLKA) and a coordinate attention module (CA).

[0062] GLKA employs the concept of large kernel convolution, generating 3×3, 5×5, and 7×7 convolutions to extract deep walnut feature information at different scales. Multi-scale convolution not only provides contextual information at different scales but also adapts to differences caused by varying distance and perspective. CA attention, on the other hand, aggregates feature information horizontally and vertically, capturing long-range dependencies while maintaining positional sensitivity.

[0063] By outputting two attention maps in different directions, which represent feature weights in different spatial directions, the model can integrate attention in different directions to enhance the local details of shallow feature maps. Through the synergistic effect of GLKA and CA, the global information of deep features is effectively integrated with the local details of shallow features, enabling the model to capture multi-level walnut features, improving the ability to express walnut objects of different scales, thereby enhancing detection accuracy.

[0064] In this embodiment, on the other hand, based on the original C2f module architecture, a C2f module with the function of enhancing the edge features in the walnut image is proposed, which is called MEEC2f module in this embodiment.

[0065] Specifically and optionally, the function of enhancing edge features in the walnut image may be implemented by a multi-scale edge enhancement module (MEEM).

[0066] Reference Figure 3The MEEC2f architecture shown in the figure is based on the traditional MEEM architecture. In this embodiment, the following improvements are made to its Bottleneck area:

[0067] First, feature extraction is performed through two 3×3 depth-separable convolutions. The extracted feature map is enhanced with the edge information through the MEEM structure. The feature map after two convolutions is represented as ,in , the execution process of the MEEM structure can be expressed by the following formula:

[0068]

[0069]

[0070] Where, represents a 1×1 convolution with a ReLU activation function and a batch normalization layer, Represents feature layers at different scales, Represents 3×3 average pooling. After the pooling convolution operation, the edge enhancer is used To enhance edge details:

[0071]

[0072]

[0073]

[0074]

[0075] Where, is the edge enhancer, represents a 1×1 convolution with a ReLU activation function and a batch normalization layer, Represents feature layers at different scales, represents 3×3 average pooling, Indicates that the input feature layer has not been enhanced by the edge. represents the original input feature layer, Represents the difference between the input feature layer and the pooling layer, represents the output after edge enhancement, Indicates the final output result.

[0076] wherein the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio;

[0077] In this embodiment, the layer adaptive amplitude score (LAMP) score of each convolutional layer is calculated to evaluate the contribution of the layer to the model output. Convolutional layers with a layer adaptive amplitude score less than a preset pruning ratio are removed, and target convolutional layers with a layer adaptive amplitude score greater than the preset pruning ratio are retained, thereby adaptively selecting the sparsity of each layer during the pruning process.

[0078] Among them, the walnut occlusion phenomenon detection model is fine-tuned and trained through binary classification distillation loss and intersection-over-union (IoU)-based positioning distillation loss.

[0079] In this embodiment, walnut detection, as part of dense object detection, often faces a serious foreground-background imbalance problem when predicting classification scores on dense feature maps. To this end, this embodiment proposes a knowledge distillation method (BCKD) specifically for dense object detectors. BCKD consists of two novel losses: a binary classification distillation loss and a positioning loss based on intersection-over-union (IoU).

[0080] BCKD treats the numerical image in the classification task as multiple binary classification images and calculates each image by The binary classification score , , and thus calculate the classification distillation loss based on these two-class scores:

[0081]

[0082]

[0083]

[0084] in, is the binary cross entropy loss, , Respectively , In the Position No. The binary classification score of the class, is a loss-weighted strategy, represents the classification distillation loss

[0085] The IoU-based localization loss transfers knowledge from the teacher model to the student model without having to rely on complex bounding box transformations. ,in and Represents the teacher model and student model in the Position prediction for each location, Indicates the anchor points, so the student model and the teacher model are The bounding box of can be expressed as:

[0086] ,

[0087] calculate and The intersection-over-union ratio is:

[0088]

[0089] Finally, the loss weighting strategy is introduced to obtain the positioning distillation loss:

[0090]

[0091] Where, is the weight value, Indicates a fixed category Take the maximum value of this category for all anchor boxes and use it as the first The weighting factor for each position, Indicates the number of candidate boxes corresponding to each input sample.

[0092] Combining the binary classification distillation loss and the IoU-based positioning loss, the final BCKD distillation loss is:

[0093]

[0094] in, and To adjust the weights of classification loss and positioning loss.

[0095] In the technical solution provided in this embodiment, in order to realize the detection of walnut occlusion phenomenon, a multi-attention feature fusion module is proposed to balance shallow local information and deep global information; on the other hand, based on the original C2f module architecture, a C2f module with the function of enhancing the edge features in the walnut image is proposed to improve the detection accuracy of the model; on the other hand, the convolutional layers with layer adaptive amplitude scores less than the preset pruning ratio are eliminated to retain the target convolutional layers with layer adaptive amplitude scores greater than the preset pruning ratio, thereby adaptively selecting the sparsity of each layer during the pruning process; on the other hand, in order to overcome the problem of foreground and background imbalance when predicting classification scores on dense feature maps, a knowledge distillation method specifically for dense target detectors is proposed - binary classification distillation loss and intersection-over-union-based positioning loss are fine-tuned for training, so as to realize accurate detection of walnut occlusion phenomenon in walnut images collected by drones based on deep learning models.

[0096] Second embodiment

[0097] Based on the first embodiment, this embodiment provides a training method for a walnut occlusion phenomenon detection model:

[0098] The first step is data collection. We use a DJI Matrice-300-RTK drone equipped with a Zenmuse P1 camera sensor to collect walnut data. Before collecting, we need to plan the route in advance.

[0099] During the flight of the unmanned aerial vehicle in the present embodiment, the distance from the ground is 25m, the flight speed is 2m / s, the lateral overlap rate is 80%, the heading overlap rate is 90%, and the shooting angle is vertically downward 90 °. A total of 180 unmanned aerial vehicle pictures are obtained, and the picture resolution is 8192 × 5460, and the picture format is JPG format. The walnut picture of collection is cut according to the size of 640 × 640, and after removing the picture without target, a total of 3219 walnut data are obtained. Calculate average grayscale and grayscale standard deviation to map light intensity. Specifically, the grayscale standard deviation of image is larger, and the illumination distribution is more uneven. In this example, the mean value of the grayscale standard deviation of data set is 44.1, and the median is 45.0. Take its average value, the image with grayscale standard deviation greater than 44.6 is divided into non-uniform illumination, and the image with grayscale standard deviation less than or equal to 44.6 is divided into uniform illumination. A total of 1439 non-uniform illumination images and 1726 uniform illumination images are obtained. The average grayscale value of each image is calculated in both uniform and non-uniform illumination data. In this example, grayscale values ​​less than 60 are classified as low illumination, grayscale values ​​between 60 and 100 are classified as medium illumination, and grayscale values ​​greater than 100 are classified as strong illumination.

[0100] In the second step, we annotated the walnuts using the LabelImg annotation software in the YOLO format. Walnuts obscured by overlapping fruit, branches, and leaves were marked as unobstructed, and those completely exposed to the field of view were marked as unobstructed. In this example, we obtained 19,380 obscured objects and 8,759 unobstructed objects. In this example, the walnut data under different lighting conditions was divided into an 8:1:1 ratio, as shown in Table 1.

[0101] Table 1 Different illumination datasets

[0102]

[0103] The third step is to optimize the YOLOv8s model by introducing the Multi-Attention Feature Fusion (MAFF) module and the MEEC2f module. When walnut data is input into the model, simply using a concat operation to fuse shallow and deep features often fails to effectively capture detailed features such as the walnut's edge texture. Furthermore, because walnuts are typically small objects, the deep feature maps, after multiple convolutions and pooling, have low resolution, making it difficult to accurately capture walnut details. MAFF enhances the integration of deep and shallow information through the GLKA and CA modules. GLKA expands the receptive field of the convolution kernel to effectively capture a wide range of contextual information, helping the model better understand the relationship between the walnut and its background. CA, on the other hand, aggregates features from two different directions, capturing long-range dependencies while maintaining position sensitivity. By outputting perceptual attention maps along the horizontal and vertical directions, which represent the weighted features in different spatial directions, the model integrates attention from different directions, further strengthening the representation of regions of interest. This multi-scale fusion of feature information allows the model to capture detailed information at different levels, further enhancing its expressive power.

[0104] After fusing deep and shallow features through MAFF, C2f performs further feature extraction on the fused feature map. C2f first adjusts the number of input channels through 1×1 convolutions, then separates the feature map and extracts features at different levels through multiple Bottleneck layers. This feature extraction method relies on the function of the Bottleneck layer. In this example, we optimize the Bottleneck layer in C2f and propose a more efficient and lightweight C2f architecture, MEEC2f, which uses 3×3 convolutions for feature extraction. The MEEM edge enhancement module then enhances the edge features of the walnut. The MEEM module primarily extracts features through convolution and pooling operations and further enhances the extracted features using an edge enhancer. During this process, 1×1 convolutions are first used to convert the input channels into hidden channels to reduce computational overhead. In this example, the effects of varying numbers of hidden channels are evaluated. The results are shown in Table 2. As can be seen from the results in Table 2, the model performs best when the number of hidden channels is half the number of output channels.

[0105] Table 2 Effects of different hidden channel numbers

[0106]

[0107] The fourth step is to prune the improved walnut detection model using the LAMP pruning method to obtain a more lightweight model. The flowchart of the LAMP pruning method is shown in Figure 4Specifically, LAMP calculates the LAMP score for each layer of connections by measuring the relative importance of all connections within the target layer. Based on the weight magnitude, LAMP selects smaller connections for global pruning until the predetermined global sparsity requirement is met, thus completing the pruning task.

[0108] In this embodiment, the dataset is first input into the improved model for training, and the pre-trained weight file obtained from the training is obtained as the input of LAMP pruning. Then, the pruning algorithm calculates the LAMP score of each input connection layer. Connection layers with lower LAMP scores are considered to contribute less to the model and can be pruned. When the pruning meets the predetermined global sparsity requirement, the pruning task is completed. The predetermined sparsity requirement is determined by the pruning ratio. To map, the calculation formula is:

[0109]

[0110] We are different The values ​​were evaluated and the results are shown in Table 3. From the data in the table, it can be seen that when When the value is 2.2, the detection accuracy and model complexity are both within an acceptable range.

[0111] Table 3 Experiments with different pruning ratios

[0112]

[0113] The fifth step is to use the BCKD knowledge distillation method to fine-tune the pruned model. The flow chart of the BCKD knowledge distillation method is as follows: Figure 5 shown.

[0114] Specifically, the improved walnut detection model is selected as the teacher model, and the pruned model is selected as the student model. Generally, the teacher model has a higher model complexity, higher accuracy, and better detection effect; while the student model has a lower complexity and relatively weaker detection performance. The goal of knowledge distillation is to use the teacher model as a guide so that the student model can learn the knowledge of the teacher model during training, thereby training a lightweight and efficient model. The BCKD knowledge distillation method is mainly composed of two novel distillation losses-binary classification distillation loss and IoU-based positioning distillation loss. The two loss calculation methods are mainly used to quantify the difference between the teacher model and the student model, guiding the student model to gradually learn the knowledge of the teacher model. Combining the binary distillation loss and the IoU-based positioning loss, the BCKD knowledge distillation loss is finally obtained:

[0115]

[0116] in, and In order to adjust the weights of classification loss and positioning loss, in this embodiment, different and The weight combination is used to evaluate the impact of two different losses on the knowledge distillation effect. The specific results are shown in Table 4 and Table 5. The proportion settings in the table are only for researchers' reference. The specific experimental results should be determined according to the specific data set.

[0117] Table 4. Experiments with different loss ratios

[0118]

[0119] Table 5. Experiments with different loss ratios

[0120]

[0121] Easy to know, Take 4.0, 7.5 is the preferred value.

[0122] Step 6: Verify the model training effect.

[0123] In this embodiment, on the Jetson platform, we used PyTorch 1.12.0 based on Python 3.8, CUDA 11.4 and Jetpak 5.1 for development and testing, and deployed the final walnut detection model after knowledge distillation to the Jetson platform for testing.

[0124] Reference Figure 6 The following diagram shows the test visualization results, comparing non-uniform low light, non-uniform medium light, and non-uniform strong light conditions. The original YOLOv8s model achieved an inference time of 378.4ms per image, while the improved model after knowledge distillation achieved an average inference time of 283ms. The original model would miss detections in uneven lighting conditions, but the improved Ours significantly reduced this phenomenon and achieved superior performance in detecting occluded walnuts.

[0125] As an implementation solution, this embodiment further provides a walnut occlusion phenomenon detection model, in which the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio, and fine-tuning training is performed using a binary classification distillation loss and an intersection-over-union (IoU)-based positioning distillation loss. The walnut occlusion phenomenon detection model further includes:

[0126] YOLOv8s architecture;

[0127] a multi-attention feature fusion module, configured to fuse deep information and shallow information in the walnut image;

[0128] The C2f module is used to enhance edge features in the walnut image.

[0129] Optionally, the preset pruning ratio is 2.2.

[0130] Optionally, the multi-attention feature fusion module includes an attention module and a large-core attention module;

[0131] The large-core attention module includes three depth-wise separable convolutions of different scales: 3×3, 5×5, and 7×7, which are used to extract deep walnut feature information of different scales.

[0132] The attention module obtains global features by performing global average pooling operations in both horizontal and vertical directions. The first level learns the relationship between different channels through a fully connected layer to generate a channel attention vector.

[0133] Optionally, the C2f module includes two 3×3 depth-separable convolutions for feature extraction, and a memory-enhanced encoder-decoder memory network structure for enhancing edge feature functions in the walnut image.

[0134] As an implementation solution, the embodiment of the present application also describes a computer system.

[0135] The computer system may include: a processor 1001, such as a CPU, a memory 1005, a user interface 1003, a network interface 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as a disk storage device. The memory 1005 may also be a storage device independent of the processor 1001.

[0136] The memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module and a computer program. The operating system is a program that manages and controls the hardware and software resources of a computer system, and the operation of computer programs and other software or programs.

[0137] In the computer system, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used to communicate data with the background server; the processor 1001 can be used to call the computer program stored in the memory 1005.

[0138] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:

[0139] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0140] Inputting the collected walnut image into a trained walnut occlusion phenomenon detection model to determine a walnut occlusion phenomenon detection result in the walnut image through the walnut occlusion phenomenon detection model;

[0141] The walnut occlusion phenomenon detection model includes a multi-attention feature fusion module for fusing deep information and shallow information in the walnut image, and a C2f module with a function of enhancing edge features in the walnut image;

[0142] wherein the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio;

[0143] The walnut occlusion detection model is fine-tuned using binary classification distillation loss and intersection-over-union (IoU)-based positioning distillation loss.

[0144] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0145] The weighted sum of the binary classification distillation loss and the intersection-over-union (IoU)-based positioning distillation loss is used as the loss function of the walnut occlusion detection model. :

[0146]

[0147] Where, To adjust the classification loss weight, is the binary distillation loss, is the weight of the positioning loss, To locate distillation losses.

[0148] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0149] The numerical image in the classification task is regarded as multiple binary classification images, and each image is calculated by The binary classification score , , according to the binary classification score , Calculate the binary distillation loss :

[0150]

[0151] in:

[0152]

[0153]

[0154] Where, is the binary cross entropy loss, , Respectively , In the Position No. The binary classification score of the class, Loss weighting strategy.

[0155] When the processor 1001 calls the computer program stored in the memory 1005, it performs the following operations:

[0156] Determine the bounding box of the preset walnut image sample in the student model , and the corresponding bounding box in the teacher model :

[0157]

[0158]

[0159] in, and Represents the teacher model and student model in the Position prediction for each location, Indicates the Anchor points;

[0160] calculate and The intersection-over-union ratio of

[0161]

[0162] Introducing the loss weighting strategy to obtain the positioning distillation loss for:

[0163]

[0164] Where, is the weight value, Indicates a fixed category Take the maximum value of this category for all anchor boxes and use it as the first The weighting factor for each position, Indicates the number of candidate boxes corresponding to each input sample.

[0165] Furthermore, those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the steps in the process of the above-described method embodiment.

[0166] Therefore, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the walnut image processing method based on the walnut occlusion phenomenon detection model as described in the above embodiment.

[0167] The computer-readable storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0168] It should be noted that since the storage medium provided in the embodiments of this application is the storage medium used to implement the method of the embodiments of this application, based on the method described in the embodiments of this application, those skilled in the art will be able to understand the specific structure and deformation of the storage medium, and therefore will not be described in detail here. All storage media used in the method of the embodiments of this application fall within the scope of protection to be provided by this application.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0174] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0175] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A walnut image processing method based on a walnut occlusion phenomenon detection model, characterized in that: The method comprises the following steps: Inputting the collected walnut image into a trained walnut occlusion phenomenon detection model to determine a walnut occlusion phenomenon detection result in the walnut image through the walnut occlusion phenomenon detection model; The walnut occlusion phenomenon detection model includes a multi-attention feature fusion module for fusing deep information and shallow information in the walnut image, and a C2f module with a function of enhancing edge features in the walnut image; wherein the layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio; Among them, the walnut occlusion phenomenon detection model is fine-tuned and trained through binary classification distillation loss and intersection-over-union (IoU)-based positioning distillation loss.

2. The method according to claim 1, wherein The calculation expression of the layer adaptive amplitude score is: ； Where, Represents the weight tensor The index is The LAMP score of the element, Represents the weight tensor The position index of a specific weight in , Represents the weight tensor of the current convolutional layer, and Indicates a specific weight value.

3. The method according to claim 1, wherein The steps of fine-tuning training include: The weighted sum of the binary classification distillation loss and the intersection-over-union (IoU)-based positioning distillation loss is used as the loss function of the walnut occlusion detection model. : ； Where, To adjust the classification loss weight, is the binary distillation loss, is the weight of the positioning loss, To locate distillation losses.

4. The method according to claim 3, wherein The calculation steps of the binary distillation loss include: The numerical image in the classification task is regarded as multiple binary classification images, and each image is calculated by The binary classification score , , according to the binary classification score , Calculate the binary distillation loss : ； in: ； ； Where, is the binary cross entropy loss, , Respectively , In the Position No. The binary classification score of the class, Loss weighting strategy.

5. The method according to claim 3, wherein The calculation steps of the positioning distillation loss based on the intersection-over-union ratio include: Determine the bounding box of the preset walnut image sample in the student model , and the corresponding bounding box in the teacher model : ； ； in, and Represents the teacher model and student model in the Position prediction for each location, Indicates the Anchor points; calculate and The intersection-over-union ratio of ； Introducing the loss weighting strategy to obtain the positioning distillation loss for: ； Where, is the weight value, Indicates a fixed category Take the maximum value of this category for all anchor boxes and use it as the first The weighting factor for each position, Indicates the number of candidate boxes corresponding to each input sample.

6. A walnut occlusion phenomenon detection model, characterized in that: The layer adaptation amplitude score of each convolutional layer in the walnut occlusion phenomenon detection model is greater than a preset pruning ratio, and fine-tuning training is performed using a binary classification distillation loss and an intersection-over-union (IoU)-based positioning distillation loss. The walnut occlusion phenomenon detection model further includes: YOLOv8s architecture; a multi-attention feature fusion module, configured to fuse deep information and shallow information in the walnut image; The C2f module is used to enhance edge features in the walnut image.

7. The walnut occlusion phenomenon detection model according to claim 6, wherein: The preset pruning ratio is 2.

2.

8. The walnut occlusion phenomenon detection model according to claim 6, wherein: The multi-attention feature fusion module includes a coordinate attention module and a large core attention module; The large-core attention module includes three depth-wise separable convolutions of different scales: 3×3, 5×5, and 7×7, which are used to extract deep walnut feature information of different scales. The attention module obtains global features by performing global average pooling operations in both horizontal and vertical directions. The first level learns the relationship between different channels through a fully connected layer to generate a channel attention vector.

9. The walnut occlusion phenomenon detection model according to claim 6, wherein: The C2f module includes two 3×3 depth-wise separable convolutions for feature extraction and a memory-enhanced encoder-decoder memory network structure for enhancing the edge feature function in the walnut image.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the walnut image processing method based on the walnut occlusion phenomenon detection model as described in any one of claims 1 to 5.

Images