One-stage bad weather fruit detection method based on dynamic coding

Abstract

The invention discloses a one-stage severe weather fruit detection method based on dynamic coding, and the method comprises the steps: collecting fruit image data on site in an agricultural scene, carrying out the data preprocessing of the collected image data, setting the hyper-parameters of a to-be-constructed one-stage severe weather fruit detection model based on dynamic coding, and carrying out the calculation of the hyper-parameters of the to-be-constructed one-stage severe weather fruit detection model. Constructing and training a first-stage severe weather fruit detection model based on dynamic coding; detecting fruits in the test set image, outputting a detection result mask, and comprehensively evaluating the model; the method solves the problems of poor generalization and applicability, tedious steps of a monitoring method and low real-time performance in the prior art, and has the characteristics that accurate detection can be carried out without pre-processing the image, and the training time and tedious steps of image pre-processing are reduced.

Description

Technical Field

[0001] The present invention belongs to the field of agricultural science and technology, and particularly relates to a one-stage fruit detection method for bad weather based on dynamic coding. Background Art

[0002] Traditional object detection algorithms mainly rely on manual feature extraction, requiring manual selection of regions of interest, classifying features for regions that may contain the object, and finally detecting the features. With the progress of technology, the computing power of computers can already handle the intensive calculations required for feature extraction in object detection tasks. Object detection algorithms based on deep learning have been widely used and have now become the most mainstream and rapid method in the field of object detection. For object detection in bad weather scenarios, the current methods are divided into two types: the first is algorithms for various different weather conditions, and each method is fixed to solve object detection in one type of weather, such as overexposure or haze. Such methods have extremely strong limitations and high costs, and require collecting object datasets under various weather conditions to ensure the applicability of the detection method; the second is an all-in-one method, which can learn the features of noise challenges through training and solve object detection in all difficult scenarios existing in the dataset at one time. There are some imperfections in the existing bad weather fruit detection technologies; 1) Some of the existing methods rely on the richness of the dataset and cannot solve the detection failures caused by all weather noises at one time; 2) Dynamic coding is not used in the existing methods during coding, which will cause the model to ignore some details during coding, and these details are crucial for low visibility object detection; 3) Most of the existing methods are two-stage detection methods, and during the training process, image denoising operations will be explicitly or implicitly performed on the images, hoping to improve the image quality to improve the detection accuracy, which reduces the real-time performance of the method; 4) The existing methods only consider the impact of weather factors on the detection performance of the model and do not consider the adaptability of the model to specific agricultural scenarios, which results in the inability to accurately detect fruit targets even when the image quality is high. Therefore, a one-stage fruit detection method for bad weather based on dynamic coding is needed to solve the above problems. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a one-stage fruit detection method for bad weather based on dynamic coding, aiming to solve the problems of poor generalization and applicability of the existing technology, and the cumbersome steps and low real-time performance of the monitoring method. It has the characteristics of being able to perform accurate detection without preprocessing the image, reducing the training time and cumbersome steps of image preprocessing.

[0004] To achieve the above technical effects, the technical solution adopted by the present invention is: A one-stage fruit detection method for bad weather based on dynamic coding, comprising the following steps: S1. Collect fruit image data in agricultural scenarios. The agricultural scenarios specifically include farmlands of individual business operators, commercial farms, experimental fields of universities, and large-scale planting bases. S2. Perform data preprocessing on the collected image data, including data format conversion, name unification, data cleaning, data annotation, and data augmentation. Divide the preprocessed image data into a training set and a test set. S3. Set the hyperparameters of the one-stage bad weather fruit detection model based on dynamic encoding to be constructed. S4. Input the fruit images in the training set, construct and train the one-stage bad weather fruit detection model based on dynamic encoding. S5. Detect the fruits in the test set images and output the detection result mask. S6. Conduct a comprehensive evaluation of the model.

[0005] Preferably, step S2 includes: S201. Name the image data obtained in step S1 in sequence and uniformly convert the image data into the png format. S202. Delete images with insufficient challenges and no fruit targets. S203. Label the fruits in the images as binary mask pictures as the ground truth. S204. Degrade the image quality, add weather conditions of rain, snow, and fog to the images. Take the original data set as the "simple" samples, and the data-augmented data set as the "difficult" samples, and perform the same training set and test set division on both types of samples.

[0006] Preferably, the hyperparameters set in step S3 include the initial learning rate, learning rate change, number of training epochs, optimizer, loss function, and iterative weight parameters.

[0007] Preferably, step S4 includes inputting the training set image data, constructing a one-stage bad weather fruit detection model based on dynamic encoding for the bad weather noise characteristics, learning and predicting the position and shape of the fruits, and outputting a binary mask image. Specifically as follows: Divide the training process of the one-stage bad weather fruit detection model based on dynamic encoding into feature encoding, feature fusion, feature decoding, and feature iterative optimization. Feature encoding includes a dynamic feature encoder module, a multi-scale feature encoder module, and a dynamic convolution module. The Vision Pyramid Transformer (PVT) demonstrates excellent performance in object detection tasks. Different from traditional Transformer models, PVT adopts a pyramid structure when processing image data to better capture multi-scale information in images. Specifically, PVT gradually reduces the resolution at different levels, similar to pyramid pooling in convolutional neural networks (CNNs), enabling it to process large-sized images more efficiently. By introducing hierarchical feature extraction in a pyramid shape, it not only improves computational efficiency but also better preserves the detailed information of the image. This enables PVT to exhibit superior performance in tasks such as image classification and object detection. The decoding stage can learn multi-scale features from high-level to low-level, which is very important for exploring the detailed information of the target. For a given input , the Vision Pyramid Transformer model PVT performs multi-scale encoding on the features, and the final feature representation is , where represents features of different sizes, n represents the levels of features of different sizes, R represents the feature shape, B represents the number of input images in a batch, C represents the number of feature channels, and H and W respectively represent the width and height of the features.

[0008] Preferably, under low visibility conditions, the environmental visibility is low and the light intensity difference is large, which will significantly affect the image quality and pose a huge challenge to the object detection model. Not every region in the image features is equally important. For low-contrast images caused by light or blurred images caused by noise such as rain and snow, traditional convolution may ignore some weak feature signals. Therefore, different convolution kernels should be set for different feature inputs, including the size, shape of the convolution kernel, and the weights of the convolution results. Based on this idea, this paper sets up a dynamic convolution module. Dynamic convolution can extract local features in the image more meticulously, and these local features are particularly important under low visibility conditions. Dynamic convolution captures key information in the image at a finer-grained level by adjusting the convolution kernel, such as the contours and textures of the target. The dynamic convolution module calculates weights and flexibly selects appropriate convolution kernels to process these images with large differences, optimizes for specific situations such as haze, rain, or snow under different environmental conditions, helps the model accurately locate and identify targets under low visibility conditions, thus better adapting to these low visibility environments, improving the fineness of image feature extraction, and helping the model more accurately identify targets.

[0009] The dynamic convolution module for each input feature , the dynamic convolution module calculates the attention through cascaded adaptive pooling layers, fully connected layers, and ReLU activation layers. Based on this, it dynamically changes the size of the convolution kernel and the weights of the convolution. The weights of the convolution are multiplied by the results of the convolution calculation to obtain the final result of each convolution. The results of each dynamic convolution are batch-normalized through the BN layer and activated by the ReLU activation layer to output the final dynamic convolution result. The final feature representation after dynamic convolution is .

[0010] Preferably, the feature fusion includes a global attention gate (GAG) module; Low visibility conditions such as raindrops, snowflakes, strong light, and haze degrade the image quality to varying degrees, and the model's discriminative ability for fruit targets under low visibility thus decreases. Therefore, this paper designs a learnable adaptive global attention gate. This parameter-learnable normalization method enables the model to learn noise features such as low image contrast and haze occlusion caused by low visibility during feature decoding calculations, and adaptively adjusts the model parameters to enhance the model's discriminative ability for targets. GAG performs instance normalization on the input features, and then linearly combines the original input with the normalized result, that is, multiplies the original input by the normalized result, and then adds the product to the normalized result and the original input element-wise; this way of linear combination can ensure that the model learns both the features after adaptive normalization and retains the details in the original features, ensuring that the model does not favor one type of feature expression, thereby avoiding model overfitting. The global attention gate calculation can be expressed as: ; ; where, represents instance normalization, , represent learnable weight and bias parameters, represents mean calculation, represents variance calculation, represents element-wise addition, represents element-wise multiplication, represents the final feature after linear combination; R represents the feature shape, B represents the number of images in the batch input, C represents the number of feature channels, H and W represent the original width and height of the feature, and n represents the feature level; To enhance the model's discriminative ability for targets and backgrounds under low visibility and reduce the adverse effects caused by the occlusion of targets by medium rain and snow under low visibility conditions, a spatial attention mechanism is applied to the features; the features of different channels play different roles in target detection under different low visibility noises. To balance the weights of the features of each channel, a channel attention mechanism is applied after the spatial attention mechanism to dynamically adjust the weights of different channels during feature decoding, ensuring that the important features corresponding to different noises are fully utilized. Specifically, it is expressed as: ; Among them, represents channel attention, represents channel attention, represents element-wise multiplication, represents the features output by the attention mechanism.

[0011] Preferably, feature decoding includes a decoder based on cross-attention, Under low visibility agricultural scenarios, the noise distribution is complex. In the image, it is manifested that the targets are mainly concentrated in a certain area, while there are more pure background areas without targets. Learning the global noise features can help the model distinguish the feature learning and target discrimination specific to the target area. A local attention module LAB and a global attention module GAB are constructed to learn local features and global features respectively; under low visibility agricultural scenarios, the background is more complex, and the target detection task has high requirements for details. Further, a cross-shaped cross-attention module is designed to enhance the perception ability of different directions and scales by performing cross-attention calculations in the horizontal and vertical directions, which can help the model effectively fuse multi-scale spatial information, effectively capture the complex associations between fruit targets and the surrounding environment in the image, and can dynamically adjust the focus of attention according to the feature distribution in different regions of the image, thereby reducing the interference of bad weather on fruit detection. The decoding process is expressed as: ; ; ; ; Among them, represents the local attention module, is the feature of the input CAD, represents the cascaded "Conv-BN-SiLU-Conv-BN" layer, represents the global attention module, represents adaptive average pooling, represents the feature representation after the fusion of local features and global features, represents the rough prediction map output by the CAD, represents channel attention, represents a convolutional layer, represents concatenation in the channel dimension, represents cross - attention.

[0012] Preferably, the iterative optimization includes an Iterative Feature Attention (IFA) module. After each level of features is decoded by CAD, a corresponding rough prediction map is output. Higher - level features contain prediction results with more abstract and higher - level semantic information, while lower - level features contain more basic information such as shape and texture. Usually, the rough prediction map decoded from the higher - level features is connected to the next - level features to ensure that the model makes full use of all semantic features and basic features. The iterative feature attention mechanism IFA explores the relationship between the rough prediction map and the next - level features from the local range to the global range of the features, calculates the attention using the rough prediction map of the previous level and the features of the current level, and guides the current - layer decoder to pay more attention to the target area, enhancing the model's discrimination ability for the target. IFA can be expressed as: ; ; ; ; where, and represent the attention values calculated by two parts, represents the sigmoid function, represents the local attention module, represents the global attention module, represents the rough prediction map of the decoder, represents the next - level features, represents the transitional features between the attention of the front and back parts of IFA, represents the optimized features output by IFA; n represents the feature level.

[0013] In addition to performing iterative feature attention between different decoders in each iteration, starting from the second iteration, iterative feature attention is also performed between different rounds of iteration. The iteration between different rounds is expressed as: ; where, represents the features after the iterative update of the 4th - level features in the (i + 1) - th iteration, represents the features before the iterative update of the 4th - level features in the (i + 1) - th iteration, represents the final prediction map in the i - th iteration, represents iterative feature attention; After the last iteration is completed, in order to retain the basic information such as shape and texture contained in the underlying features, the final optimized features are upsampled to the same size and then fused with the bottommost features . After concatenating the two features, the final prediction result is obtained by fusing with the multi-scale convolution module MC composed of convolution layers of multiple scales , which can be specifically expressed as: ; where represents the feature fusion module composed of multi-scale convolution layers, represents upsampling

[0014] Preferably, the loss supervision is specifically set as follows: Feature optimization is set in the decoding stage, and a rough feature map is output in each iteration. The finally set loss function supervises the loss of the rough prediction maps of multiple steps of iteration and the loss of the final prediction map R, so that each iteration is constrained by the loss function; in order to accurately evaluate the loss and enable the model to accurately detect difficult targets, the weighted binary cross-entropy loss and the weighted intersection over union loss are used to supervise these prediction results, so that the supervision function can efficiently evaluate the error during verification in the model training process. The prediction loss is ; for the rough prediction maps in the iterative optimization stage, different weights are set for different iteration rounds; given the weight parameter , the final loss function is defined as follows: ; In the formula, R is the final prediction map, GT represents the ground truth, represents the prediction output of each level of decoder, represents the weight parameter, and i represents the decoder level corresponding to the prediction map

[0015] Preferably, step S5 includes: Input the image and the point cloud pseudo-feature map into the model for training, and predict and output a binary mask image for the fruit images in the test set; For the simultaneous input of the image and the point cloud pseudo-feature map, they are concatenated in the batch dimension to obtain the input feature as , B represents the number of images input in a batch, C represents the number of channels of the feature, H and W represent the height and width of the input feature. In each feature downsampling, the width and height are downsampled to half of the original; after training is completed, the trained model is tested

[0016] Preferably, step S6 includes comprehensive evaluation using the IOU and metrics, specifically: ; ; Among them, TP represents the number of samples that are actually positive classes and are predicted as positive classes, FN represents the number of samples that are actually positive classes and are predicted as negative classes, TN represents the number of samples that are actually negative classes and are predicted as negative classes, and FP represents the number of samples that are actually negative classes and are predicted as positive classes.

[0017] The beneficial effects of the present invention are as follows: 1. The present invention designs various convolutional neural network modules at different stages of encoding and decoding. Each module helps the model learn noise features, pay more attention to the target area, and combines them into an overall detection method. Through the learning of image features by the deep neural network of deep learning in this solution, including multi-scale encoding, dynamic encoding, and local and global attention fusion, etc., it is ensured that the model can accurately detect the target in only one stage, solving the problem that the existing methods rely on image quality for detection and simplifying the training steps; the cooperation of each module in this solution enables the model to adapt to various harsh weather scenarios and has strong generalization ability; the design of each module in this solution and the training data set used for the model are closely related to the agricultural scenario, so the proposed solution is completely suitable for fruit detection in the agricultural scenario. The proposed one-stage harsh weather fruit detection method with dynamic encoding effectively solves various inadaptability problems of the existing methods in the agricultural scenario under harsh weather, improves the real-time performance and accuracy, and finally achieves better detection effects.

[0018] 2. The present invention cleans, annotates, and enhances the data of the collected data set in a specific way to obtain "simple" samples and "difficult" samples for training a harsh weather detection model suitable for the agricultural scenario, and tests the performance of the model under different difficulties; designs a complete detection method, which includes a multi-scale dynamic encoder, normalized feature fusion, a decoder based on cross-shaped attention, and a feature optimization iteration mechanism, which can effectively improve the discrimination ability of the model for fruit targets under harsh weather; establishes a detection benchmark for fruit detection in the harsh weather agricultural scenario with single-modal data input. This solution uses IOU as an index to evaluate the effectiveness and reliability of the detection method, and verifies that this solution can achieve the best detection performance; compared with the existing methods, the model has the smallest performance degradation in the conversion from "simple" samples to "difficult" samples. Brief Description of the Drawings

[0019] Figure 1 is a schematic flow chart of the present invention; Figure 2 is a schematic diagram of the structure of the dynamic convolution layer DCB in an embodiment of the present invention; Figure 3 is a structural diagram of the decoder CAD based on cross-shaped attention in an embodiment of the present invention; Figure 4 It is the structural diagram of the iterative feature attention module in the embodiment of the present invention; Figure 5 It is the sample diagram of the detection effect of this method in the embodiment of the present invention; Figure 6 It is the sample diagram of the detection effect of other methods in the embodiment of the present invention. Specific implementation manner

[0020] Embodiment 1: As Figure 1 shown, a one-stage bad weather fruit detection method based on dynamic coding includes the following steps: S1, Collect fruit image data in the agricultural scene on the spot; The agricultural scene specifically includes the farmland of individual business households, operating farms, university experimental fields and large-scale planting bases on the spot; S2, Perform data preprocessing on the collected image data, including data format conversion, name unification, data cleaning, data annotation and data augmentation, and divide the preprocessed image data into a training set and a test set; S3, Set the hyperparameters of the one-stage bad weather fruit detection model to be constructed based on dynamic coding; S4, Input the fruit images in the training set, and construct and train a one-stage bad weather fruit detection model based on dynamic coding; S5, Detect the fruits in the test set images and output the detection result mask; S6, Conduct a comprehensive evaluation of the model.

[0021] Preferably, step S2 includes: S201, Name the image data obtained in step S1 in sequence, and uniformly convert the image data into the png format; S202, Delete the images with insufficient challenges and no fruit targets; S203, Label the fruits in the images as binary mask pictures as the ground truth; S204, Degrade the image quality, add weather conditions of rain, snow and fog to the images, use the original data set as the "simple" samples, and the data-augmented data set as the "difficult" samples, and perform the same training set and test set division on the two types of samples.

[0022] Preferably, the hyperparameters set in step S3 include the initial learning rate, learning rate change, number of training epochs, optimizer, loss function and iterative weight parameters.

[0023] Preferably, step S4 includes inputting training set image data, constructing a one-stage bad weather fruit detection model based on dynamic encoding for bad weather noise features, learning and predicting the position and shape of fruits, and outputting a binary mask image; specifically as follows: The training process of the one-stage bad weather fruit detection model based on dynamic encoding is divided into feature encoding, feature fusion, feature decoding, and feature iterative optimization; feature encoding includes a dynamic feature encoder module, a multi-scale feature encoder module, and a dynamic convolution module; The Pyramid Vision Transformer (PVT) has shown excellent performance in object detection tasks. Different from traditional Transformer models, PVT adopts a pyramid structure when processing image data to better capture multi-scale information in images. Specifically, PVT gradually reduces the resolution at different levels, similar to pyramid pooling in convolutional neural networks (CNNs), so as to be able to process large-size images more efficiently. By introducing hierarchical feature extraction in a pyramid shape, it not only improves the computational efficiency but also better preserves the detailed information of the image. This enables PVT to exhibit superior performance in tasks such as image classification and object detection. The decoding stage can learn multi-scale features from high-level to low-level, which is very important for exploring the detailed information of the target. For a given input , the Pyramid Vision Transformer model PVT is used to perform multi-scale encoding on the features, and the final feature representation is , where represents features of different sizes, n represents different levels of feature sizes, R represents the feature shape, B represents the number of batch input images, C represents the number of feature channels, and H and W respectively represent the width and height of the features.

[0024] Preferably, under low visibility conditions, the environmental visibility is low and the light intensity difference is large, which will significantly affect the image quality, posing a huge challenge to the target detection model. Not every region in the image features is equally important. For low-contrast images caused by light or blurred images caused by noise such as rain and snow, traditional convolution may ignore some weak feature signals. Therefore, different convolution kernels should be set for different feature inputs, including the size, shape of the convolution kernel, and the weight of the convolution result. Based on this idea, this paper sets up a dynamic convolution module. Dynamic convolution can extract local features in the image more carefully, and these local features are particularly important under low visibility conditions. Dynamic convolution captures key information in the image at a finer granularity level by adjusting the convolution kernel, such as the contour and texture of the target. The dynamic convolution module calculates the weights and flexibly selects appropriate convolution kernels to process these images with large differences, optimizes for specific situations such as haze, rain, or snow under different environmental conditions, helps the model accurately locate and identify targets under low visibility conditions, thus better adapting to these low visibility environments, improving the fineness of image feature extraction, and helping the model more accurately identify targets.

[0025] As Figure 2 shown, the dynamic convolution module DCB for each input feature , the dynamic convolution module calculates the attention through a cascaded adaptive pooling layer, fully connected layer, and ReLU activation layer. Based on this, it dynamically changes the size of the convolution kernel and the weight of this convolution. The weight of the convolution is multiplied by the result of the convolution calculation to obtain the final result of each convolution. The result of each dynamic convolution is batch-normalized by the BN layer and activated by the ReLU activation layer to output the final dynamic convolution result. The feature after the final dynamic convolution is represented as .

[0026] Preferably, the feature fusion includes a global attention gate GAG module; Low visibility conditions such as raindrops, snowflakes, strong light, and haze cause varying degrees of degradation in image quality, and the model's ability to discriminate fruit targets in low visibility is therefore reduced. Therefore, this paper designs a learnable adaptive global attention gate. This parameter-learnable normalization method enables the model to learn noise features such as low image contrast and haze occlusion caused by low visibility during feature decoding calculations, and adaptively adjusts model parameters to enhance the model's ability to discriminate targets. GAG performs instance normalization on the input features, and then linearly combines the original input with the normalized result, that is, multiplying the original input with the normalized result, and then adding the product and the normalized result to the original input element by element; this linear combination method ensures that the model can learn the adaptively normalized features while retaining the details in the original features, ensuring that the model does not favor one feature expression, thereby avoiding model overfitting. The global attention gate calculation can be expressed as: ; ; in, represents instance normalization, , represents the learnable weight and bias parameters, represents the mean calculation, represents variance calculation, represents element-by-element addition, represents element-wise multiplication, represents the final feature after linear combination; R represents the feature shape, B represents the number of images in the batch input, C represents the number of feature channels, H and W represent the original width and height of the feature, and n represents the feature level; In order to enhance the model's ability to distinguish targets and backgrounds in low visibility and reduce the adverse effects of rain and snow on target occlusion in low visibility conditions, a spatial attention mechanism is applied to the features; the features of different channels play different roles in target detection under different low visibility noises. In order to balance the weights of the features of each channel, a channel attention mechanism is used after the spatial attention mechanism to dynamically adjust the weights of different channels during feature decoding to ensure that the important features corresponding to the alleviation of different noises can fully play their role, which is specifically expressed as follows: ; in, represents channel attention, represents channel attention, represents element-wise multiplication, Represents the features of the attention mechanism output.

[0027] like Figure 3As shown, the feature decoding includes a cross-attention based decoder CAD. In low visibility agricultural scenarios, the noise distribution is complex, manifested in the image as the target mainly concentrated in a certain area, while more are pure background areas without targets. Learning the global noise features can help the model distinguish the feature learning specific to the target area and target discrimination. A local attention module LAB and a global attention module GAB are constructed to learn local features and global features respectively. The agricultural scenario under low visibility has a more complex background, and the target detection task has high requirements for details. Further, a cross cross-attention module CAD is designed to enhance the perception ability of different directions and scales by performing cross-attention calculations in the horizontal and vertical directions, which can help the model effectively fuse multi-scale spatial information and effectively capture the complex associations between fruit targets and the surrounding environment in the image. And it can dynamically adjust the focus of attention according to the feature distribution in different regions of the image, thereby reducing the interference of bad weather on fruit detection. The decoding process is expressed as: ; ; ; ; Among them, represents the local attention module, is the feature input to CAD, represents the cascaded "Conv-BN-SiLU-Conv-BN" layer, represents the global attention module, represents adaptive average pooling, represents the feature representation after fusing local features and global features, represents the rough prediction map output by CAD, represents channel attention, represents the convolutional layer, represents concatenation in the channel dimension, represents cross cross-attention.

[0028] Such as Figure 4As shown, preferably, the iterative optimization includes an iterative feature attention (IFA) module. After each level of features is decoded by CAD, a corresponding rough prediction map is output. The higher-level features contain prediction results with more abstract and higher-level semantic information, while the lower-level features contain more basic information such as shape and texture. Usually, the rough prediction map decoded from the higher-level features is connected to the next-level features to ensure that the model makes full use of all semantic features and basic features. The iterative feature attention mechanism IFA explores the relationship between the rough prediction map and the next-level features from the local range to the global range of the features, calculates the attention using the rough prediction map of the previous level and the features of the current level, and guides the current layer decoder to pay more attention to the target area, enhancing the model's discrimination ability for the target. IFA can be expressed as: ; ; ; ; where, and represent the attention values calculated by two parts, represents the sigmoid function, represents the local attention module, represents the global attention module, represents the rough prediction map of the decoder, represents the features of the next level, represents the transitional features between the attention of the front and back parts of IFA, represents the optimized features output by IFA; n represents the feature level.

[0029] In addition to performing iterative feature attention between different decoders in each iteration, starting from the second iteration, iterative feature attention is performed between different rounds of iteration. The iteration between different rounds is expressed as: ; where, represents the features after the iterative update of the 4th-level features in the (i + 1)-th iteration, represents the features before the iterative update of the 4th-level features in the (i + 1)-th iteration, represents the final prediction map of the i-th iteration, represents the iterative feature attention; After the last iteration is completed, in order to retain the basic information such as shape and texture contained in the underlying features, the final optimized features are upsampled to the same size and then fused with the bottommost features . After the two features are concatenated, they are fused by a multi-scale convolution module MC composed of convolution layers of multiple scales to obtain the final prediction result , which can be specifically expressed as: ; Among them represents a feature fusion module composed of multi-scale convolutional layers, represents upsampling.

[0030] Preferably, the loss supervision is specifically set as follows: Feature optimization is set in the decoding stage, and a rough feature map is output in each iteration. The finally set loss function supervises the loss of the rough prediction map of multiple steps of iteration and the loss of the final prediction map R, so that each iteration is constrained by the loss function; in order to accurately evaluate the loss and enable the model to accurately detect difficult targets, the weighted binary cross-entropy loss and the weighted intersection over union loss are used to supervise these prediction results, so that the supervision function can efficiently evaluate the error during verification in the model training process, and the prediction loss is ; for the rough prediction map in the iterative optimization stage, different weights are set for different iteration rounds; given the weight parameter , the final loss function is defined as follows: ; In the formula, R is the final prediction map, GT represents the ground truth, represents the prediction output of each level of decoder, represents the weight parameter, and i represents the decoder level corresponding to the prediction map.

[0031] Preferably, step S5 includes: Input the picture and the point cloud pseudo-feature map into the model for training, and predict the fruit image in the test set and output a binary mask image; For the simultaneous input of the picture and the point cloud pseudo-feature map, they are concatenated in the batch dimension to obtain the input feature as , B represents the number of images input in a batch, C represents the number of channels of the feature, H and W represent the height and width of the input feature. In this scheme, B = 8, C = 3, H = W = 704. In each feature downsampling, the width and height are downsampled to half of the original; after training is completed, the trained model is tested, and the input size of the model during testing is also , and the input batch is 1, that is, only one image is input for each test.

[0032] Preferably, in step S6, in order to evaluate the performance of the model, Intersection over Union (IOU), F1-measure ( ) are used for comprehensive evaluation, and the formula is specifically: ; ; The confusion matrix is shown in Table 1 below: Table 1: Confusion matrix;

[0033] Among them, TP represents the number of samples that are actually positive and predicted as positive, FN represents the number of samples that are actually positive and predicted as negative, TN represents the number of samples that are actually negative and predicted as negative, and FP represents the number of samples that are actually negative and predicted as positive.

[0034] Example 2: In order to objectively test the fruit detection method in bad weather of the present invention, in this example, "simple" samples and "difficult" samples after field collection and data augmentation are used to verify the reliability of this method under various conditions. The ratio of the training set to the test set is set to 8:2. The data of the training set and the test set are randomly allocated, but it is ensured that the "simple" and "difficult" samples are allocated in the same way.

[0035] At the same time, in order to verify the superiority of this method, this example respectively designs comparative experiments with (A) the current best detection model; (B) the denoising model + the best detection model; (C) the target segmentation method dedicated to bad weather. The experimental results are shown in Table 2: Table 2: Comparative experimental results;

[0036] From the experimental data in Table 2, it can be seen that the dynamic coding, cross-attention decoding and iterative optimization used in this paper have achieved remarkable effects in enhancing the performance of the detection method under low visibility. Compared with the second-ranked method, it has improved by 2% in terms of evaluation indicators, and has strong anti-interference ability. The performance loss from "simple" samples to "difficult" samples is the least, which proves the reliability and robustness of this solution.

[0037] Figure 5 and Figure 6 By showing the comparison of the detection results of the method in this paper and other methods, it can be seen that the method in this paper can achieve higher detection accuracy on any bad weather and different fruit targets, and the "false detection" and "missed detection" are significantly reduced compared with other methods.

[0038] In view of the problems of low real-time performance, poor applicability, and lack of direct targeting at agricultural scenarios in the existing methods, the present invention proposes to design a one-stage fruit detection method based on dynamic coding suitable for agricultural scenarios. Through multi-scale dynamic feature coding, normalized feature fusion, cross-shaped multi-directional attention feature decoding, and feature optimization iteration mechanism, a fruit detection method with strong robustness and high reliability in bad weather is proposed. A large number of experimental results show that the one-stage fruit detection method based on dynamic coding proposed by the present invention can effectively solve the problems of low applicability of the existing model and poor performance in agricultural scenarios, and can detect fruits with the highest performance in extremely challenging scenarios compared with the existing methods. Facing scenes from simple to difficult, the performance loss of this solution is the lowest; during the model training process, it only relies on the learning ability of the model, the decoding ability of features, and the feature optimization mechanism, without the need for pre-processing of images, and achieves good results in terms of accuracy and speed, providing a reliable solution for fruit detection in bad weather agricultural scenarios.

Claims

1. A one-stage bad weather fruit detection method based on dynamic coding, characterized in that: The following steps are involved: S1, collect fruit image data in agricultural scenes; agricultural scenes specifically include farmlands of individual business owners, commercial farms, university experimental fields and large-scale planting bases; S2, preprocessing the collected image data, including data format conversion, name unification, data cleaning, data labeling and data enhancement, and dividing the preprocessed image data into training sets and test sets; S3, setting the hyperparameters of the one-stage severe weather fruit detection model based on dynamic encoding to be constructed; S4, inputting the fruit images of the training set, constructing and training a one-stage severe weather fruit detection model based on dynamic coding; S5, detecting the fruits in the test set image and outputting the detection result mask; S6, conduct a comprehensive evaluation of the model.

2. A one-stage bad weather fruit detection method based on dynamic coding according to claim 1, characterized in that: Step S2 includes: S201, naming the image data acquired in step S1 in order, and converting the image data into png format; S202, deleting images that are not challenging enough and do not have fruit targets; S203, marking the fruit in the image as a binary mask image as a true value; S204, degrades the image quality by adding rain, snow and foggy weather conditions to the image. The original dataset is taken as the "easy" sample, and the dataset after data enhancement is taken as the "difficult" sample. The two samples are divided into the same training set and test set.

3. A one-stage bad weather fruit detection method based on dynamic coding according to claim 1, characterized in that: The hyperparameters set in step S3 include initial learning rate, learning rate change, number of training rounds, optimizer, loss function, and iterative weight parameters.

4. A one-stage bad weather fruit detection method based on dynamic coding according to claim 1, characterized in that: Step S4 includes inputting training set image data, constructing a one-stage severe weather fruit detection model based on dynamic coding for severe weather noise characteristics, learning and predicting the position and shape of the fruit, and outputting a binary mask image; specifically, as follows: The training process of the one-stage severe weather fruit detection model based on dynamic coding is divided into feature coding, feature fusion, feature decoding and feature iterative optimization; feature coding includes dynamic feature encoder module, multi-scale feature encoder module and dynamic convolution module; For a given input , the visual pyramid Transformer model PVT is used to perform multi-scale encoding on the features, and the final feature representation is .

5. A one-stage bad weather fruit detection method based on dynamic coding according to claim 4, characterized in that: The dynamic convolution module is used for each input feature. The dynamic convolution module obtains attention through the cascaded adaptive pooling layer, full connection, and ReLU activation layer. Based on this, the size of the convolution kernel and the weight of the convolution are dynamically changed. The weight of the convolution is multiplied by the result of the convolution calculation to obtain the final result of each convolution. The result of each dynamic convolution is batch normalized by the BN layer and activated by the ReLU activation layer to output the final dynamic convolution result. The final feature after dynamic convolution is represented as ;in, Represents features of different sizes, n represents the level of features of different sizes, R represents the feature shape, B represents the number of images input in a batch, C represents the number of feature channels, H and W represent the width and height of the feature respectively.

6. A one-stage bad weather fruit detection method based on dynamic coding according to claim 5, characterized in that: Feature fusion includes the global attention gate GAG ​​module; GAG performs instance normalization on the input features, and then linearly combines the original input with the normalized result, that is, multiplying the original input with the normalized result, and then adding the product, the normalized result and the original input element by element; the global attention gate calculation can be expressed as: ； ； in, represents instance normalization, , represents the learnable weight and bias parameters, represents the mean calculation, represents variance calculation, represents element-by-element addition, represents element-wise multiplication, represents the final feature after linear combination; R represents the feature shape, B represents the number of images in the batch input, C represents the number of feature channels, H and W represent the original width and height of the feature, and n represents the feature level; The spatial attention mechanism is applied to the features. The features of different channels play different roles in target detection under different low-visibility noises. In order to balance the weights of the features of each channel, the channel attention mechanism is used after the spatial attention mechanism to dynamically adjust the weights of different channels during feature decoding to ensure that the important features corresponding to the alleviation of different noises can fully play their role. The specific expression is: ； in, represents channel attention, represents channel attention, represents element-wise multiplication, Represents the features of the attention mechanism output.

7. A one-stage bad weather fruit detection method based on dynamic coding according to claim 6, characterized in that: Feature decoding includes a cross-attention based decoder, The local attention module LAB and the global attention module GAB are constructed to learn local features and global features respectively; the cross attention module is also included to enhance the perception ability of different directions and scales by performing cross attention calculations in the horizontal and vertical directions. The decoding process is expressed as: ； ； ； ； in, represents the local attention module, To input CAD features, represents the cascaded "Conv-BN-SiLU-Conv-BN" layer, represents the global attention module, represents adaptive average pooling, It represents the feature representation after the fusion of local features and global features. A rough prediction of the CAD output is shown. represents channel attention, represents the convolutional layer, Indicates connection in channel dimension, Indicates crossed attention.

8. A one-stage bad weather fruit detection method based on dynamic coding according to claim 7, characterized in that: Iterative optimization includes iterative feature attention IFA module. The IFA module explores the connection between the coarse prediction map and the next level features from the local range of features to the global range. It uses the coarse prediction map of the previous level and the features of the current level to calculate attention, guides the current layer decoder to pay more attention to the target area, and enhances the model's ability to identify the target. IFA can be expressed as: ； ； ； ； in, , Represents the attention value calculated from the two parts, represents the sigmoid function, represents the local attention module, represents the global attention module, represents the coarse prediction graph of the decoder, Indicates the characteristics of the next level, It represents the transition characteristics between the two parts of attention before and after IFA. represents the optimized features output by IFA; n represents the feature level; In addition to performing iterative feature attention between different decoders in each iteration, starting from the second round of iteration, iterative feature attention is performed between different rounds of iterations. The iterations between different rounds are expressed as: ； in, It represents the feature after the i+1th iterative update of the 4th level feature. represents the features before the i+1th level 4 feature iteration update, represents the final prediction graph of the i-th iteration, represents iterative feature attention; After the last iteration, in order to retain the basic information such as shape and texture contained in the underlying features, the final optimized features After upsampling to the same size, it is fused with the bottom-level features After the two features are concatenated, the multi-scale convolution module MC composed of convolutional layers of multiple scales is fused to obtain the final prediction result. , which can be specifically expressed as: ； in represents the feature fusion module composed of multi-scale convolutional layers, Indicates upsampling.

9. A one-stage bad weather fruit detection method based on dynamic coding according to claim 8, characterized in that: The specific settings of loss supervision are as follows: In the decoding stage, feature optimization is set, and a rough feature map is output in each iteration. The final loss function supervises the rough prediction map of multiple iterations. The loss of the final prediction graph R is used to constrain each round of iteration. In order to accurately evaluate the loss and enable the model to accurately detect difficult targets, a weighted binary cross entropy loss is applied. and weighted intersection-over-union loss Supervising these predictions allows the supervision function to efficiently evaluate the error during validation during model training, with the prediction loss being ; For the rough prediction graph in the iterative optimization phase, different weights are set for different iteration rounds; given the weight parameter , the final loss function is defined as follows: ； In the formula, R is the final prediction graph, GT represents the true value, represents the predicted output of each level of decoder, represents the weight parameter, and i represents the decoder level corresponding to the prediction graph.

10. A one-stage bad weather fruit detection method based on dynamic coding according to claim 1, characterized in that: Step S6 includes using IOU and The indicators are comprehensively evaluated, specifically: ； ； Among them, TP represents the number of samples that are actually positive and predicted to be positive, FN represents the number of samples that are actually positive and predicted to be negative, TN represents the number of samples that are actually negative and predicted to be negative, and FP represents the number of samples that are actually negative and predicted to be positive.

Images