Dense citrus quantity detection method, equipment, storage medium and device
By using the deformable convolutional network and SimAM attention mechanism of the DS-YOLO network model in dense citrus detection, the problem of low accuracy of dense citrus detection is solved, and citrus quantity detection with high accuracy and reliability is achieved.
Patent Information
- Application Number
- CN202210405888.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-18
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-04-18
AI Technical Summary
The prior art has low detection accuracy in intensive citrus detection, especially in natural environments, small-sized citrus, overlapping and obstructed citrus, and citrus with similar texture colors to the environment, resulting in serious missed inspections and missed inspections, making it difficult to achieve high accuracy. Dense fruit positioning.
Using the deformable convolutional network and SimAM attention mechanism based on the DS-YOLO network model, feature extraction and feature fusion of citrus images are performed. Through the deformable sampling points and attention weight adjustment, target feature maps of different scales are generated for use in Determine the coordinates and size of the target prediction box to accurately detect the number of citrus.
It improves the accuracy and reliability of dense citrus detection, effectively solves the problems of high similarity and easy overlap between targets in dense scenarios, and improves the detection accuracy and training speed of the model.
Smart Images

Figure CN114821102B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of citrus detection, and in particular to a method, equipment, storage medium and device for detecting the number of dense citrus. Background Art
[0002] With the rise and development of deep learning, smart agriculture and agricultural automation have received more and more attention. Using deep learning for target detection has become a current research hotspot. How to use computer vision technology to realize the recognition of smaller-sized citrus in natural environments, the recognition of overlapping and occluded citrus, the recognition of citrus with similar texture colors to the environment, and the recognition of citrus in multi-angle images that will cause repeated counting have become difficult problems in dense citrus detection. How to accurately locate dense fruits has become an important prerequisite for achieving early fruit yield estimation, and also provides effective technical support for picking robots.
[0003] At present, mainstream deep learning-based target detection algorithms focus on detecting images with scattered and regular targets. They are less effective when applied to dense scenes. In addition, the similarity between targets in dense scenes is very high, and the target intervals are small, which makes it easy to miss detections. In natural scenes, there are many citrus fruits, and some of them may be too small, stick together, occlude each other, or be blocked by branches and leaves. Currently, there are few detection methods for dense targets. If the detection methods used for sparse targets are used for dense targets, missed detections and false detections will be particularly serious, and the detection accuracy will not be high.
[0004] Object detection is a part of computer vision. It mainly describes the entire image content and combines the characteristic information of the target object to determine the category and location of the object. The dense object detection task is similar to general object detection. Both require the location of the target in the input image. However, due to the particularity of dense scenes, dense object detection does not require the labeling of the category of each target. The objects in the detected image usually have dense arrangement and easy overlap, such as crowds, supermarket goods, and parking lot vehicles.
[0005] The current general object detection methods based on deep learning can be roughly divided into two types: one is a two-stage method based on region proposal; the other is a one-stage end-to-end method: YOLO series, SSD, RetinaNet, etc.
[0006] In the first stage of the end-to-end detection method, the existing YOLO algorithm limits the number of neighboring targets that YOLO can detect due to spatial constraints, resulting in low detection accuracy for small targets clustered together, leading to inaccurate positioning.
[0007] In the two-stage detection method based on region proposal, cropping will cause information loss, and scaling will cause the object to deform and lose its original proportion, thus affecting the accuracy of recognition; the SPPNet method introduces the spatial pyramid pooling layer (SpatialPyramid Pooling, SPP), realizes multi-size input, and makes the size of the input image arbitrary, but SPPNet calculates convolution features for the entire image, which includes multiple separate steps and has a slow detection speed.
[0008] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0009] The main purpose of the present invention is to provide a method, equipment, storage medium and device for detecting the number of dense citrus fruits, aiming to solve the technical problem that the detection effect of dense citrus fruits in the prior art is not ideal, resulting in low detection accuracy.
[0010] To achieve the above object, the present invention provides a method for detecting the number of dense citrus fruits, the method comprising the following steps:
[0011] Perform image preprocessing on the citrus image to be identified to obtain a processed target citrus image;
[0012] Based on the deformable convolutional network in the preset DS-YOLO network model, feature extraction is performed on the target citrus image to obtain a feature map containing dense citrus shape features and position features;
[0013] Perform feature fusion on the feature maps according to a preset SimAM attention mechanism to obtain target feature maps of different scales;
[0014] Determine the coordinates and size of the target prediction box according to the candidate box adjustment parameters corresponding to the target feature map;
[0015] The positions of the citrus fruits in the citrus image to be identified are marked according to the coordinates and size of the target prediction frame, and the number of the citrus fruits in the citrus image to be identified is determined according to the marking result.
[0016] Optionally, the step of extracting features of the target citrus image based on a deformable convolutional network in a preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features includes:
[0017] The target citrus image is sampled based on a deformable convolutional network in a preset DS-YOLO network model to obtain deformable sampling points;
[0018] Adaptively adjusting the weight coefficient of the deformable sampling point according to the target citrus image to obtain adjusted deformable sampling points;
[0019] Feature extraction is performed on the target citrus image according to the adjusted deformable sampling points to obtain a feature map containing dense citrus shape features and position features.
[0020] Optionally, the step of performing feature fusion on the feature map according to a preset SimAM attention mechanism to obtain target feature maps of different scales includes:
[0021] Calculate the neuron weights corresponding to the feature map according to a preset SimAM attention mechanism and a preset energy function, and determine the target neuron according to the calculation results;
[0022] The feature maps are subjected to feature fusion according to the target neurons to obtain target feature maps of different scales.
[0023] Optionally, the step of performing feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales includes:
[0024] Performing feature fusion on the feature map according to the target neuron to obtain a first feature map after feature fusion;
[0025] Performing anchor coordinate matching on the first feature map according to a preset K-Mean clustering algorithm to obtain a target detection scale;
[0026] Multi-scale detection is performed on the first feature map according to the target detection scale to obtain target feature maps of different scales.
[0027] Optionally, before the step of extracting features of the target citrus image based on a preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features, the step further includes:
[0028] The collected dense citrus images are input into the original YOLOv4 model to obtain the feature map of the preset scale;
[0029] According to the preset SPP network, the feature map of the preset scale is stacked and convolved to obtain the processed feature map;
[0030] Performing feature fusion processing on the processed feature graph based on a path aggregation network to obtain a second feature graph after feature fusion;
[0031] A DS-YOLO network model is constructed based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale.
[0032] Optionally, after the step of constructing a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale, the step further includes:
[0033] The convolutional layer of the residual unit of the residual module in the DS-YOLO network model is replaced with the improved deformable convolutional layer, and the preset SimAM attention mechanism is added to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model;
[0034] The new DS-YOLO network model is used as the preset DS-YOLO network model.
[0035] Optionally, after the step of using the new DS-YOLO network model as the preset DS-YOLO network model, the method further includes:
[0036] Testing the preset DS-YOLO network model using a dense citrus test set without image preprocessing to obtain a test result;
[0037] The test results are quality evaluated according to a preset average detection accuracy, and the accuracy of the test results is determined according to the quality evaluation result.
[0038] In addition, to achieve the above-mentioned purpose, the present invention also proposes a dense citrus quantity detection device, which includes a memory, a processor, and a dense citrus quantity detection program stored in the memory and executable on the processor, and the dense citrus quantity detection program is configured to implement the steps of dense citrus quantity detection as described above.
[0039] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a dense citrus number detection program is stored, and when the dense citrus number detection program is executed by a processor, the steps of the dense citrus number detection method described above are implemented.
[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a dense citrus number detection device, the dense citrus number detection device comprising:
[0041] An image preprocessing module is used to perform image preprocessing on the citrus image to be identified to obtain a processed target citrus image;
[0042] A feature extraction module, used to extract features of the target citrus image based on a deformable convolutional network in a preset DS-YOLO network model, to obtain a feature map containing dense citrus shape features and position features;
[0043] A feature fusion module is used to perform feature fusion on the feature maps according to a preset SimAM attention mechanism to obtain target feature maps of different scales;
[0044] A citrus detection module, used to determine the coordinates and size of the target prediction box according to the candidate box adjustment parameters corresponding to the target feature map;
[0045] The citrus detection module is further used to mark the positions of the citrus in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determine the number of citrus in the citrus image to be identified according to the marking result.
[0046] The present invention performs image preprocessing on the citrus image to be identified, obtains the processed target citrus image, and extracts features of the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model, obtains a feature map containing dense citrus shape features and position features, fuses the feature map according to the preset SimAM attention mechanism, obtains target feature maps of different scales, and determines the coordinates and size of the target prediction frame according to the candidate frame adjustment parameters corresponding to the target feature map; marks the position of the citrus in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determines the number of citrus in the citrus image to be identified according to the marking result. Since the present invention performs feature fusion on the citrus image to be identified based on the deformable convolutional network in the preset DS-YOLO network model and the preset SimAM attention mechanism, and detects the number of citrus in the citrus image to be identified according to the target prediction frame corresponding to the target feature map of different scales, the detection effect of the present invention on dense citrus is not ideal compared with the prior art, resulting in low detection accuracy. The present invention improves the detection accuracy of the model and realizes high-reliability dense citrus quantity detection to make up for the shortcomings of the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 It is a structural diagram of a dense citrus quantity detection device in a hardware operating environment involved in an embodiment of the present invention;
[0048] Figure 2 It is a schematic diagram of the flow chart of the first embodiment of the method for detecting the number of dense citrus fruits of the present invention;
[0049] Figure 3 This is a schematic diagram of the original YOLOv4 model structure of the first embodiment of the dense citrus quantity detection method of the present invention;
[0050] Figure 4 A schematic diagram of the SimAM attention mechanism of the first embodiment of the dense citrus quantity detection method of the present invention;
[0051] Figure 5This is a schematic diagram of the DS-YOLO model structure of the first embodiment of the method for dense citrus quantity detection of the present invention;
[0052] Figure 6 It is a schematic diagram of the flow chart of the second embodiment of the method for detecting the number of dense citrus fruits of the present invention;
[0053] Figure 7 This is a flow chart of the deformable convolution implementation of the second embodiment of the dense citrus quantity detection method of the present invention;
[0054] Figure 8 This is a structural block diagram of the first embodiment of the device for detecting the number of dense citrus fruits according to the present invention.
[0055] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0056] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0057] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a dense citrus quantity detection device in the hardware operating environment involved in the embodiment of the present invention.
[0058] like Figure 1 As shown, the intensive citrus quantity detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable memory (Non-volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0059] Those skilled in the art will understand that Figure 1The structure shown in the figure does not constitute a limitation on the intensive citrus quantity detection device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0060] like Figure 1 As shown, the memory 1005 identified as a computer storage medium may include an operating system, a network communication module, a user interface module, and a dense citrus number detection program.
[0061] exist Figure 1 In the dense citrus quantity detection device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the user device; the dense citrus quantity detection device calls the dense citrus quantity detection program stored in the memory 1005 through the processor 1001, and executes the dense citrus quantity detection method provided by the embodiment of the present invention.
[0062] Based on the above hardware structure, an embodiment of the dense citrus quantity detection method of the present invention is proposed.
[0063] Reference Figure 2 , Figure 2 It is a flow chart of the first embodiment of the method for detecting the number of dense citrus fruits of the present invention, and provides the first embodiment of the method for detecting the number of dense citrus fruits of the present invention.
[0064] In this embodiment, the method for detecting the number of dense citrus fruits comprises the following steps:
[0065] Step S10: performing image preprocessing on the citrus image to be identified to obtain a processed target citrus image.
[0066] It should be noted that the executor of this embodiment may be a device with a function of intensive citrus quantity detection. The device may be a computing device including a citrus quantity detection system, such as a computer, a notebook, etc., or a device including a citrus quantity detection APP (such as a mobile phone, an iPad). The citrus quantity detection APP has a function of intensive citrus quantity detection. This embodiment uses a intensive citrus quantity detection device as an example, and this embodiment does not limit this. In this embodiment and the following embodiments, a computer is taken as an example to illustrate the intensive citrus quantity detection method of the present invention.
[0067] It is understandable that the citrus images to be identified may refer to citrus images collected in a natural environment that require citrus quantity detection. The citrus images include images of smaller sized citrus, overlapping and occluded citrus, and citrus with a grain color similar to the environment. For example: the collection object is an image of citrus grown naturally in a natural environment. In order to simulate the natural scene, when taking images, citrus plants are photographed on sunny days, cloudy days, and at different angles. The collection distance is 1.0m to 2.0m, totaling 2365 citrus sample images in different environments, each image contains 50 or more citrus, and the image resolution is 4032×3024 pixels.
[0068] It should be understood that the preprocessing of citrus images includes image cleaning, image cropping, image resizing and other processing operations, and the preprocessed images are input into the preset DS-YOLO network model for quantity detection.
[0069] In the specific implementation, the collected dense citrus images are sorted and classified, and after removing invalid and abnormal images, data augmentation operations are performed, including image cropping, flipping, brightness adjustment, etc., to avoid background interference or redundant target information, and to remove irrelevant or redundant features. For example: First, the collected dense citrus images are sorted and classified, and invalid and abnormal images are removed (such as: images with too biased shooting angles, and fruits that are difficult to detect, etc.), and after image cleaning, the image annotation tool LabelImg is used to mark the citrus in the image, and the location and category of the citrus are marked respectively. After the annotation is completed, the Crop method in the OpenCV library is used to crop part of the image to avoid background interference or redundant target information, and to remove irrelevant or redundant features to reduce the processing burden of model detection, and then the Resize method in the OpenCV library is used to adjust the image size to 416*416, and the processed image is saved in the preset folder.
[0070] Step S20: extracting features of the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features.
[0071] It should be noted that the preset DS-YOLO network model is a pre-trained model for detecting the number of dense citrus fruits. The preset DS-YOLO network model is a model built based on D-CSPDarnet53 (deformable convolutional feature extraction network), PANet, and Head detection algorithm.
[0072] It is understandable that, under normal circumstances, there are significant intra-class variations in the collection of outdoor orchard images. Due to different lighting conditions, camera viewpoints, etc., the sizes of the collected citrus images are variable and the number of citrus is large. Therefore, the deformable convolutional network in the preset DS-YOLO network model is used to extract features of the target citrus image to obtain a feature map containing dense citrus shape features and position features. Compared with traditional network models, which limit the ability of geometric transformation modeling and can only scan with windows of fixed shape and size, the preset DS-YOLO network model in this scheme can recognize the shapes and sizes of different dense citrus, thereby adaptively adjusting according to the shape and size to better extract features.
[0073] Furthermore, in order to improve the detection accuracy of the model, before step S20, the method includes: inputting the collected dense citrus image into the original YOLOv4 model to obtain a feature map of a preset scale; stacking and convolving the feature map of the preset scale according to a preset SPP network to obtain a processed feature map; performing feature fusion processing on the processed feature map based on a path aggregation network to obtain a second feature map after feature fusion; and constructing a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale.
[0074] It should be noted that by collecting dense citrus images and constructing an image dataset, the collected images can be named according to the format of the Pascal VOC dataset, and three folders named Annotations, ImageSets, and JPEGImages are created at the same time; the dataset is a self-built dataset. The image dataset consists of pictures taken in the orchard and images collected online. The four factors of shooting are shooting distance (0.5m / 1.0m), fruit size (large / small), fruit quantity (many / medium / few), and weather conditions (cloudy / sunny), totaling 2365 pictures, and the image distribution rate can be set to 4032×3024 pixels. Preprocess the images in the dense citrus image dataset, for example: first, sort and classify the collected dense citrus images, remove invalid images and abnormal images, and after the image cleaning is completed, use the image annotation tool LabelImg to mark the citrus in the image, and mark the location and category of the citrus respectively. After the annotation is completed, the image dataset is divided into a training set and a test set as the input of the model according to a certain ratio, where the training set accounts for 90% of the total number of samples and the test set accounts for 10% of the total number of samples. They are named train.txt and text.txt respectively and saved in the subfolder Main under the ImageSets folder. Then use the Crop method in the OpenCV library to crop part of the image to avoid background interference or redundant target information, eliminate irrelevant or redundant features, simplify the model, and then use the Resize method in the OpenCV library to adjust the image size to 416*416, and input the adjusted image into the original YOLOv4 model for training. The above values are not specifically limited and are only used as examples.
[0075] It is understandable that the preset scale refers to the scale for feature extraction based on the original YOLOv4 model, such as: inputting the adjusted image into the original YOLOv4 model, after passing through the backbone network, YOLOv4 will output feature maps of three scales of 13*13, 26*26, and 52*52. Feature maps of different scales contain semantic information of different dimensions. The preset SPP network refers to the network for calculating convolutional features in the original YOLOv4 model, such as: SPP (Spatial Pyramid Pooling) network structure. The path aggregation network refers to the network for fusing features. The above values are not specifically limited and are only used as examples.
[0076] It should be understood that the detection scale refers to the scale of the YOLO detection head corresponding to the second feature map. Each detection head scale includes candidate box adjustment parameters, which include confidence parameters, parameters for adjusting length, width and coordinate offset, and category parameters.
[0077] In the specific implementation, refer to Figure 3 The original YOLOv4 model structure diagram shown in the figure inputs the adjusted image into the original YOLOv4 model. After passing through the backbone network, YOLOv4 will output feature maps of three scales: 13*13, 26*26, and 52*52. In the feature fusion part, the feature map of 13*13 size will enter the SPP (Spatial Pyramid Pooling) structure. SPP stacks and convolves the obtained new feature map with the feature map before entering the network and outputs it to the feature fusion network (path aggregation network) PANet. PANet upsamples the 13*13 feature map twice, and the results are stacked with the 26*26 and 52*52 feature maps and then convolved. Then, a series of similar downsampling and stacked convolutions are performed from bottom to top to fully integrate the features of the three feature maps of different scales. Finally, three YOLO detection heads of 13*13, 26*26, and 52*52 are output respectively. Each detection head in the YOLOv4 algorithm will contain 3 sets of candidate box adjustment parameters, and each set of candidate box adjustment parameters will include 1 confidence parameter, 4 parameters for adjusting length, width and coordinate offset, and 20 category parameters (the VOC2007 dataset contains 20 categories). The DS-YOLO network model is constructed based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale. Through these adjustment parameters, the YOLOv4 algorithm will adjust the coordinates and width and height of the candidate box to generate the final prediction box, which is used to calibrate the position of the citrus in the citrus image, thereby ultimately determining the number of citrus.
[0078] Furthermore, after the step of constructing a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale, it also includes: replacing the convolution layer of the residual unit of the residual module in the DS-YOLO network model with an improved deformable convolution layer, and adding the preset SimAM attention mechanism to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model; and using the new DS-YOLO network model as the preset DS-YOLO network model.
[0079] It should be noted that the new DS-YOLO network model is obtained by improving the problem of missed detection caused by insufficient feature extraction capability of the original YOLOv4 model when the target is dense citrus. It not only further improves the model's detection capability for dense citrus with large size changes, but also improves the model's ability to extract citrus features, with fast training speed and high model accuracy. The DS-YOLO network model is built based on the original YOLOv4 model. Therefore, in order to solve the problem that the original YOLOv4 algorithm only uses three detection scales for target prediction, which easily causes missed detection for dense citrus, the convolutional layer of the residual unit of the residual module in the DS-YOLO network model is replaced with the improved deformable convolutional layer, and the preset SimAM attention mechanism is added to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model.
[0080] It is understandable that due to different lighting conditions, camera viewpoints, etc., the size of citrus images collected in outdoor orchards varies, and there are serious overlaps and occlusions between citrus and citrus, and between citrus and leaves, which causes the shape of citrus to deform, which brings great difficulties to citrus recognition. Therefore, in order to solve the problem of citrus overlap and occlusion, the improved deformable convolution is used to replace the convolution layer of some residual units of the CSPDarknet53 network in the DS-YOLO network model, that is, in the DCSP module of the D-CSPDarkNet53 network, more deformable convolution layers with offset learning capabilities are used, so that the model can determine the shape and size according to the changes in the citrus image. In the feature fusion network of the original YOLOv4, the SimAM attention mechanism is added. The SimAM attention mechanism combines the channel dimension and the spatial dimension, and derives the three-dimensional attention weights of the unified channel dimension and spatial dimension, so that the network pays more attention to the deep feature space information of dense citrus.
[0081] Furthermore, in order to detect the detection accuracy of the trained preset DS-YOLO network model, after the step of using the new DS-YOLO network model as the preset DS-YOLO network model, it also includes: testing the preset DS-YOLO network model through a dense citrus test set without image preprocessing to obtain a test result; performing a quality assessment on the test result according to a preset average detection accuracy, and determining the accuracy of the test result according to the quality assessment result.
[0082] It should be noted that the dense citrus test set without image preprocessing refers to an image dataset constructed from citrus images taken in an orchard and original citrus images collected online. The original citrus images in the image dataset are input into the trained preset DS-YOLO network model for dense citrus quantity detection, and the quantity results are output. The average detection accuracy is used as the main evaluation indicator of the model to evaluate the quality of the results of the dense citrus detection model, and the loss function is used as the objective function evaluation indicator to evaluate the degree of difference between the model prediction value and the true value.
[0083] It should be understood that the average detection accuracy is measured by the mAP (mean average precision) value, which indicates the proportion of correct predictions (positive and negative) to the total; precision refers to the proportion of correct predictions to all positive predictions; recall refers to the proportion of correct positive predictions to all actual positive predictions; and the F1 value is the harmonic mean of precision and recall.
[0084] In the specific implementation, after the model training is completed, the dense citrus test set without data preprocessing is used to verify the effect. Since the target is small, in order to effectively observe the model detection results, this paper captures a part of the detection result image. The results show that under the same training parameters, the DS-YOLO model proposed in this paper can accurately locate citrus that are numerous, small, dense, and occluded in natural scenes, while the original YOLOv4 model has a certain degree of missed detection and false detection, which further improves the average detection accuracy of dense citrus.
[0085] Step S30: Perform feature fusion on the feature map according to the preset SimAM attention mechanism to obtain target feature maps of different scales.
[0086] It should be noted that the preset SimAM attention mechanism is pre-set to improve the attention to the recognition of citrus features in the feature image. The preset SimAM attention mechanism combines the channel dimension and the spatial dimension, and derives the attention weights of the unified channel dimension and the spatial dimension for the feature map. This attention further enhances the network's ability to extract important information without adding parameters to the original network and does not increase the number of model parameters.
[0087] In the specific implementation, refer to Figure 4The schematic diagram of SimAM attention mechanism shown in the figure shows that the traditional convolutional network uses block convolution. The regular sampling points limit its ability to model geometric transformations and can only scan with windows of fixed shape and size. Each element of the deformable convolution kernel has a learnable parameter bias, so that the sampling points of the deformable convolution can be adaptively adjusted according to the feature map, and the receptive field can change with the shape and size of the object. Although the deformable convolution allows the sampling points to be adaptively adjusted according to the feature map for better feature extraction, it may introduce useless context information, such as background. Therefore, to address this problem, this method expands the scope of use of the deformable convolution layer in the YOLOv4 network, and uses more deformable convolution layers with offset learning capabilities in the CSPDarknet53 network, so that the deformable convolution can control sampling on a wider range of feature layers, and adds a weight coefficient to each sampling point on the basis of the original deformable convolution to distinguish whether the sampling point is the target area. If it is not the target area, the weight of this point is set to 0 to ensure the accurate extraction of the target feature.
[0088] Step S40: Determine the coordinates and size of the target prediction box according to the candidate box adjustment parameters corresponding to the target feature map.
[0089] It should be noted that the candidate box adjustment parameters corresponding to the target feature map refer to the candidate box adjustment parameters corresponding to different scales. Figure 5 The schematic diagram of the DS-YOLO model structure is shown in the figure. In this scheme, the features of four feature maps of different scales are obtained, and four YOLO detection heads of 13*13, 26*26, 52*52, and 104*104 are output respectively. Each detection head in the DS-YOLO algorithm will contain three sets of adjustment parameters for candidate boxes. The adjustment parameters of each set of candidate boxes will include one confidence parameter, four parameters for adjusting the length, width, and coordinate offset, and one category parameter. Through these adjustment parameters, the DS-YOLO algorithm will adjust the coordinates and size (width and height) of the candidate box to generate the target prediction box.
[0090] Step S50: Marking the positions of the citrus fruits in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determining the number of citrus fruits in the citrus image to be identified according to the marking result.
[0091] It should be noted that the citrus fruits in the citrus image to be identified are annotated according to the coordinates and size of the target prediction box, that is, the citrus fruits are framed by the prediction box, and the number of citrus fruits in the citrus image is determined according to the number of prediction boxes.
[0092] In this embodiment, the citrus image to be identified is preprocessed to obtain the processed target citrus image, and the deformable convolution network in the preset DS-YOLO network model is used to extract the features of the target citrus image, so as to obtain a feature map containing dense citrus shape features and position features, and the feature map is feature-fused according to the preset SimAM attention mechanism to obtain target feature maps of different scales, and the coordinates and size of the target prediction box are determined according to the candidate box adjustment parameters corresponding to the target feature map; the citrus in the citrus image to be identified is position-marked according to the coordinates and size of the target prediction box, and the number of citrus in the citrus image to be identified is determined according to the marking result. Since this embodiment performs feature fusion on the citrus image to be identified based on the preset DS-YOLO network model and the preset SimAM attention mechanism, and detects the number of citrus in the citrus image to be identified according to the target prediction box corresponding to the target feature map of different scales, the detection effect of this embodiment on dense citrus is not ideal compared with the prior art, resulting in low detection accuracy. This embodiment improves the detection accuracy of the model and realizes high-reliability dense citrus quantity detection to make up for the shortcomings of the prior art.
[0093] refer to Figure 6 , Figure 6 This is a flow chart of the second embodiment of the present invention. Figure 2 The first embodiment shown provides a second embodiment of the method for detecting the number of dense citrus fruits of the present invention.
[0094] In this embodiment, step S20 includes:
[0095] Step S201: sampling the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model to obtain deformable sampling points.
[0096] It should be noted that the deformable convolution network refers to a deformable convolution feature extraction network that replaces the convolution layer of some residual units of the original YOLOv4 network model CSPDarknet53 network. The feature extraction network is a network built based on the D-CSPDarnet53 algorithm. Since the deformable convolution network can control sampling on a wider range of feature layers, and add a weight coefficient to each sampling point on the basis of the original deformable convolution, it is used to distinguish whether the sampling point is the target area. If it is not the target area, the weight of this point is set to 0 to ensure the accurate extraction of the target feature.
[0097] It can be understood that the deformable sampling points are multiple sampling points set by the deformable convolutional network when sampling the feature map. The sampling points correspond to a weight coefficient, and the sampling area is judged to be the target area based on the size of the weight coefficient, thereby improving the accuracy of feature extraction.
[0098] Step S202: adaptively adjusting the weight coefficients of the deformable sampling points according to the target citrus image to obtain adjusted deformable sampling points.
[0099] It should be noted that in order to better learn the shape and size of citrus, each element of the deformable convolution kernel has a learnable parameter bias, so that the sampling points of the deformable convolution can be adaptively adjusted according to the feature map, so that the area of the sampling points can be changed with the shape and size of the object. In this way, the shape and size of citrus can be better learned to avoid recognition failure due to occlusion of citrus.
[0100] It is understandable that in order to better perform feature extraction and avoid the influence of useless information (such as background) on the recognition results, weight coefficients are added to the deformable sampling points, and the weight coefficients are adaptively adjusted. The target area is determined according to the weight coefficients, thereby ensuring the accurate extraction of the target features.
[0101] Step S203: extracting features of the target citrus image according to the adjusted deformable sampling points to obtain a feature map containing dense citrus shape features and position features.
[0102] In the specific implementation, the traditional convolution includes two steps: 1) sampling the input feature map through a fixed-size convolution kernel; 2) calculating the sampling according to the convolution kernel weight and adding the calculation results. For example: define the convolution kernel of size 3×3 and expansion 1 as R, and the traditional convolution sliding window offset vector is a 9-point square, with 8 points offset in 8 directions in the middle:
[0103] R={(-1,-1),(-1,0),...,(0,1),(1,1)};
[0104] Then for each position p0 on the output feature map y, the traditional convolution can be expressed as:
[0105]
[0106] Among them, each pixel point p in the convolution n There is a weight w, p0 represents each pixel output by the convolution window, and x is the input feature map.
[0107] Deformable convolution adds a 2D offset {Δp n |n=1,...,N},N=|R|.
[0108] like Figure 7The deformable convolution implementation flow chart shown in the figure shows that for each position p0 on the output feature map y, the calculation process of the deformable convolution is as follows:
[0109]
[0110] In order to solve the problem of some useless upper and lower information introduced by deformable convolution, a weight coefficient is added to each sampling point on the basis of the original deformable convolution to distinguish whether the sampling point is the target area. If it is not the target area, the weight of this point is set to 0, Δw n ∈[0,1].
[0111]
[0112] Due to the offset Δp n It is usually a decimal, so the convolution feature map x needs to be calculated using bilinear interpolation:
[0113]
[0114] Where p represents any position after the offset, that is, p = p0 + p n +Δp n , q is the spatial position of all points in the feature map, G(·,·) is the weight corresponding to the point, which can be divided into two one-dimensional kernels:
[0115] G(q,p)=g(q x ,p x )·g(q y ,p y );
[0116] in,
[0117] g(a,b)=max(0,1-|ab|).
[0118] In this embodiment, step S30 includes:
[0119] Step S301: Calculate the neuron weights corresponding to the feature map according to the preset SimAM attention mechanism and the preset energy function, and determine the target neuron according to the calculation results.
[0120] It should be noted that the preset energy function is a pre-set function for calculating the attention weight.
[0121] In the specific implementation, the SimAM attention mechanism is mainly based on neuroscience theory, and a pre-designed energy function is used to calculate the weight of attention. The weight function is defined as follows:
[0122]
[0123] in,
[0124]
[0125] Represent the target neuron and other neurons of the input feature map, X∈R C×H×W , i refers to the index in the spatial dimension, M = H × W represents the number of all neurons in a channel, and w t and b t Refers to the weight and bias of a neuron transformation, respectively, and introduces a binary label instead of y t and y0, with values of 1 and -1 respectively, to find the solution to minimize e t , to determine the linear separability of the current target neuron and other neurons, the final weight function is as follows:
[0126]
[0127] Theoretically, each channel has M = H × W weight functions, and the following analytical solution is obtained:
[0128]
[0129] in, Separately w t and b t Substituting into the original formula, we can find the minimum weight:
[0130]
[0131] From a statistical point of view, in order to reduce the amount of calculation, and can be replaced by the mean and variance respectively, the mean variance The smaller the value, the more separable the current target neuron is from other neurons, and the more important the target neuron is.
[0132] Step S302: performing feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales.
[0133] It should be noted that the feature fusion of the feature map is performed through the target neuron, so as to focus on the deep feature space information of dense citrus, thereby improving the rate and accuracy of feature fusion of the model.
[0134] Furthermore, the step S302 includes: performing feature fusion on the feature map according to the target neuron to obtain a first feature map after feature fusion; performing Anchor coordinate matching on the first feature map according to a preset K-Mean clustering algorithm to obtain a target detection scale; performing multi-scale detection on the first feature map according to the target detection scale to obtain target feature maps of different scales.
[0135] It should be noted that since the Anchor size of the original network is not suitable for dense targets, the feature fusion of the feature map is performed through the target neuron to obtain the first feature map after feature fusion, and the K-means clustering algorithm is used to match the Anchor coordinates, and the size of the Anchor and the target detection scale suitable for the first feature map data set are calculated. The first feature map is subjected to multi-scale detection according to the size of the Anchor and the target detection scale to obtain target feature maps of different scales.
[0136] In the specific implementation, in the feature fusion module, the K-means method is used to match the new Anchor coordinates, multiple detection scales are added, and the detection accuracy of the model is further improved, and the three detection scales in the original YOLOv4 network are expanded to four. The original YOLOv4 multi-scale detection structure has only three layers, which is easy to cause missed detection for dense targets. Therefore, on the basis of the original network structure, a 104×104 scale feature map is added to reduce the false detection rate of small targets. Since the anchor size of the original network is not suitable for dense targets, the K-Mean clustering method is used to calculate the 12 anchor sizes suitable for the first feature map data set, corresponding to four feature layers (detection scales), one feature layer corresponds to three anchor boxes, and applied to the training network, the 12 anchor boxes are: 3×4, 5×6, 6×8, 7×11, 8×10, 9×12, 10×14, 11×11, 11×16, 13×15, 14×18, 18×23. After the 11th CSP layer, the feature map is continued to be upsampled and other processes are performed to expand the feature map. At the same time, at the 19th CSP layer, the obtained feature map of size 52×52 is tensor-concatenated and fused with the feature map of the third layer in the backbone network to obtain a feature map of size 104×104 for the detection of small target objects. After the improvement, the entire model uses a total of four detection layers (detection scales) for detection, which further deepens the network depth and can better extract feature information in deeper networks, so as to enhance the model's ability to learn at multiple scales under dense targets, thereby better learning the multi-level feature information of dense targets and improving the model's detection performance in dense scenes.
[0137] In this embodiment, the citrus image to be identified is subjected to image preprocessing to obtain a processed target citrus image, and the target citrus image is sampled based on a deformable convolutional network in a preset DS-YOLO network model to obtain deformable sampling points; the weight coefficients of the deformable sampling points are adaptively adjusted according to the target citrus image to obtain adjusted deformable sampling points; feature extraction is performed on the target citrus image according to the adjusted deformable sampling points to obtain a feature map containing dense citrus shape features and position features, the neuron weights corresponding to the feature map are calculated according to a preset SimAM attention mechanism and a preset energy function, and the target neurons are determined according to the calculation results; feature fusion is performed on the feature map according to the target neurons to obtain target feature maps of different scales, and the coordinates and size of the target prediction box are determined according to the candidate box adjustment parameters corresponding to the target feature map; the position of the citrus in the citrus image to be identified is marked according to the coordinates and size of the target prediction box, and the number of citrus in the citrus image to be identified is determined according to the marking result. Since this embodiment performs feature fusion on the citrus image to be identified based on the preset DS-YOLO network model and the preset SimAM attention mechanism, and detects the quantity of citrus in the citrus image to be identified according to the target prediction box corresponding to the target feature map of different scales, this embodiment is not ideal for the detection of dense citrus compared with the prior art, resulting in low detection accuracy. This embodiment improves the detection accuracy of the model and realizes high-reliability dense citrus quantity detection to make up for the shortcomings of the prior art.
[0138] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a dense citrus number detection program is stored, and when the dense citrus number detection program is executed by a processor, the steps of the dense citrus number detection method described above are implemented.
[0139] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the device for detecting the number of dense citrus fruits according to the present invention.
[0140] like Figure 8 As shown, the dense citrus quantity detection device proposed in the embodiment of the present invention includes:
[0141] An image preprocessing module 10 is used to perform image preprocessing on the citrus image to be identified to obtain a processed target citrus image;
[0142] A feature extraction module 20 is used to extract features of the target citrus image based on a deformable convolutional network in a preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features;
[0143] A feature fusion module 30 is used to perform feature fusion on the feature maps according to a preset SimAM attention mechanism to obtain target feature maps of different scales;
[0144] A citrus detection module 40, configured to determine the coordinates and size of a target prediction box according to the candidate box adjustment parameters corresponding to the target feature map;
[0145] The citrus detection module 40 is further used to mark the positions of the citrus in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determine the number of citrus in the citrus image to be identified according to the marking result.
[0146] In this embodiment, the citrus image to be identified is preprocessed to obtain the processed target citrus image, and the deformable convolution network in the preset DS-YOLO network model is used to extract the features of the target citrus image, so as to obtain a feature map containing dense citrus shape features and position features, and the feature map is feature-fused according to the preset SimAM attention mechanism to obtain target feature maps of different scales, and the coordinates and size of the target prediction box are determined according to the candidate box adjustment parameters corresponding to the target feature map; the citrus in the citrus image to be identified is position-marked according to the coordinates and size of the target prediction box, and the number of citrus in the citrus image to be identified is determined according to the marking result. Since this embodiment performs feature fusion on the citrus image to be identified based on the preset DS-YOLO network model and the preset SimAM attention mechanism, and detects the number of citrus in the citrus image to be identified according to the target prediction box corresponding to the target feature map of different scales, the detection effect of this embodiment on dense citrus is not ideal compared with the prior art, resulting in low detection accuracy. This embodiment improves the detection accuracy of the model and realizes high-reliability dense citrus quantity detection to make up for the shortcomings of the prior art.
[0147] Furthermore, the feature extraction module 20 is also used to sample the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model to obtain deformable sampling points; adaptively adjust the weight coefficients of the deformable sampling points according to the target citrus image to obtain adjusted deformable sampling points; perform feature extraction on the target citrus image according to the adjusted deformable sampling points to obtain a feature map containing dense citrus shape features and position features.
[0148] Furthermore, the feature fusion module 30 is also used to calculate the neuron weights corresponding to the feature map according to a preset SimAM attention mechanism and a preset energy function, and determine the target neuron according to the calculation result; perform feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales.
[0149] Furthermore, the feature fusion module 30 is also used to perform feature fusion on the feature map according to the target neuron to obtain a first feature map after feature fusion; perform Anchor coordinate matching on the first feature map according to a preset K-Mean clustering algorithm to obtain a target detection scale; perform multi-scale detection on the first feature map according to the target detection scale to obtain target feature maps of different scales.
[0150] Furthermore, the dense citrus quantity detection device also includes: a model construction module, which is used to input the collected dense citrus image into the original YOLOv4 model to obtain a feature map of a preset scale; stack and convolve the feature map of the preset scale according to a preset SPP network to obtain a processed feature map; perform feature fusion processing on the processed feature map based on a path aggregation network to obtain a second feature map after feature fusion; and construct a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale.
[0151] Furthermore, the model building module is also used to replace the convolution layer of the residual unit of the residual module in the DS-YOLO network model with the improved deformable convolution layer, and add the preset SimAM attention mechanism to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model; and use the new DS-YOLO network model as the preset DS-YOLO network model.
[0152] Furthermore, the model building module is also used to test the preset DS-YOLO network model using a dense citrus test set without image preprocessing to obtain test results; perform quality assessment on the test results according to a preset average detection accuracy, and determine the accuracy of the test results according to the quality assessment results.
[0153] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.
[0154] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.
[0155] In addition, for technical details not fully described in this embodiment, reference can be made to the dense citrus quantity detection method provided in any embodiment of the present invention, and will not be repeated here.
[0156] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0157] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as names.
[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.
[0159] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for detecting the number of dense citrus fruits, characterized in that: The method for detecting the number of dense citrus fruits comprises the following steps: Perform image preprocessing on the citrus image to be identified to obtain a processed target citrus image; Based on the deformable convolutional network in the preset DS-YOLO network model, feature extraction is performed on the target citrus image to obtain a feature map containing dense citrus shape features and position features; Perform feature fusion on the feature maps according to a preset SimAM attention mechanism to obtain target feature maps of different scales; Determine the coordinates and size of the target prediction box according to the candidate box adjustment parameters corresponding to the target feature map; Marking the positions of the citrus fruits in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determining the number of citrus fruits in the citrus image to be identified according to the marking result; The step of extracting features of the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features includes: The target citrus image is sampled based on a deformable convolutional network in a preset DS-YOLO network model to obtain deformable sampling points; Adaptively adjusting the weight coefficient of the deformable sampling point according to the target citrus image to obtain adjusted deformable sampling points; Performing feature extraction on the target citrus image according to the adjusted deformable sampling points to obtain a feature map including dense citrus shape features and position features; The step of performing feature fusion on the feature map according to the preset SimAM attention mechanism to obtain target feature maps of different scales includes: Calculate the neuron weights corresponding to the feature map according to a preset SimAM attention mechanism and a preset energy function, and determine the target neuron according to the calculation results; Performing feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales; Before the step of extracting features from the target citrus image based on the preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features, the method further includes: The collected dense citrus images are input into the original YOLOv4 model to obtain the feature map of the preset scale; According to the preset SPP network, the feature map of the preset scale is stacked and convolved to obtain the processed feature map; Performing feature fusion processing on the processed feature graph based on a path aggregation network to obtain a second feature graph after feature fusion; Building a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale; The convolutional layer of the residual unit of the residual module in the DS-YOLO network model is replaced with the improved deformable convolutional layer, and the preset SimAM attention mechanism is added to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model; The new DS-YOLO network model is used as the preset DS-YOLO network model.
2. The method for detecting the number of dense citrus fruits as claimed in claim 1, characterized in that: The step of performing feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales includes: Performing feature fusion on the feature map according to the target neuron to obtain a first feature map after feature fusion; Performing anchor coordinate matching on the first feature map according to a preset K-Mean clustering algorithm to obtain a target detection scale; Multi-scale detection is performed on the first feature map according to the target detection scale to obtain target feature maps of different scales.
3. The method for detecting the number of dense citrus fruits as claimed in claim 1, characterized in that: After the step of using the new DS-YOLO network model as the preset DS-YOLO network model, the method further includes: Testing the preset DS-YOLO network model using a dense citrus test set without image preprocessing to obtain a test result; The test results are quality evaluated according to a preset average detection accuracy, and the accuracy of the test results is determined according to the quality evaluation result.
4. A dense citrus quantity detection device, characterized in that: The intensive citrus quantity detection device comprises: a memory, a processor, and a intensive citrus quantity detection program stored in the memory and executable on the processor. When the intensive citrus quantity detection program is executed by the processor, the steps of the intensive citrus quantity detection method according to any one of claims 1 to 3 are implemented.
5. A storage medium, characterized in that: The storage medium stores a dense citrus number detection program, which, when executed by a processor, implements the steps of the dense citrus number detection method according to any one of claims 1 to 3.
6. A device for detecting the number of dense citrus fruits, characterized in that: The dense citrus quantity detection device comprises: An image preprocessing module is used to perform image preprocessing on the citrus image to be identified to obtain a processed target citrus image; A feature extraction module, used to extract features of the target citrus image based on a deformable convolutional network in a preset DS-YOLO network model, to obtain a feature map containing dense citrus shape features and position features; A feature fusion module is used to perform feature fusion on the feature maps according to a preset SimAM attention mechanism to obtain target feature maps of different scales; A citrus detection module, used to determine the coordinates and size of the target prediction box according to the candidate box adjustment parameters corresponding to the target feature map; The citrus detection module is further used to mark the positions of the citrus in the citrus image to be identified according to the coordinates and size of the target prediction frame, and determine the number of citrus in the citrus image to be identified according to the marking result; The step of extracting features of the target citrus image based on the deformable convolutional network in the preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features includes: The target citrus image is sampled based on a deformable convolutional network in a preset DS-YOLO network model to obtain deformable sampling points; Adaptively adjusting the weight coefficient of the deformable sampling point according to the target citrus image to obtain adjusted deformable sampling points; Performing feature extraction on the target citrus image according to the adjusted deformable sampling points to obtain a feature map including dense citrus shape features and position features; The step of performing feature fusion on the feature map according to the preset SimAM attention mechanism to obtain target feature maps of different scales includes: Calculate the neuron weights corresponding to the feature map according to a preset SimAM attention mechanism and a preset energy function, and determine the target neuron according to the calculation results; Performing feature fusion on the feature map according to the target neuron to obtain target feature maps of different scales; Before the step of extracting features from the target citrus image based on the preset DS-YOLO network model to obtain a feature map containing dense citrus shape features and position features, the method further includes: The collected dense citrus images are input into the original YOLOv4 model to obtain the feature map of the preset scale; According to the preset SPP network, the feature map of the preset scale is stacked and convolved to obtain the processed feature map; Performing feature fusion processing on the processed feature graph based on a path aggregation network to obtain a second feature graph after feature fusion; Building a DS-YOLO network model based on the detection scale corresponding to the second feature map and the candidate box adjustment parameters corresponding to the detection scale; The convolutional layer of the residual unit of the residual module in the DS-YOLO network model is replaced with the improved deformable convolutional layer, and the preset SimAM attention mechanism is added to the path aggregation network of the DS-YOLO network model to generate a new DS-YOLO network model; The new DS-YOLO network model is used as the preset DS-YOLO network model.
