Jujube fruit detection method, device and equipment based on improved YOLOv5s, medium and product

Abstract

The invention provides a jujube fruit detection method and device based on improved YOLOv5s, equipment, a medium and a product. The method comprises the following steps: acquiring a current jujube fruit image shot by shooting equipment in real time; the current jujube fruit image is input into a jujube fruit detection model, a jujube fruit detection result output by the jujube fruit detection model is obtained, and the jujube fruit detection result comprises a bounding box, a confidence coefficient, a category probability and a number of jujube fruits; wherein the jujube fruit detection model is constructed based on an improved YOLOv5s algorithm model, and is obtained by training based on a training set and a verification set; the training set and the verification set comprise a plurality of historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; the reference detection result comprises a reference bounding box, a reference confidence coefficient, a reference category probability and a reference number of the jujube fruits. According to the scheme, the accuracy and efficiency of jujube fruit detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agricultural technology, and in particular to a jujube fruit detection method, device, equipment, medium and product based on improved YOLOv5s. Background Art

[0002] Jujube is an important cash crop in my country, widely used in food processing, medicine, and healthcare. The quality of jujube fruit directly affects its market value, making accurate and rapid jujube fruit inspection crucial during harvesting, sorting, and processing. Traditional manual inspection methods suffer from subjectivity, low efficiency, and high costs, making them inadequate for large-scale production. In recent years, deep learning-based object detection technology has been widely adopted in agriculture, providing a new solution for intelligent jujube fruit inspection.

[0003] Traditional fruit object detection algorithms are based on a combination of feature extractors and classifiers. They capture key visual information in images through manually designed feature descriptors, followed by classification and localization using shallow machine learning models. While initially successful, these algorithms have gradually exposed limitations due to their strong reliance on features and difficulty extracting effective features in complex backgrounds. In recent years, deep learning methods such as convolutional neural networks (CNNs) and region-based convolutional neural networks (R-CNNs) have driven breakthroughs in object detection technology, particularly in terms of accuracy and speed. Deep learning methods in the field of object detection can be broadly categorized into two types: region proposal-based methods and regression-based methods. Region proposal methods, such as Faster R-CNN, first generate candidate boxes followed by classification and regression. While these methods can achieve high detection accuracy, they are computationally intensive, have slow inference speed, and are unsuitable for real-time detection tasks. Regression-based methods such as YOLO (You Only Look Once) and SSD (Single Shot Multi Box Detector) achieve target detection and positioning through an end-to-end network structure. They have higher real-time performance and are widely used in practical scenarios. In particular, the YOLO series of models has become an important research direction in the current field of target detection due to its good balance between accuracy and speed.

[0004] Currently, many algorithms are being applied in the agricultural sector, achieving significant breakthroughs in crop recognition. However, for small fruits, such as jujubes, target recognition accuracy and efficiency are limited in complex situations, such as occlusion and overlap. In practice, deep learning models are typically deployed on mobile devices. Model size, detection accuracy, and speed are important metrics for evaluating models for target detection and mobile inference in complex natural environments. Existing jujube detection methods, while offering high accuracy, have a large number of parameters and computational complexity, and are slow. These methods fail to meet the demanding requirements for an effective balance between speed and accuracy, making direct deployment on resource-constrained devices difficult.

[0005] Therefore, the existing jujube fruit detection methods have poor detection efficiency and accuracy. Summary of the Invention

[0006] The present invention provides a jujube fruit detection method, device, equipment, medium and product based on improved YOLOv5s, which are used to solve the defects of poor detection efficiency and accuracy of the jujube fruit detection method in the prior art, realize a lightweight and high-precision jujube fruit detection method, and improve the detection efficiency and accuracy of the jujube fruit detection method.

[0007] The present invention provides a jujube fruit detection method based on improved YOLOv5s, comprising the following steps: Obtain the current jujube fruit image captured by the camera in real time; Inputting the current jujube fruit image into a jujube fruit detection model to obtain a jujube fruit detection result output by the jujube fruit detection model, wherein the jujube fruit detection result includes a bounding box, a confidence score, a category probability, and a quantity of the jujube fruit; Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and reference detection results corresponding to each historical jujube fruit image; the reference detection results include: a reference bounding box of the jujube fruit, a reference confidence, a reference category probability, and a reference quantity.

[0008] According to a jujube fruit detection method based on improved YOLOv5s provided by the present invention, before acquiring the current jujube fruit image captured in real time by a shooting device, the method further includes: Improve the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model; Among them, the improvements to the original YOLOv5s algorithm model include: replacing the 7th layer convolution module in the backbone network with the GhostConv module; replacing all convolution modules in the neck network with GhostConv modules, and replacing all C3 modules in the neck network with C3Ghost modules; introducing the CA attention mechanism in the 2nd and 4th layer C3 modules of the backbone network; introducing the CBAM attention mechanism in the 6th and 8th layer C3 modules of the backbone network; applying the CARAFE method in the neck network to replace the nearest neighbor interpolation method of the original YOLOv5s algorithm model; applying the Soft-NMS algorithm in the detection head to replace the NMS algorithm of the original YOLOv5s algorithm model.

[0009] According to a jujube fruit detection method based on improved YOLOv5s provided by the present invention, after improving the original YOLOv5s algorithm model to obtain the improved YOLOv5s algorithm model, the method further includes: An initial jujube fruit detection model was constructed based on the improved YOLOv5s algorithm model; Using the photographing equipment to photograph multiple varieties of jujube trees from multiple angles, a plurality of historical jujube fruit images are acquired; Marking the jujube fruit targets in each of the historical jujube fruit images, and determining a reference detection result corresponding to each of the historical jujube fruit images; Constructing a jujube fruit dataset and dividing the jujube fruit dataset into a training set, a validation set, and a test set; the jujube fruit dataset includes: a plurality of historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; The initial jujube fruit detection model is trained based on the training set and the validation set, and is validated by the test set to obtain the jujube fruit detection model.

[0010] According to a jujube fruit detection method based on improved YOLOv5s provided by the present invention, the jujube fruit target in each historical jujube fruit image is marked, and a reference detection result corresponding to each historical jujube fruit image is determined, including: Screening multiple historical jujube fruit images to remove simple, blurred, targetless, and unfocused historical jujube fruit images, and obtaining multiple selected historical jujube fruit images; The LabelImg software was used to label the jujube fruit targets in each selected historical jujube fruit image, and the reference detection results corresponding to each selected historical jujube fruit image were obtained.

[0011] According to a jujube fruit detection method based on improved YOLOv5s provided by the present invention, constructing a jujube fruit dataset and dividing the jujube fruit dataset into a training set, a validation set and a test set, comprising: Performing data enhancement processing on the plurality of historical jujube fruit images in the jujube fruit dataset, wherein the data enhancement processing comprises at least one of the following: rotation, translation, random brightness, scaling, cutout, and adding Gaussian noise; The data-enhanced jujube fruit dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0012] According to a jujube fruit detection method based on improved YOLOv5s provided by the present invention, before inputting the current jujube fruit image into the jujube fruit detection model, the method further includes: The current jujube fruit image is preprocessed, where the preprocessing includes at least one of the following: screening, data enhancement, normalization, image denoising, and format conversion.

[0013] The present invention also provides a jujube fruit detection device based on improved YOLOv5s, comprising the following modules: An acquisition module is used to acquire the current jujube fruit image captured by a shooting device in real time; a processing module, configured to input the current jujube fruit image into a jujube fruit detection model, and obtain a jujube fruit detection result output by the jujube fruit detection model, wherein the jujube fruit detection result includes a bounding box, a confidence score, a category probability, and a quantity of the jujube fruit; Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and reference detection results corresponding to each historical jujube fruit image; the reference detection results include: a reference bounding box of the jujube fruit, a reference confidence, a reference category probability, and a reference quantity.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for detecting jujube fruits based on the improved YOLOv5s.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for detecting jujube fruits based on the improved YOLOv5s.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for detecting jujube fruits based on the improved YOLOv5s.

[0017] The present invention provides a jujube fruit detection method, device, equipment, medium, and product based on improved YOLOv5s. By inputting the current jujube fruit image obtained by real-time shooting with a shooting device into a jujube fruit detection model, the jujube fruit detection result output by the jujube fruit detection model is obtained. The jujube fruit detection model is constructed based on an improved YOLOv5s algorithm model and trained based on a training set and a validation set. The improved YOLOv5s algorithm model significantly improves the detection accuracy of jujube fruits by introducing a lightweight module, an optimized attention mechanism, a lightweight upsampling operator, and an improved post-processing algorithm. In particular, it can more accurately and efficiently identify and locate jujube fruits in situations such as complex backgrounds, small targets, and overlapping targets. Therefore, the solution of the present invention improves the accuracy and efficiency of jujube fruit detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the jujube fruit detection method based on improved YOLOv5s provided by the present invention.

[0020] Figure 2 This is the second flow chart of the jujube fruit detection method based on improved YOLOv5s provided by the present invention.

[0021] Figure 3 It is a structural diagram of the 3CS-YOLOv5s network provided by the present invention.

[0022] Figure 4 Schematic diagram of the BottleNeck structure of the GhostConv module structure provided by the present invention.

[0023] Figure 5 It is a schematic diagram of the CA attention mechanism provided by the present invention.

[0024] Figure 6 It is a schematic structural diagram of C3_CA provided by the present invention.

[0025] Figure 7 It is a schematic diagram of the CBAM attention mechanism provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the C3_CBAM provided by the present invention.

[0027] Figure 9 It is a structural diagram of the CARAFE upsampling operator provided by the present invention.

[0028] Figure 10 It is a structural schematic diagram of the jujube fruit detection device based on improved YOLOv5s provided by the present invention.

[0029] Figure 11 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0031] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0032] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components is not necessarily limited to those components explicitly listed, but may include other components not explicitly listed or inherent to such products or devices. The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the function associated with the element.

[0033] The following specific embodiments are used to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Figures 1-9 The present invention describes the jujube fruit detection method based on improved YOLOv5s.

[0034] Figure 1 This is one of the flow charts of the jujube fruit detection method based on the improved YOLOv5s provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 and 102.

[0035] Step 101: Acquire the current jujube fruit image captured in real time by a shooting device.

[0036] Step 102: Input the current jujube fruit image into the jujube fruit detection model to obtain the jujube fruit detection result output by the jujube fruit detection model. The jujube fruit detection result includes the jujube fruit's bounding box, confidence, category probability, and quantity.

[0037] Among them, the jujube fruit detection model is built based on the improved YOLOv5s algorithm model and trained based on the training set and validation set; the training set and validation set include multiple historical jujube fruit images and the reference detection results corresponding to each historical jujube fruit image; the reference detection results include: the reference bounding box of the jujube fruit, the reference confidence, the reference category probability and the reference quantity.

[0038] In practical applications, the jujube fruit detection method based on the improved YOLOv5s can be implemented as a jujube fruit detection device based on the improved YOLOv5s. The jujube fruit detection device based on the improved YOLOv5s can be implemented in various ways, for example, through a computer program, such as application software, or a chip. It can also be implemented as a medium storing the relevant computer program, such as a USB flash drive or cloud disk, or through a physical device integrated or installed with the relevant computer program, such as a server or smart device.

[0039] The following describes a jujube fruit detection device based on the improved YOLOv5s as the execution body of the jujube fruit detection method based on the improved YOLOv5s.

[0040] Specifically, step 101 includes: obtaining a current jujube fruit image captured in real time by a shooting device.

[0041] In actual application, the jujube fruit is photographed in real time by a photographing device to obtain the current jujube fruit image. In this embodiment, there is no specific limitation on the photographing device, and the photographing device can be a smart phone, a digital camera, an industrial camera, a drone, an intelligent surveillance camera, etc.

[0042] It can be understood that the jujube fruit detection device based on the improved YOLOv5s is directly connected or communicated with the shooting device to obtain the current jujube fruit image captured by the shooting device in real time.

[0043] Furthermore, step 102 includes: inputting the current jujube fruit image into the jujube fruit detection model, obtaining the jujube fruit detection result output by the jujube fruit detection model, and the jujube fruit detection result includes the jujube fruit's bounding box, confidence, category probability, and quantity.

[0044] In this embodiment, the jujube fruit detection model is constructed based on an improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; the reference detection results include: a reference bounding box of the jujube fruit, a reference confidence, a reference category probability, and a reference quantity.

[0045] It's understandable that the improved YOLOv5s algorithm model is an improvement on the original YOLOv5s algorithm model. For example, by introducing lightweight modules (such as GhostConv and C3Ghost), attention mechanisms (such as CA and CBAM), lightweight upsampling operators (such as CARAFE), and improved post-processing algorithms (such as Soft-NMS), the improved YOLOv5s algorithm model is obtained. This shows that the improved YOLOv5s algorithm model is a lightweight object detection model that can achieve more efficient and accurate object detection.

[0046] On this basis, a jujube fruit detection model was constructed and trained based on the improved YOLOv5s algorithm model. This model can be used to perform target recognition on the current jujube fruit image, accurately and efficiently obtaining jujube fruit detection results. Therefore, the solution of this embodiment significantly improves the detection accuracy of jujube fruits, especially in complex backgrounds, small objects, and overlapping objects, enabling more accurate and efficient identification and location of jujube fruits.

[0047] It should be noted that this application can identify different types of jujube fruits. For convenience, different varieties of jujube can be labeled as the same category. In this case, the reference detection result includes the reference bounding box and reference confidence level of the jujube fruit. The detection result of the jujube fruit includes the bounding box, confidence level, and number of jujube fruits.

[0048] Optionally, in an example, after step 102, the method further includes: According to the detection results of the jujube fruits output by the jujube fruit detection model, the estimated yield of the jujube fruits corresponding to the jujube garden is calculated.

[0049] Specifically, for a date orchard whose yield is to be estimated, a camera is used to capture images of date fruits from different angles on each date tree. Multiple date fruit detection results are obtained through the aforementioned steps. Based on these multiple detection results, the total number of date fruits in each tree is determined, and the total number of date fruits in each tree is stored in a list corresponding to the orchard. Furthermore, based on the total number of date fruits in each tree in the list, the estimated date fruit yield for the orchard is calculated.

[0050] In practical applications, before applying the jujube fruit detection model to detect the current jujube fruit image, it is necessary to pre-build and train the jujube fruit detection model. In one embodiment, Figure 2 This is the second flow chart of the jujube fruit detection method based on the improved YOLOv5s provided by the present invention. Figure 2 As shown, before the above step 101, the method further includes steps 201 to 206.

[0051] Step 201: Improve the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model.

[0052] Step 202: construct an initial jujube fruit detection model based on the improved YOLOv5s algorithm model.

[0053] Step 203: Use a photographing device to photograph multiple varieties of jujube trees from multiple angles to acquire multiple historical jujube fruit images.

[0054] Step 204 : Label the jujube fruit targets in each historical jujube fruit image, and determine a reference detection result corresponding to each historical jujube fruit image.

[0055] Step 205: construct a jujube fruit dataset and divide the jujube fruit dataset into a training set, a validation set, and a test set; the jujube fruit dataset includes: a plurality of historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image.

[0056] Step 206: Train the initial jujube fruit detection model based on the training set and the validation set, and verify it with the test set to obtain the jujube fruit detection model.

[0057] Specifically, step 201 includes: improving the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model.

[0058] Among them, the improvements to the original YOLOv5s algorithm model include: replacing the 7th layer convolution module in the backbone network with the GhostConv module; replacing all convolution modules in the neck network with GhostConv modules, and replacing all C3 modules in the neck network with C3Ghost modules; introducing the CA attention mechanism in the 2nd and 4th layer C3 modules of the backbone network; introducing the CBAM attention mechanism in the 6th and 8th layer C3 modules of the backbone network; applying the CARAFE method in the neck network to replace the nearest neighbor interpolation method of the original YOLOv5s algorithm model; applying the Soft-NMS algorithm in the detection head to replace the NMS algorithm of the original YOLOv5s algorithm model.

[0059] The original YOLOv5s (You Only Look Once version 5 small) algorithm model refers to the existing YOLOv5s algorithm model. Specifically, the original YOLOv5s algorithm model includes the input end, backbone network (Backbone), neck network (Neck), and output end.

[0060] Specifically, the input side is responsible for receiving input images and preprocessing them to meet the model's input requirements. Specific operations on the input side include, but are not limited to, image resizing, normalization, and data augmentation. For example, the original YOLOv5s algorithm model uses Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling on the input side. Mosaic data augmentation stitches multiple images together using random scaling, cropping, and arrangement, enabling the model to recognize objects within a smaller range.

[0061] Furthermore, the backbone network is used to extract feature maps from the input image, providing rich feature representations for subsequent detection tasks. For example, the backbone network of the original YOLOv5s algorithm model includes: Focus module, Convolutional Layer (Conv) module, C3 module, and Spatial Pyramid Pooling - Fast (SPPF) module.

[0062] Among them, the Focus module is used to slice the input image, reduce the image resolution by half, and extract key features at the same time. The specific operation is to get a value for every other pixel in an image, and finally obtain a double-downsampled feature map without information loss. The advantage is that it improves computing power without losing information.

[0063] The Conv module is a standard convolutional module, consisting of a convolutional layer (conv2d), batch normalization (BN), and the Sigmoid Linear Unit (SiLU) activation function, used to extract key target features. In practice, the padding of the Conv2d layer in the Conv module is automatically calculated, and the stride is modified to determine the reduction factor of the feature map. In the backbone network, the stride of all Conv modules is 2, and the convolution kernel is 3. Therefore, the Conv module halves the width and height of the feature map each time, downsampling the feature map while extracting the target features.

[0064] The C3 module combines convolution operations with residual connections for feature extraction and fusion. The C3 module consists of three Conv modules and a Bottleneck module. In practice, after a feature map enters the C3 module, it is divided into two paths. The first path passes through a 1×1 convolutional layer. The second path passes through the Conv module and Bottleneck, first through a 1×1 convolutional layer, then through a 3×3 convolutional layer, and finally through another 1×1 convolutional layer. The first 1×1 convolutional layer reduces the number of channels by half, the 3×3 convolutional layer extracts local features, and the second 1×1 convolutional layer restores the number of channels. Initial dimensionality reduction helps the convolution kernel better understand feature information, while dimensionality increase helps extract more detailed features. Finally, a residual structure is used to sum the input and output to avoid the vanishing gradient problem.

[0065] Among them, the SPPF module is used to convert feature maps with non-fixed scales into a unified scale, that is, to concatenate the feature representations of the same feature map at different scales.

[0066] Furthermore, the neck network (Neck) is used to fuse feature maps of different scales, enhancing the expressiveness of features and providing richer contextual information to the detection head. For example, the neck network (Neck) comprises a feature fusion module, an upsampling module, and a convolution module. Feature fusion is used to fuse multiple feature maps of different scales output by the backbone network, typically achieved through upsampling and concatenation. The upsampling module uses upsampling methods (such as nearest neighbor interpolation) to scale low-resolution feature maps to high resolution. The convolution module performs further convolution on the fused feature maps, reducing computational effort and parameter count while maintaining feature diversity. Specifically, the neck architecture is a feature pyramid (FPN) structure that combines shallow graphical features with shallow semantic features. The neck network extracts relatively shallow features from the backbone network and concatenates them with deeper semantic features to obtain more complete features.

[0067] Furthermore, the output end is used to convert the feature map into the final detection result, including the bounding box, confidence, and category probability. For example, the output end includes: a multi-scale detection head, a bounding box regression module, a confidence prediction module, a category prediction module, and a post-processing module. The multi-scale detection head contains multiple detection heads, each responsible for detecting targets of different scales (such as large targets, medium targets, and small targets). The bounding box regression module uses a regression network to predict the bounding box coordinates of the target. The confidence prediction module is used to predict the confidence of the target's existence, that is, the probability of the target and the background. The category prediction module predicts the category probability of the target. The post-processing module uses the NMS (non-maximum suppression) algorithm to filter the predicted boxes, reduce redundant boxes, and retain the most reliable prediction results.

[0068] On this basis, the original YOLOv5s algorithm model is improved to obtain an improved YOLOv5s algorithm model.

[0069] For example, the improved YOLOv5s algorithm model is the triple-optimized YOLOv5s (3CS-YOLOv5s) model. Specifically, the triple optimization refers to lightweight design, attention mechanism optimization, and post-processing optimization. The lightweight design replaces traditional convolutional modules with lightweight convolutional modules (such as GhostConv) to reduce the model's parameter count and computational complexity. Furthermore, the lightweight universal upsampling operator (such as CARAFE) replaces traditional upsampling methods to improve the quality of feature map reconstruction. The attention mechanism optimization introduces coordinate attention (CA) and convolutional block attention modules (CBAM) into the backbone network to enhance the model's focus on key features and improve detection accuracy against complex backgrounds and small objects. Among them, post-processing optimization refers to using the Soft-NMS (Soft Non-Maximum Suppression) algorithm to replace the traditional NMS (Non-Maximum Suppression) algorithm, reducing target deletion and missed detection, and improving the model's recall rate and detection accuracy.

[0070] Figure 3 This is a schematic diagram of the structure of the 3CS-YOLOv5s network provided by the present invention. Figure 3 As shown, step 201 specifically includes: Step 1: Applying the principles of GhostNet, we designed the GhostConv and C3Ghost modules to construct a lightweight convolutional network. Specifically, we replaced the seventh convolutional module in the Backbone network and all convolutional modules in the Neck network with GhostConv, and replaced all C3 modules in the Neck network with C3Ghost. Specifically, GhostConv first convolves the input feature map with a regular convolution kernel with half the number of channels to extract key information from the input feature map and generate an intrinsic feature map. This intrinsic feature map is then linearly transformed to generate a Ghost feature map. This linear transformation uses depthwise convolution, which is computationally inexpensive and can generate a large number of feature maps without significantly increasing the computational load, thereby reducing the number of parameters and computation required. The intrinsic feature map and the generated Ghost feature map are concatenated to produce the final output feature map. This approach preserves key information from the original feature map while increasing feature diversity through the Ghost feature map.

[0071] For example, Figure 4 : is a schematic diagram of the structure of BottleNeck of the GhostConv module structure provided by the present invention, such as Figure 4 The figure shows two lightweight BottleNeck structure designs based on the GhostConv module, with Stride=1 and Stride=2 types respectively. The core is to replace the standard convolution with the GhostConv module, combined with residual connection and DWConv, to achieve a balance between computational efficiency and feature expression capabilities. Figure 4 (a) is a stride 1 model. The backbone consists of two Ghost modules connected in series. The first GhostConv module expands the number of channels, while the second GhostConv module reduces the number of channels to the same as the input channels. Since the stride is 1, the height and width of the input feature layer remain unchanged, increasing the depth of the network. Figure 4 (b) is a Stride = 2 type, in which a DWConv with a stride of 2 is added between the GhostConv modules in the backbone to compress the feature map height and width for feature map downsampling. The input feature map is added to the main path output via a skip connection, preserving the original feature information. The skip connection also fuses shallow and deep features, alleviating the vanishing gradient problem and improving network training stability. Batch normalization is used after each module to accelerate model convergence and stabilize the training process. The ReLU activation function is introduced to enhance nonlinear expression capabilities and prevent model overfitting.

[0072] Step 2: Add the Coordinate Attention (CA) mechanism to the C3 modules in the second and fourth layers of the Backbone network. As you can understand, since feature maps in shallow neural networks contain rich spatial information, adding the CA mechanism to the C3 modules in the second and fourth layers of the Backbone network improves the model's ability to extract features of small objects and occluded fruits in complex environments and reduces background interference. This guides the network to focus on key areas during the early stages of feature extraction, enhancing the model's ability to locate small objects (such as jujube fruits).

[0073] Specifically, Figure 5 is a schematic diagram of the CA attention mechanism provided by the present invention, such as Figure 5 As shown, the CA attention mechanism first pools each channel horizontally and vertically in the X and Y directions using pooling kernels of size (H, 1) and (1, W), respectively, generating feature maps of shape C×H×1 and C×1×W. This generated feature map enables the CA attention mechanism to capture long-range dependencies within a channel, helping to preserve precise location information and enabling the network to more accurately localize objects. The extracted feature maps are concatenated according to the spatial dimensions to produce a feature tensor of size C×1×(H+W). A 1×1 convolution is used to compress the number of channels to C / r, resulting in an output size of C / r×1×(H+W). BatchNorm and non-linear functions are then used to enhance feature representation. The processed feature map is then split into two parts along the spatial dimension, recovering the horizontal and vertical feature vectors. A 1×1 convolution is then performed to restore the number of channels to C, respectively. Finally, a sigmoid function is applied to the feature maps in both directions to generate the gh and gw coordinate attention. Finally, gh and gw are multiplied with the original input to obtain feature maps in the X and Y directions with attention weights. Figure 6 : is a schematic diagram of the structure of C3_CA provided by the present invention, wherein: Figure 6 (a) To improve the C3 module structure, Figure 6 (b) Improved structure of the Attention-BottleNeck part in the C3 module.

[0074] Step 3: Add the Convolutional Block Attention Module (CBAM) attention mechanism to the C3 modules in the 6th and 8th layers of the Backbone network. This improved architecture is called C3_CBAM. As can be understood, as forward propagation progresses, feature map information in deep neural networks propagates from the spatial dimension to the channel dimension. However, the reduction in feature map resolution and the expansion of channels can lead to the loss of some semantic information. Furthermore, a lightweight network can reduce the model's expressiveness, resulting in the loss of important information when detecting complex scenes or small objects. To enhance semantic information and compensate for the information loss caused by the lightweight network, while also better processing the characteristics of jujube fruits at different scales, shapes, and orientations, the CBAM attention mechanism is added to the C3 modules in the 6th and 8th layers of the Backbone network.

[0075] Specifically, Figure 7 is a schematic diagram of the CBAM attention mechanism provided by the present invention, such as Figure 7 As shown in Figure 2, the CBAM attention mechanism consists of a channel attention module and a spatial attention module. Each channel in a feature map typically represents a specific feature, such as the edge, texture, or color of a jujube fruit. The channel attention mechanism assigns a weight to each channel, highlighting those channels that are more important to the task and suppressing those that are less important. Figure 7 (a) is the channel attention module, such as Figure 7 As shown in (a), the channel attention module performs global max pooling and global average pooling on the input feature map F in the spatial dimension to extract global information. The two pooling results are then fed into a multi-layer perceptron (MLP) with shared parameters to generate two channel attention vectors. These are fused and activated to obtain the channel attention weights MC. The channel attention weights MC are multiplied by each channel of the original feature map to emphasize channels that are helpful for the current task and suppress irrelevant channels, resulting in an attention-weighted channel feature map. The spatial attention mechanism assigns a weight to each spatial location in the feature map, highlighting locations with key contextual information. Figure 7 (b) is the spatial attention module, such as Figure 7 As shown in (b), the spatial attention module generates the attention weight MS in the spatial dimension by performing a pooling operation on the feature map in the channel dimension, and then processing it through convolution and activation function. Figure 7 (c) is a hybrid attention module, such as Figure 7(c) shows that the resulting spatial attention weights (MS) are applied to the original feature map, weighting the features at each spatial location. This emphasizes important image regions and minimizes the influence of unimportant areas. Finally, the output features of the channel attention module and the spatial attention module are element-wise multiplied. This cascade of channel-spatial attention optimization enhances semantic feature recognition in deep networks. Figure 8 It is a structural schematic diagram of the C3_CBAM provided by the present invention.

[0076] Step 4: Apply CARAFE (Content-Aware ReAssembly of FEatures), a lightweight, universal upsampling operator, to replace the original model's Nearest Neighbor Interpolation method. Specifically, the nearest neighbor interpolation method only considers sub-pixel neighborhoods and cannot capture the rich semantic information required for dense prediction tasks. CARAFE, on the other hand, is a content-aware module with two core steps. First, it adaptively generates convolution kernels based on input features. Second, it convolves the input features with the generated dynamic convolution kernels to achieve an upsampling effect. As can be seen, using CARAFE improves the model's detection accuracy for small objects, thereby better preserving feature details and improving detection performance.

[0077] Specifically, Figure 9 Schematic diagram of the structure of the CARAFE upsampling operator provided by the present invention, such as Figure 9 As shown, the first is the kernel prediction module, the feature size is C×H×W Figure X , Channel Compressor Figure X The channel is compressed to Cm to reduce computational overhead. The content encoder then generates an adaptive convolution kernel for each pixel position and outputs an adaptive convolution kernel matrix of size σH×σW, where σ is the upsampling factor and kup is the upsampling kernel size. Finally, after normalization, it is input to the feature reconstruction module. The input of the feature reconstruction module is still the feature Figure X , and then the adaptive convolution kernel is weightedly combined with the input feature map pixel by pixel to obtain an upsampled feature Figure X ', effectively retaining the spatial details of the jujube fruit and improving detection accuracy.

[0078] Step 5: Replace the NMS algorithm in the post-processing phase with the Soft-NMS algorithm. In YOLOv5s, NMS is a core step in object detection post-processing, used to eliminate redundant detection boxes and retain the most reliable predictions. The NMS algorithm filters and ranks object boxes based on their confidence and overlap, resulting in high-quality detection results. However, in some cases, the NMS algorithm suffers from the problem of false object deletion. This occurs when an object appears in the overlapping area of another object and the two object boxes are very close. The box with the lower score will be deleted due to exceeding the intersection over union (IoU) threshold, resulting in object detection failure and reduced model recall. Jujube fruit grows densely, with high overlap and occlusion by branches and leaves. Using NMS results in a high probability of missed detections. Soft-NMS reduces the score of predicted boxes (rather than directly removing them) to reduce false object deletion and improve object retention.

[0079] Specifically, the NMS algorithm is replaced by the Soft-NMS algorithm, which smoothly decays the boxes whose IoU is greater than the threshold Nt instead of directly deleting them. The core idea is shown in the formula: in, Si Indicates the remaining i The score of the candidate prior frame, IoU is the intersection-over-union ratio of the current high-score frame and the frame to be processed, M represents the candidate prior box with a higher score, Bi Represents the candidate prior box to be processed, σ is the penalty factor, and the higher the overlap of the target fruit, the greater the attenuation of the score.

[0080] In summary, the improved YOLOv5s algorithm model can be obtained through the above steps. Furthermore, an initial jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model, and the initial jujube fruit detection model is trained to obtain a jujube fruit detection model.

[0081] Specifically, step 202 includes: constructing an initial jujube fruit detection model based on the improved YOLOv5s algorithm model.

[0082] Among them, the initial jujube fruit detection model refers to a model that has not been trained.

[0083] Step 203 includes: using a photographing device to photograph multiple varieties of jujube trees from multiple angles to acquire multiple historical jujube fruit images.

[0084] In this embodiment, the shooting device is not specifically limited, and the shooting device can be a smart phone, a digital camera, an industrial camera, a drone, an intelligent surveillance camera, etc. For example, the jujube tree varieties include but are not limited to: winter jujube, lucky jujube, teapot jujube, and pear jujube.

[0085] In one example, the data was collected at the Forestry and Fruit Research Institute of the Academy of Agricultural and Forestry Sciences in City A. The main jujube varieties included winter jujube, lucky jujube, teapot jujube, and pear jujube. Shooting took place from June to September 2024, between 7:00 AM and 6:00 PM, covering a range of weather conditions (sunny and overcast) and light intensities (morning and afternoon). The cameras were primarily three different models of mobile phones, all in automatic mode, capturing images from multiple angles at a distance of 20-200 cm. Taking into account occlusion caused by the jujube fruit, images were randomly collected in various conditions, including no occlusion, occlusion, and overlap, to ensure the comprehensiveness of the constructed dataset.

[0086] Step 204 includes: marking the jujube fruit targets in each historical jujube fruit image, and determining a reference detection result corresponding to each historical jujube fruit image.

[0087] Specifically, after collecting multiple historical jujube fruit images, the jujube fruit objects in each historical jujube fruit image are annotated, and a reference detection result corresponding to each historical jujube fruit image is determined. The reference detection result includes a reference bounding box of the jujube fruit, a reference confidence score, a reference category probability, and a reference number.

[0088] Optionally, in one embodiment, the above step 204 includes: Screening multiple historical jujube fruit images to remove simple, blurred, targetless, and unfocused historical jujube fruit images, and obtaining multiple selected historical jujube fruit images; The LabelImg software was used to label the jujube fruit targets in each selected historical jujube fruit image, and the reference detection results corresponding to each selected historical jujube fruit image were obtained.

[0089] Specifically, after collecting and obtaining a plurality of historical jujube fruit images, the plurality of historical jujube fruit images are screened to remove historical jujube fruit images with simple targets, blurred targets, no targets, and unfocusable targets, thereby obtaining a plurality of selected historical jujube fruit images.

[0090] Furthermore, LabelImg software was used to label the jujube fruit targets in each selected historical jujube fruit image, frame the target with a minimum circumscribed rectangular frame, obtain the reference bounding box of the jujube fruit, and determine the reference confidence and reference category probability.

[0091] LabelImg software is a widely used open-source image annotation tool, primarily used in computer vision and machine learning projects, to help users annotate image data for use in training object detection models. LabelImg software features a bounding box annotation feature. Users can drag a mouse to draw a bounding box on an image to mark the location of an object and assign it a category label. Specifically, in this study, LabelImg was used to annotate an image of jujube fruit, framing the fruit with a minimum bounding rectangle.

[0092] Step 205 includes: constructing a jujube fruit dataset, and dividing the jujube fruit dataset into a training set, a validation set, and a test set; the jujube fruit dataset includes: a plurality of historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image.

[0093] Optionally, in one embodiment, the step 205 includes: Performing data enhancement processing on the plurality of historical jujube fruit images in the jujube fruit dataset, wherein the data enhancement processing includes at least one of the following: rotation, translation, random brightness, scaling, cutout, and adding Gaussian noise; The data-enhanced jujube fruit dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0094] In practical applications, data augmentation is performed on the captured images to prevent overfitting, enable the model to learn more robust features, and improve generalization in complex situations. Data augmentation can include at least one of the following: rotation, translation, random brightness, scaling, cutout, and Gaussian noise addition.

[0095] In practice, the environment for recognizing jujube fruits in orchards is very complex, with strong light, backlight, dark light, and occlusion. It is understandable that data augmentation of the collected images can enhance the robustness of the model.

[0096] In one example, six methods (rotation, translation, random brightness, scaling, cutout, and Gaussian noise) were used to construct a method list. A random seed was used to generate a random number between 1 and 4, and the number of methods to be used was then randomly selected from the method list for data augmentation.

[0097] Furthermore, the data-enhanced jujube fruit dataset was divided into training set, validation set and test set in a ratio of 8:1:1.

[0098] Among them, the training set and the validation set are used to train the initial jujube fruit detection model, and the validation set is used to verify the current jujube fruit detection model. After the verification is passed, the jujube fruit detection model is obtained.

[0099] Step 206 includes: training the initial jujube fruit detection model based on the training set and the validation set, and validating it through the test set to obtain the jujube fruit detection model.

[0100] The following example illustrates the training steps of the jujube fruit detection model.

[0101] Specifically, the training environment is configured. The experimental platform is the Windows 11 operating system, the GPU is the NVIDIA GeForce GTX 4060 graphics card with a graphics card size of 8GB, the CPU is the Intel Core i5-10600kf, the environment configuration is Python 3.8.5+Pytorch 1.8.0+Cuda 11.3, and the improved model is YOLOv5s 6.0.

[0102] Furthermore, we configured the training hyperparameters: the input image size was 640×640 pixels, the training iterations were 300, the batch size was 32, the number of processes was set to 4, the optimizer was set to SGD, and the learning rate, momentum, and weight decay coefficient were set to 0.01, 0.937, and 0.0005, respectively.

[0103] Furthermore, the training set data is applied to start the initial jujube fruit detection model training. In each training iteration, the loss of the current jujube fruit detection model is calculated and the weight of the current jujube fruit detection model is updated.

[0104] Furthermore, during the training process, the current jujube fruit detection model will output training indicators, including loss value and accuracy index, after each round of training, and will be verified on the validation set to obtain the performance indicators of the current jujube fruit detection model.

[0105] Furthermore, the test set is used to test and evaluate the current jujube fruit detection model. Specifically, evaluation metrics are selected to evaluate the current jujube fruit detection model. For example, the evaluation metrics include, but are not limited to: precision (P), recall (R), mean average precision (mAP@0.5) at an IoU threshold of 0.5, mean average precision (mAP@0.5:0.95) at IoU thresholds ranging from 0.5 to 0.95 with a step size of 0.05, floating-point operations (FLOPs), number of parameters, memory usage of the weight model (weights size), and average inference time for a single image.

[0106] Among them, the accuracyP It represents the proportion of correctly predicted samples in all samples. The calculation formula is: Among them, the recall rate R It indicates the proportion of correctly predicted samples to all positive samples. The calculation formula is: Among them, the mean average precision mAP is the average precision of all categories, and the calculation formula is: In the above formula, TP represents the number of positive samples correctly identified as positive samples, FP represents the number of negative samples correctly identified as positive samples, FN represents the number of positive samples incorrectly identified as negative samples, and N represents the number of categories. When all categories of jujube fruits are considered as one category, N = 1.

[0107] As can be understood, on the one hand, the GhostNet concept was adopted in the last Conv layer and Neck network of the original YOLOv5s Backbone. The GhostConv and C3Ghost modules were designed to replace the original Conv and C3 modules. This reduced the number of model parameters and computational complexity, making the model more lightweight, improving inference speed, and enhancing its applicability on embedded devices. On the other hand, by embedding the CA and CBAM attention mechanisms in the Backbone C3 module, the model's focus on key features was enhanced, improving the robustness of object detection. Furthermore, the adoption of CARAFE, replacing the original YOLOv5s' nearest neighbor upsampling method, enhanced the reconstruction of high-resolution feature maps, enabling the model to perform better when processing jujube fruits of varying scales, shapes, and orientations. This improvement effectively enhances the ability to preserve detail at the object edge and reduces information loss caused by upsampling. Furthermore, the adoption of Soft-NMS, replacing the original NMS for object bounding box screening, effectively alleviates the problems of false and missed detections of densely packed and overlapping objects. This method achieves better detection performance when detecting densely packed jujube fruits. Therefore, on this basis, the jujube fruit detection model was constructed and trained based on the improved YOLOv5s algorithm model to perform jujube fruit detection, which can effectively improve the accuracy and efficiency of jujube fruit detection.

[0108] In addition, in a possible implementation manner, before step 102, the method further includes: The current jujube fruit image is preprocessed, where the preprocessing includes at least one of the following: screening, data enhancement, normalization, image denoising, and format conversion.

[0109] In this embodiment, the current jujube fruit image is preprocessed before being input into the jujube fruit detection model, which can effectively improve the accuracy and reliability of jujube fruit detection.

[0110] In the jujube fruit detection method based on improved YOLOv5s provided by the present invention, the current jujube fruit image obtained by real-time shooting of the shooting device is input into the jujube fruit detection model, and the jujube fruit detection result output by the jujube fruit detection model is obtained. Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on the training set and the validation set. The improved YOLOv5s algorithm model significantly improves the detection accuracy of jujube fruits by introducing a lightweight module, an optimized attention mechanism, a lightweight upsampling operator, and an improved post-processing algorithm. In particular, it can more accurately and efficiently identify and locate jujube fruits in situations such as complex backgrounds, small targets, and overlapping targets. Therefore, the solution of the present invention improves the accuracy and efficiency of jujube fruit detection.

[0111] The following describes the jujube fruit detection device based on the improved YOLOv5s provided by the present invention. The jujube fruit detection device based on the improved YOLOv5s described below and the jujube fruit detection method based on the improved YOLOv5s described above can refer to each other.

[0112] Figure 10 Schematic diagram of the structure of the jujube fruit detection device based on the improved YOLOv5s provided by the present invention. Figure 10 As shown, the jujube fruit detection device based on the improved YOLOv5s includes: an acquisition module 1001 and a processing module 1002.

[0113] The acquisition module 1001 is used to acquire the current jujube fruit image captured in real time by a shooting device.

[0114] The processing module 1002 is used to input the current jujube fruit image into the jujube fruit detection model to obtain the jujube fruit detection result output by the jujube fruit detection model. The jujube fruit detection result includes the jujube fruit's bounding box, confidence, category probability, and quantity.

[0115] Among them, the jujube fruit detection model is built based on the improved YOLOv5s algorithm model and trained based on the training set and validation set; the training set and validation set include multiple historical jujube fruit images and the reference detection results corresponding to each historical jujube fruit image; the reference detection results include: the reference bounding box of the jujube fruit, the reference confidence, the reference category probability and the reference quantity.

[0116] Optionally, in a possible implementation manner, the above-mentioned jujube fruit detection device based on improved YOLOv5s further includes: Improvement module, used to improve the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model; Among them, the improvements to the original YOLOv5s algorithm model include: replacing the 7th layer convolution module in the backbone network with the GhostConv module; replacing all convolution modules in the neck network with GhostConv modules, and replacing all C3 modules in the neck network with C3Ghost modules; introducing the CA attention mechanism in the 2nd and 4th layer C3 modules of the backbone network; introducing the CBAM attention mechanism in the 6th and 8th layer C3 modules of the backbone network; applying the CARAFE method in the neck network to replace the nearest neighbor interpolation method of the original YOLOv5s algorithm model; applying the Soft-NMS algorithm in the detection head to replace the NMS algorithm of the original YOLOv5s algorithm model.

[0117] The training module is used to build an initial jujube fruit detection model based on the improved YOLOv5s algorithm model; An acquisition module is used to use a camera to shoot multiple varieties of jujube trees from multiple angles to acquire multiple historical jujube fruit images; a labeling module, for labeling the jujube fruit targets in each historical jujube fruit image and determining a reference detection result corresponding to each historical jujube fruit image; A partitioning module is used to construct a jujube fruit dataset and divide the jujube fruit dataset into a training set, a validation set, and a test set; the jujube fruit dataset includes: multiple historical jujube fruit images and reference detection results corresponding to each historical jujube fruit image; The training module is used to train the initial jujube fruit detection model based on the training set and the validation set, and verify it through the test set to obtain the jujube fruit detection model.

[0118] Optionally, in a possible implementation manner, the above-mentioned annotation module is specifically used to: Screening multiple historical jujube fruit images to remove simple, blurred, targetless, and unfocused historical jujube fruit images, and obtaining multiple selected historical jujube fruit images; The LabelImg software was used to label the jujube fruit targets in each selected historical jujube fruit image, and the reference detection results corresponding to each selected historical jujube fruit image were obtained.

[0119] Optionally, in a possible implementation manner, the above-mentioned division module is specifically used to: Performing data enhancement processing on the plurality of historical jujube fruit images in the jujube fruit dataset, wherein the data enhancement processing includes at least one of the following: rotation, translation, random brightness, scaling, cutout, and adding Gaussian noise; The data-enhanced jujube fruit dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0120] In addition, in a possible implementation manner, the above device further includes: The preprocessing module is used to preprocess the current jujube fruit image, and the preprocessing includes at least one of the following: screening, data enhancement, normalization, image denoising, and format conversion.

[0121] In the jujube fruit detection device based on improved YOLOv5s provided by the present invention, the processing module inputs the current jujube fruit image obtained by real-time shooting of the shooting device into the jujube fruit detection model, and obtains the jujube fruit detection result output by the jujube fruit detection model. Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on the training set and the validation set. The improved YOLOv5s algorithm model significantly improves the detection accuracy of jujube fruits by introducing a lightweight module, an optimized attention mechanism, a lightweight upsampling operator, and an improved post-processing algorithm. In particular, it can more accurately and efficiently identify and locate jujube fruits in situations such as complex backgrounds, small targets, and overlapping targets. Therefore, the solution of the present invention improves the accuracy and efficiency of jujube fruit detection.

[0122] Figure 11 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 11 As shown, the electronic device may include: a processor (processor) 1110 , a communication interface (Communications Interface) 1120 , a memory (memory) 1130 and a communication bus 1140 , wherein the processor 1110 , the communication interface 1120 , and the memory 1130 communicate with each other via the communication bus 1140 . The processor 1110 can call the logic instructions in the memory 1130 to execute the jujube fruit detection method based on the improved YOLOv5s, which includes: obtaining the current jujube fruit image obtained by real-time shooting of the shooting device; inputting the current jujube fruit image into the jujube fruit detection model to obtain the jujube fruit detection result output by the jujube fruit detection model, and the jujube fruit detection result includes the jujube fruit bounding box, confidence, category probability and quantity; wherein, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on the training set and the validation set; the training set and the validation set include multiple historical jujube fruit images and the reference detection result corresponding to each historical jujube fruit image; the reference detection result includes: the reference bounding box, reference confidence, reference category probability and reference quantity of the jujube fruit.

[0123] Furthermore, the logic instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the jujube fruit detection method based on the improved YOLOv5s provided by the above methods, the method including: obtaining a current jujube fruit image obtained by real-time shooting of a shooting device; inputting the current jujube fruit image into a jujube fruit detection model to obtain the jujube fruit detection result output by the jujube fruit detection model, and the jujube fruit detection result includes the jujube fruit bounding box, confidence, category probability and quantity; wherein, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; the reference detection result includes: a reference bounding box, a reference confidence, a reference category probability and a reference quantity of the jujube fruit.

[0125] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the jujube fruit detection method based on the improved YOLOv5s provided by the above-mentioned methods, the method comprising: obtaining a current jujube fruit image obtained by real-time shooting of a shooting device; inputting the current jujube fruit image into a jujube fruit detection model to obtain a detection result of the jujube fruit output by the jujube fruit detection model, the detection result of the jujube fruit including the bounding box, confidence, category probability and quantity of the jujube fruit; wherein, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; the reference detection result includes: a reference bounding box, a reference confidence, a reference category probability and a reference quantity of the jujube fruit.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0127] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A jujube fruit detection method based on improved YOLOv5s, characterized in that: include: Obtain the current jujube fruit image captured by the camera in real time; Inputting the current jujube fruit image into a jujube fruit detection model to obtain a jujube fruit detection result output by the jujube fruit detection model, wherein the jujube fruit detection result includes a bounding box, a confidence score, a category probability, and a quantity of the jujube fruit; Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and reference detection results corresponding to each historical jujube fruit image; the reference detection results include: a reference bounding box of the jujube fruit, a reference confidence, a reference category probability, and a reference quantity.

2. The jujube fruit detection method based on improved YOLOv5s according to claim 1, characterized in that: Before acquiring the current jujube fruit image captured in real time by the shooting device, the method further includes: Improve the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model; Among them, the improvements to the original YOLOv5s algorithm model include: replacing the 7th layer convolution module in the backbone network with the GhostConv module; replacing all convolution modules in the neck network with GhostConv modules, and replacing all C3 modules in the neck network with C3Ghost modules; introducing the CA attention mechanism in the 2nd and 4th layer C3 modules of the backbone network; introducing the CBAM attention mechanism in the 6th and 8th layer C3 modules of the backbone network; applying the CARAFE method in the neck network to replace the nearest neighbor interpolation method of the original YOLOv5s algorithm model; applying the Soft-NMS algorithm in the detection head to replace the NMS algorithm of the original YOLOv5s algorithm model.

3. The jujube fruit detection method based on improved YOLOv5s according to claim 2, characterized in that, After improving the original YOLOv5s algorithm model to obtain an improved YOLOv5s algorithm model, the method further includes: An initial jujube fruit detection model was constructed based on the improved YOLOv5s algorithm model; Using the photographing equipment to photograph multiple varieties of jujube trees from multiple angles, a plurality of historical jujube fruit images are acquired; Marking the jujube fruit targets in each of the historical jujube fruit images, and determining a reference detection result corresponding to each of the historical jujube fruit images; Constructing a jujube fruit dataset and dividing the jujube fruit dataset into a training set, a validation set, and a test set; the jujube fruit dataset includes: a plurality of historical jujube fruit images and a reference detection result corresponding to each historical jujube fruit image; The initial jujube fruit detection model is trained based on the training set and the validation set, and is validated by the test set to obtain the jujube fruit detection model.

4. The jujube fruit detection method based on improved YOLOv5s according to claim 3 is characterized in that, The step of labeling the jujube fruit target in each historical jujube fruit image and determining a reference detection result corresponding to each historical jujube fruit image includes: Screening multiple historical jujube fruit images to remove simple, blurred, targetless, and unfocused historical jujube fruit images, and obtaining multiple selected historical jujube fruit images; The LabelImg software was used to label the jujube fruit targets in each selected historical jujube fruit image, and the reference detection results corresponding to each selected historical jujube fruit image were obtained.

5. The jujube fruit detection method based on improved YOLOv5s according to claim 3 is characterized in that, The method of constructing a jujube fruit dataset and dividing the jujube fruit dataset into a training set, a validation set, and a test set comprises: Performing data enhancement processing on the plurality of historical jujube fruit images in the jujube fruit dataset, wherein the data enhancement processing comprises at least one of the following: rotation, translation, random brightness, scaling, cutout, and adding Gaussian noise; The data-enhanced jujube fruit dataset is divided into training set, validation set and test set in a ratio of 8:1:

1.

6. The jujube fruit detection method based on improved YOLOv5s according to any one of claims 1 to 5, characterized in that: Before inputting the current jujube fruit image into the jujube fruit detection model, the method further includes: The current jujube fruit image is preprocessed, where the preprocessing includes at least one of the following: screening, data enhancement, normalization, image denoising, and format conversion.

7. A jujube fruit detection device based on improved YOLOv5s, characterized in that: include: An acquisition module is used to acquire the current jujube fruit image captured by a shooting device in real time; a processing module, configured to input the current jujube fruit image into a jujube fruit detection model, and obtain a jujube fruit detection result output by the jujube fruit detection model, wherein the jujube fruit detection result includes a bounding box, a confidence score, a category probability, and a quantity of the jujube fruit; Among them, the jujube fruit detection model is constructed based on the improved YOLOv5s algorithm model and is trained based on a training set and a validation set; the training set and the validation set include multiple historical jujube fruit images and reference detection results corresponding to each historical jujube fruit image; the reference detection results include: a reference bounding box of the jujube fruit, a reference confidence, a reference category probability, and a reference quantity.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the jujube fruit detection method based on the improved YOLOv5s as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the jujube fruit detection method based on the improved YOLOv5s as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the jujube fruit detection method based on the improved YOLOv5s as described in any one of claims 1 to 6 is implemented.

Images