Tree palm fruit maturity detection method, device and equipment based on target segmentation and colorimetric analysis and medium

Abstract

The invention relates to an on-tree palm fruit maturity detection method and device based on target segmentation and colorimetric analysis, equipment and a medium, and the method comprises the steps: updating a backbone network of a first maturity detection model to a StarNet framework, updating all C2F modules in the backbone network and a neck network to C2F-Star modules, a detection head network is updated to be a lightweight shared convolution detection head, an original CIOU loss function is updated to be a DIOU loss function to construct a second maturity detection model, and a C2F-Star module is used for updating a Bottleneck structure in an original C2F module to be a Star Block module; and inputting a to-be-detected palm fruit image into the second maturity detection model, outputting a polygon mask corresponding to each target palm fruit, counting a red pixel proportion in a polygon mask area, and determining a maturity category corresponding to each target palm fruit according to the red pixel proportion. According to the method, the target can be stably identified even in the scenes of interlaced branches and overlapped leaves, and the problem of detection failure caused by shielding is solved.

Description

Method, apparatus, equipment, and media for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis. Technical Field

[0001] This application relates to the field of agricultural production, and in particular to a method, apparatus, electronic device and computer-readable storage medium for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis. Background Technology

[0002] Palm fruit is one of the world's highest-yielding sources of vegetable oil, and determining its maturity is crucial in determining the harvesting time and oil quality. Currently, palm fruit maturity assessment typically relies on visual inspection by experienced workers. This method suffers from low efficiency, high labor costs, subjective judgment criteria, and fatigue, severely hindering modern management and harvesting efficiency in palm plantations.

[0003] While some existing computer vision research has been applied to the maturity detection of fruits such as tomatoes and apples, there is limited research on the maturity determination of palm fruits, especially clusters of palm fruits growing on trees. Many existing general methods use rectangular bounding box-based object detection models (such as YOLOv5) to locate the fruits, and then combine this with features such as the fruit's color and shape for maturity analysis. However, these methods have significant limitations when dealing with clusters of palm fruits in natural scenes. First, the object detection boxes contain a large number of interfering pixels from surrounding branches and leaves, adjacent fruits, or the background sky. The inclusion of these irrelevant pixels severely contaminates subsequent color feature extraction, making it difficult to improve the accuracy of maturity determination. Second, palm fruit clusters are characterized by dense clustering of fruits and significant occlusion between fruits and branches and leaves, making object detection models prone to missed detections, false detections, and inaccurate bounding boxes in this complex scene.

[0004] In summary, existing technologies have limitations in addressing the characteristics of palm fruit clusters, which are characterized by dense fruit growth and significant shading between fruits and branches. This makes target detection models prone to issues such as missed detections, false detections, and inaccurate bounding boxes in complex scenarios. The applicant has made corresponding explorations to address these problems. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a method, device, electronic equipment and computer-readable storage medium for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis.

[0006] To meet the various objectives of this application, the following technical solution is adopted: A method for detecting the maturity of palm fruits on trees based on target segmentation and chromaticity analysis, adapted to one of the objectives of this application, includes: acquiring an image of palm fruits to be detected containing a bunch of target palm fruits; updating the backbone network of a first maturity detection model to a StarNet framework, updating all C2F modules in the backbone network and neck network to C2F-Star modules, updating the detection head network to a lightweight shared convolutional detection head, and updating the original CIOU loss function to a DIOU loss function to construct a second maturity detection model, wherein the C2F-Star module is the original C2F module with the Bottleneck structure updated to a Star Blocks module; inputting the image of the palm fruits to be detected into the second maturity detection model that has been trained to convergence, outputting a polygon mask corresponding to each target palm fruit in the bunch of target palm fruits, calculating the proportion of red pixels within the polygon mask area, and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels, thereby completing the detection of the maturity of palm fruits on trees.

[0007] Optionally, the backbone network of the second maturity detection model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a Star Blocks module, and an SPFF module connected in sequence. Each feature extraction unit is composed of a Conv module and a Star Blocks module connected in series. The Star Blocks module adopts a star topology with a central processing branch and multiple parallel radial branches. The neck network of the second maturity detection model includes a multi-scale feature adaptation module, a cross-scale feature fusion module, and a feature enhancement module. The cross-scale feature fusion module includes an Upsample module and a Concat concatenation module. The feature enhancement module includes multiple C2F-Star modules and an SPFF module.

[0008] Optionally, the image of the palm fruit to be detected is input into a second maturity detection model that has been trained to convergence, and the polygon mask corresponding to each target palm fruit in the target palm fruit cluster is output. This includes: in the backbone network of the second maturity detection model, the image of the palm fruit to be detected is sequentially extracted by the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit; the Conv module in each feature extraction unit first performs channel adjustment and noise filtering on the input feature map, and then retains global features through the central processing branch and captures multi-scale local details through the radial branch in the Star Blocks module, and fuses multi-branch features through star operations to output multi-dimensional features containing palm fruit color, outline, and texture; after further enhancement of the features by the independent Star Blocks module, they are input into the first SPFF module for multi-scale pooling and feature integration, and the backbone feature map containing multi-scale contextual information of the palm fruit is output; the backbone feature map is input into the neck network and the lightweight shared convolutional detection head to output the polygon mask corresponding to each target palm fruit in the target palm fruit cluster.

[0009] Optionally, the image of the palm fruit to be detected is input into a second maturity detection model that has been trained to convergence, and the polygon mask corresponding to each target palm fruit in the target palm fruit string is output. This includes: obtaining the backbone feature map containing multi-scale contextual information of the palm fruit output by the backbone network of the second maturity detection model; inputting the backbone feature map into the neck network, and performing channel number unification and background noise filtering on the backbone feature maps of different levels through the Conv module of the multi-scale feature adaptation module to output a dimension-matched adaptation feature map; and performing upsampling on different levels based on the Upsample module of the cross-scale feature fusion module. The adaptive feature map is resolution matched, and then the Concat stitching module performs channel stitching of cross-scale features to output a fused feature map. Based on the multiple C2F-Star modules of the feature enhancement module, the fused feature map is enhanced in two ways. Then, the SPFF module performs multi-scale pooling and spatial context information integration to output an enhanced feature map. The enhanced feature map is input into a lightweight shared convolutional detection head, which outputs a polygon mask corresponding to each target palm fruit in the target palm fruit string. The proportion of red pixels in the polygon mask area is counted, and the maturity category corresponding to each target palm fruit is determined based on the proportion of red pixels.

[0010] Optionally, the steps of inputting the enhanced feature map into a lightweight shared convolutional detection head, outputting a polygon mask corresponding to each target palm fruit in the target palm fruit cluster, calculating the proportion of red pixels within the polygon mask area, and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels include: inputting the enhanced feature map into a lightweight shared convolutional detection head, performing feature extraction and dimensionality compression on the enhanced feature map through the shared convolutional layer of the lightweight shared convolutional detection head, and outputting fusion features adapted to the target detection and segmentation tasks; based on the fusion features, outputting a polygon mask corresponding to each target palm fruit through the segmentation branch of the lightweight shared convolutional detection head, wherein the polygon mask is used to select the pixel region of the target palm fruit in the palm fruit image to be detected; for the polygon mask region corresponding to each target palm fruit, calculating the number of red pixels, green pixels, and blue pixels within the polygon mask region, calculating the ratio of the number of red pixels to the total number of pixels in the polygon mask region to determine the proportion of red pixels; and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels to complete the detection of the maturity of palm fruits on the tree.

[0011] Optionally, the step of training the second maturity detection model includes: obtaining a training dataset, wherein the training dataset includes multiple training samples and their corresponding sample labels, the training samples represent palm fruit image samples containing target palm fruit clusters, and the sample labels represent sample annotation information of the bounding box of each target palm fruit in the target palm fruit cluster in the palm fruit image samples.

[0012] The multiple training samples are input into the second maturity detection model. The loss value between the model's prediction result and the sample labeling information is calculated based on the DIOU loss function. The parameters of each module of the model are updated through the backpropagation algorithm. After each round of training, the validation dataset is input into the second maturity detection model in the current training state, and the model's detection precision, recall, and maturity classification accuracy are calculated. The above steps are repeated until the model's loss value tends to stabilize and the detection precision, recall, and maturity classification accuracy corresponding to the validation dataset no longer improve for multiple consecutive rounds. The model is then determined to have reached a convergence state, training is stopped, and the trained second maturity detection model is obtained.

[0013] Optionally, the basic network architecture of the first maturity detection model is the original YOLOv8n-Seg model, and the basic network architecture of the second maturity detection model is the improved YOLOv8n-Seg model; the maturity categories include mature superior fruit bunches, mature first-grade fruit bunches, mature second-grade fruit bunches, and immature unqualified fruit bunches.

[0014] A tree-grown palm fruit maturity detection device based on target segmentation and chromaticity analysis, provided for another purpose of this application, includes: an image acquisition module configured to acquire images of palm fruits to be detected containing clusters of target palm fruits; a model construction module configured to update the backbone network of a first maturity detection model to a StarNet framework, update all C2F modules in the backbone network and neck network to C2F-Star modules, update the detection head network to a lightweight shared convolutional detection head, and update the original CIOU loss function to a DIOU loss function to construct a second maturity detection model, wherein the C2F-Star module is an update of the Bottleneck structure in the original C2F module to a StarBlocks module; and a maturity detection module configured to input the image of the palm fruits to be detected into the second maturity detection model trained to convergence, output a polygon mask corresponding to each target palm fruit in the cluster, count the proportion of red pixels within the polygon mask area, and determine the maturity category corresponding to each target palm fruit based on the proportion of red pixels, thereby completing the detection of the maturity of the tree-grown palm fruits.

[0015] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the tree palm fruit maturity detection method based on target segmentation and colorimetric analysis described in this application.

[0016] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the method for detecting the maturity of palm fruit on a tree based on target segmentation and colorimetric analysis, which, when called by a computer, executes the steps included in the corresponding method.

[0017] Compared to existing technologies, this application addresses the challenges posed by the dense clustering and significant occlusion of palm fruit bunches with branches and leaves in complex scenarios. These characteristics make target detection models prone to missed detections, false detections, and inaccurate bounding boxes in such environments. This application offers the following advantages, including but not limited to: First, traditional target detection models (such as YOLOv5) often include interfering pixels like branches, leaves, and sky in their bounding boxes, contaminating color feature extraction. This application utilizes an improved YOLOv8n-Seg model to output a polygon mask, achieving pixel-level separation between the target palm fruit and the background, completely eliminating irrelevant interference and ensuring the purity of color feature extraction.

[0018] Secondly, traditional target detection models are prone to missed detections and false detections in scenarios with dense clusters of palm fruits and severe occlusion. This application utilizes a star topology with one central processing branch and four parallel radial branches in the C2F-Star module to accurately capture local details and global contours of occluded areas. Combined with the DIOU loss function to optimize bounding box regression, it can still stably identify targets even in scenarios with intertwined branches and overlapping leaves, thus solving the detection failure problem caused by occlusion.

[0019] Third, traditional target detection models are not adaptable to changes in outdoor lighting and different planting terrains. This application constructs a dedicated dataset that covers multiple lighting conditions, including direct sunlight, overcast daytime scattering, and low evening light, as well as different terrains such as mountains and plains. Combined with data augmentation techniques such as transposition, sharpening, and shadow simulation, the model can still output accurate results stably in complex environments, significantly improving environmental robustness.

[0020] Fourth, traditional models are too complex to be deployed on computing-constrained devices such as agricultural robots. This application replaces the original backbone network with the StarNet framework and replaces the traditional Bottleneck with the C2F-Star module, reducing the model's GFLOPs to 4.5G, parameters to 1.37M, and model size to 2.85MB, which are only 56.0%, 46.0%, and 48.0% of the original YOLOv8 model, respectively, while maintaining high detection accuracy. It can be directly deployed on edge devices to meet the real-time detection needs in the field.

[0021] Fifth, compared with the original YOLOv8n-Seg model, the improved YOLOv8n-Seg model in this application has reduced the number of parameters and the model size to 1.37M and 2.85MB, respectively, and the computational complexity is significantly reduced.

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 is a flowchart illustrating the method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis in an embodiment of this application; Figure 2 is a schematic diagram illustrating the effect of palm fruit segmentation in an embodiment of this application; Figure 3 is a schematic diagram illustrating the effect of visualizing the results of palm fruit maturity detection based on target segmentation and colorimetric analysis in an embodiment of this application; Figure 4 is an exemplary network architecture diagram of the improved YOLOv8n-Seg model in an embodiment of this application; Figure 5 is an exemplary network architecture diagram of the StarNet framework, C2F-Star module, and Star Blocks module in an embodiment of this application; Figure 6 is a principle block diagram of the device for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis in an embodiment of this application; and Figure 7 is a structural schematic diagram of the computer device in an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0024] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0025] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0026] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include radio frequency receivers, pagers, internet / intranet access, web browsers, notebooks, calendars, and / or GPS (Global Positioning System) receivers; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0027] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0028] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0029] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0030] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0031] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0032] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0033] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0034] Please refer to Figure 1. In one embodiment of the tree-borne palm fruit maturity detection method based on target segmentation and color analysis of this application, the method includes: step S10, acquiring an image of the palm fruit to be detected containing a cluster of target palm fruits; the palm fruit maturity detection system in the terminal device can acquire the image of the palm fruit to be detected containing a cluster of target palm fruits; specifically, in the palm plantation, the image of the palm fruit on the tree can be directly captured by an image input device (such as a camera or a vision module mounted on an agricultural robot) to ensure the authenticity of the scene; the palm fruit image can be a cluster of unpicked palm fruits on the tree with multiple maturity stages, multiple light conditions, and multiple background interferences. The multiple maturity stages include the immature stage, the color-changing stage, and the mature stage. The immature stage indicates that the palm fruit pericarp is yellowish-green, the color-changing stage indicates that the palm fruit pericarp is violet to purplish-red, and the mature stage indicates that the palm fruit pericarp is orange-red. The multiple light conditions include strong direct sunlight (including local reflection), cloudy diffused light, and weak light (low illuminance) in the evening, etc.; the multiple background interferences include the overlapping and shading of the main trunk and branches, the overlapping and shading of multiple layers of leaves, and different planting terrain backgrounds such as mountains or plains.

[0035] The palm fruit maturity detection system connects to the image input device via a data interface, receiving raw palm fruit images of 640×640 pixels. It specifically covers the natural growth scene of unpicked fruit clusters on the tree, encompassing different maturity levels (yellow-green, violet, crimson, orange-red), lighting conditions (direct sunlight, overcast diffused light, weak evening light), and background scenes (dense branch obstruction, multi-layered leaf interference, complex terrain background). (See Figure 2). The computing platform incorporates an improved YOLOv8n-Seg instance segmentation model and color analysis algorithm. Through network structure simplification and module optimization, it reduces computational complexity while ensuring detection accuracy. It can quickly complete target segmentation, red pixel proportion statistics, and maturity grading of the input image. Finally, it outputs a visualized result on the display screen containing bounding boxes, category labels, maturity levels, and red proportion data (see Figure 3), accurately matching the identification and discrimination needs of palm fruits on the tree.

[0036] In some embodiments, all original palm fruit images are uniformly scaled and stretched to 640×640 pixels, and the LabelImg tool is used to accurately label the palm fruit targets on the tree, clearly selecting the entire area of ​​the fruit cluster, and generating a PASCAL VOCXML format label file to provide accurate target localization information for model training, ensuring that the model focuses on feature learning of the fruit cluster on the tree.

[0037] To improve the model's adaptability and generalization ability to complex tree environments, the labeled images were processed using the Coovally tool through three methods: transposition, sharpening, and shadow simulation. Shadow simulation specifically recreates local shadow scenes formed by leaf occlusion. Transposition and sharpening enhance the model's adaptability to different shooting angles and fruit cluster densities, expanding the dataset to 2200 images. Subsequently, the XML format tags were converted to TXT format and divided into a training set (1540 images), a validation set (440 images), and a test set (220 images) in a 7:2:1 ratio to ensure the model's robustness in tree scenes.

[0038] In some embodiments, the OpenCV and PIL libraries are integrated to overlay Chinese category labels ("palm fruit on a tree"), maturity level and red proportion data on the image. Through an adaptive parameter adjustment mechanism, the label display is made consistent under different resolution images, which makes it convenient for on-site operators to quickly view the discrimination results.

[0039] Step S20: Update the backbone network of the first maturity detection model to the StarNet framework, update all C2F modules in the backbone network and neck network to C2F-Star modules, update the detection head network to a lightweight shared convolutional detection head, and update the original CIOU loss function to the DIOU loss function to construct the second maturity detection model. The C2F-Star module is formed by updating the Bottleneck structure in the original C2F module to a Star Blocks module. After acquiring the palm fruit image containing the target palm fruit cluster, update the backbone network of the first maturity detection model to the StarNet framework, update all C2F modules in the backbone network and neck network to C2F-Star modules, update the detection head network to a lightweight shared convolutional detection head, and update the original CIOU loss function to the DIOU loss function to construct the second maturity detection model. The C2F-Star module is formed by updating the Bottleneck structure in the original C2F module to a Star Blocks module. The Blocks module; wherein, the basic network architecture of the first maturity detection model is the original YOLOv8n-Seg model, and the basic network architecture of the second maturity detection model is the improved YOLOv8n-Seg model.

[0040] In some embodiments, the backbone network of the second maturity detection model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a Star Blocks module, and an SPFF module connected in sequence. Each feature extraction unit is composed of a Conv module and a Star Blocks module connected in series. The Star Blocks module adopts a star topology with a central processing branch and multiple parallel radial branches. The neck network of the second maturity detection model includes a multi-scale feature adaptation module, a cross-scale feature fusion module, and a feature enhancement module. The cross-scale feature fusion module includes an Upsample module and a Concat concatenation module. The feature enhancement module includes multiple C2F-Star modules and an SPFF module.

[0041] Specifically, please refer to Figures 4 and 5 to compare the three instance segmentation models: Mask R-CNN, YOLACT, and YOLOv8n-Seg. Mask R-CNN has a slower detection speed, YOLACT lacks sufficient object detection accuracy, while YOLOv8n-Seg combines the efficiency of single-stage detection with the fine-grained advantages of instance segmentation, making it more suitable for scenarios with densely distributed palm fruits and significant scale differences. Therefore, the original YOLOv8n-Seg model was selected as the first maturity detection model. To adapt to the needs of edge computing devices and rapid on-site inference, a targeted lightweight design was added to the original YOLOv8n-Seg model, based on the StarNet lightweight design concept, to construct the second maturity detection model. The specific process of constructing the second maturity detection model (the improved YOLOv8n-Seg model) includes: First, replacing the original backbone network with the StarNet framework. Its core advantage is... By leveraging its weight recursive reuse mechanism, the feature representation space is expanded exponentially without requiring a large number of additional parameters, thus reducing computational complexity while maintaining feature extraction capabilities. Simultaneously, all C2F modules are replaced with C2F-Star modules, and a lightweight shared convolutional detection head (LSCD detection head) is used to further reduce computational complexity. The improved YOLOv8n-Seg model achieves GFLOPs of 4.5G (only 56.0% of the original YOLOv8n-Seg model), 1.37M parameters (only 46.0% of the original YOLOv8n-Seg model), and a model size of 2.85MB (only 48.0% of the original YOLOv8n-Seg model), significantly adapting to the computing power limitations of edge devices.

[0042] Secondly, based on the star topology design concept of C2F-Star modules, all C2F modules in the original YOLOv8n-Seg model are updated to C2F-Star modules. The C2F-Star module replaces the Bottleneck structure in the original C2F module with Star Blocks modules, constructing a structure with one central processing branch and four parallel radial branches. Combined with the Concat cross-scale fusion strategy, multi-branch features are spliced ​​and integrated, weakening background interference from branches and leaves, and enhancing the discriminative power of target features. This accurately captures effective features of palm fruits on trees (including occlusion and multi-scale scenes), weakening background interference from branches and leaves, reducing computational overhead while improving feature representation capabilities, and solving the problem of insufficient feature extraction from tree targets after lightweighting.

[0043] Third, the detection head network is updated to a lightweight shared convolutional detection head (LSCD detection head). The lightweight shared convolutional detection head (LSCD detection head) adopts a design of shared convolutional layers and task-specific branches. The shared convolutional layers uniformly extract the fusion features adapted to the detection and segmentation tasks, and then output the bounding boxes and mask information through dedicated branches, avoiding repeated calculations and further reducing the model inference overhead.

[0044] Fourth, the original CIOU loss function is updated to the DIOU loss function. By optimizing the penalty mechanism of bounding box regression, the problem of decreased positioning accuracy that may occur after lightweighting is compensated, especially improving the segmentation accuracy when the palm fruits on the tree are partially occluded, and ensuring that the target box accurately fits the outline of the fruit bunch.

[0045] In some embodiments, the training parameters epochs=300, batch=8, imgsz=640 are set, the SGD optimizer is used and the initial learning rate is set to 0.01, momentum=0.937, weight_decay=0.0005 to enhance the model's adaptability to different lighting conditions, help the lightweight model converge quickly, and ensure efficient learning of the features of palm fruits on trees.

[0046] In some embodiments, the configured dataset and the optimized YOLOv8n-Seg model with lightweight parameters are input and trained on a GPU platform. During training, Mosaic image stitching technology is used to simulate the dense distribution of palm fruits on trees, increasing sample diversity. After the model converges, a training result file in best.pt format is obtained. After lightweighting, the parameter size of the model is reduced from 3.01M in the original YOLOv8n-Seg to 1.37M, and the model file size is compressed from 5.99MB to 2.85MB. At the same time, the four core indicators of accuracy, recall, mAP@50, and mAP@50-95 remain at 91.8%, 88.5%, 93.9%, and 76.0%, respectively. This achieves a significant reduction in model complexity while maintaining accuracy, fully meeting the computing power and accuracy requirements for on-site detection of palm fruits on trees.

[0047] In some embodiments, the lightweight pt file obtained from training is used to infer the input image. Through the efficient feature processing capability of the optimized model, polygonal masks and bounding boxes of palm fruit clusters on the tree are accurately generated, effectively segmenting the target area and eliminating background interference such as branches and leaves. The recognition rate reaches 96.7% on the test set, and the inference speed reaches 253 FPS, meeting the needs of real-time detection on site.

[0048] In some embodiments, based on the segmented mask region, a dual red determination criterion (R>100 and G<50 and B<50, or R>max(G,B)×2) is used to accurately count the ratio of the number of red pixels in the region to the total number of pixels. This ratio directly corresponds to the maturity characteristics of the palm fruit peel on the tree. Stable category color labels are generated through the MD5 hash algorithm, which supports visualization and provides a quantitative basis for maturity grading.

[0049] In some embodiments, leveraging the fast inference capabilities of the improved YOLOv8n-Seg model, the system processes 151 test images in a total of 15 minutes, with an average processing time of 6 seconds per image. This represents a 33% improvement in efficiency compared to human eye resolution (9 seconds per image), significantly enhancing the efficiency of palm fruit maturity grading on trees. It is suitable for large-scale plantation site batch testing and real-time deployment of edge devices.

[0050] In some embodiments, the step of training the second maturity detection model includes: step S201, obtaining a training dataset, wherein the training dataset includes multiple training samples and their corresponding sample labels, the training samples represent palm fruit image samples containing target palm fruit clusters, and the sample labels represent sample annotation information of the bounding box of each target palm fruit in the target palm fruit cluster in the palm fruit image samples.

[0051] Step S202: Input the multiple training samples into the second maturity detection model, calculate the loss value between the model prediction result and the sample labeling information based on the DIOU loss function, and update the parameters of each module of the model through the backpropagation algorithm; Step S203: After each round of training, input the validation dataset into the second maturity detection model in the current training state, and calculate the detection precision, recall rate, and maturity classification accuracy of the model; Step S204: Repeat the above steps until the model's loss value tends to stabilize and the detection precision, recall rate, and maturity classification accuracy corresponding to the validation dataset no longer improve for multiple consecutive rounds, determine that the model training has reached the convergence state, stop training, and obtain the trained second maturity detection model.

[0052] Step S30: Input the image of the palm fruit to be detected into the second maturity detection model that has been trained to convergence, output the polygon mask corresponding to each target palm fruit in the target palm fruit string, count the proportion of red pixels in the polygon mask area, and determine the maturity category corresponding to each target palm fruit according to the proportion of red pixels, so as to complete the detection of the maturity of palm fruits on the tree.

[0053] The backbone network of the first maturity detection model is updated to the StarNet framework, all C2F modules in the backbone network and neck network are updated to C2F-Star modules, the detection head network is updated to a lightweight shared convolutional detection head, and the original CIOU loss function is updated to the DIOU loss function to construct a second maturity detection model. Once the second maturity detection model has been trained to convergence, it can be used in production. The image of the palm fruit to be detected is input into the second maturity detection model, which has been trained to convergence. The model outputs a polygon mask corresponding to each target palm fruit in the target palm fruit cluster. The proportion of red pixels within the polygon mask area is counted, and the maturity category corresponding to each target palm fruit is determined based on the proportion of red pixels. The maturity categories include mature superior fruit clusters, mature first-grade fruit clusters, mature second-grade fruit clusters, and immature unqualified fruit clusters.

[0054] In some embodiments, grading is strictly based on local standards and the percentage of red pixels. A red pixel percentage ≥90% corresponds to mature premium fruit bunches with a complete bunch rate of over 90%; a red pixel percentage of 80% to 90% corresponds to mature first-grade fruit bunches with a complete bunch rate of 80% to 90%; a red pixel percentage of 70% to 80% corresponds to mature second-grade fruit bunches with a complete bunch rate of 70% to 80%; and a red pixel percentage <70% is considered immature and unqualified fruit bunches. The overall grading accuracy reaches 83.1%, with the accuracy rates for distinguishing mature premium fruit, mature first-grade fruit bunches, and mature second-grade fruit bunches being 83.3%, 68.8%, and 88.9%, respectively. This is highly consistent with the results of on-site human visual assessment. The lightweight optimization has minimal impact on grading accuracy and is fully adapted to the needs of judging the maturity of palm fruits on the tree.

[0055] In some embodiments, the communication and data flow of the palm fruit maturity detection system are achieved through a software interface. The computing platform integrates the segmentation results, color proportion data, and grading results, and displays them in real time on a screen. It also supports the export of lightweight model weight files, allowing direct deployment on computing-constrained platforms such as agricultural robots and embedded edge devices, providing efficient data support for automated harvesting equipment and intelligent grading systems for palm fruits. The system can also be further optimized to achieve functions such as automatic image import and adaptation to multiple palm fruit varieties, further enhancing its convenience and applicability in practical applications.

[0056] In some embodiments, the image of the palm fruit to be detected is input into a second maturity detection model that has been trained to convergence, and the polygon mask corresponding to each target palm fruit in the target palm fruit string is output, including: Step S31, in the backbone network of the second maturity detection model, the image of the palm fruit to be detected is sequentially extracted by the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit; Step S32, the Conv module in each feature extraction unit first performs channel adjustment and noise filtering on the input feature map, and then retains global features through the central processing branch and captures multi-scale local details through the radial branch in the Star Blocks module, and fuses multi-branch features through star operation to output multi-dimensional features including palm fruit color, outline, and texture; Step S33, after independent Star... After further enhancing the features, the Blocks module inputs them into the first SPFF module for multi-scale pooling and feature integration, and outputs a backbone feature map containing multi-scale contextual information of the palm fruit; Step S34: Input the backbone feature map into the neck network and the lightweight shared convolutional detection head, and output the polygon mask corresponding to each target palm fruit in the target palm fruit string.

[0057] As can be seen from steps S31 to S34 above, the image is processed sequentially by the first to fourth feature extraction units, realizing a gradual upgrade from basic features to higher-order features, accurately capturing the core features of palm fruits on the tree at different scales and under occlusion states (such as color gradients, contour details, and texture distribution), effectively solving the problem of insufficient feature extraction of densely clustered palm fruits with occlusion by branches and leaves in traditional models.

[0058] In each feature extraction unit, the central processing branch retains global features such as the overall outline of the fruit bunch, while the four radial branches capture local details such as the edge of a single fruit and the texture of the occluded area. Then, through star-shaped deep fusion, the features are ensured to include both the overall distribution information of the fruit bunch and the maturity-related features of a single fruit (such as the red area of ​​the exocarp), thereby improving feature discriminability.

[0059] The independent Star Blocks module further refines the progressively extracted features, weakening background interference such as branches and leaves; the SPFF module integrates the contextual relationships of features at different levels through multi-scale pooling, and the output backbone feature map has both multi-scale adaptability and noise robustness, providing high-purity feature support for subsequent grading.

[0060] In a further embodiment, the image of the palm fruit to be detected is input into a second maturity detection model that has been trained to a convergent state, and the polygon mask corresponding to each target palm fruit in the target palm fruit string is output, including: step S301, obtaining the backbone feature map containing multi-scale contextual information of palm fruit output by the backbone network of the second maturity detection model; step S302, inputting the backbone feature map into the neck network, and performing channel number unification and background noise filtering on the backbone feature maps of different levels through the Conv module of the multi-scale feature adaptation module, and outputting a dimension-matched adaptation feature map; step S303, based on the Upsample module of the cross-scale feature fusion module... The block performs resolution matching on the adaptation feature maps at different levels, and then performs channel stitching of cross-scale features through the Concat stitching module to output a fused feature map; Step S304: Based on the multiple C2F-Star modules of the feature enhancement module, the fused feature map is subjected to secondary feature enhancement, and then multi-scale pooling and spatial context information integration are performed through the SPFF module to output an enhanced feature map; Step S305: The enhanced feature map is input into the lightweight shared convolutional detection head, and the polygon mask corresponding to each target palm fruit in the target palm fruit string is output. The proportion of red pixels in the polygon mask area is counted, and the maturity category corresponding to each target palm fruit is determined according to the proportion of red pixels.

[0061] As can be seen from steps S301 to S305 above, the Conv module of the multi-scale feature adaptation module unifies the number of channels of the backbone feature maps at different levels, solving the problem of dimension mismatch during cross-scale fusion; at the same time, it filters background noise such as branches and leaves, providing clean and dimensionally consistent basic features for subsequent feature fusion, avoiding interference with pixel contamination of subsequent processing.

[0062] Upsampling is used to align the resolution of feature maps at different levels, followed by channel-level fusion via the Concat module. This ensures that the features include both high-level information such as the global outline and overall distribution of the fruit cluster, and low-level details such as individual fruit edges and color gradations, perfectly adapting to the dense clustering and significant scale differences of palm fruits. The C2F-Star module uses a star topology with one central processing branch and four parallel radial branches to further mine maturity-related information (such as the red area features of the outer peel) in the fused features, weakening interference from complex environments. The SPFF module integrates spatial context information through multi-scale pooling, giving the output enhanced feature map stronger target recognition, laying a solid foundation for subsequent mask segmentation and color analysis. A lightweight shared convolutional detection head accurately generates polygonal masks of palm fruit clusters, completely separating the target area from the background and solving the pain point of traditional bounding box detection containing interfering pixels.

[0063] In a further embodiment, the steps of inputting the enhanced feature map into a lightweight shared convolutional detection head, outputting a polygon mask corresponding to each target palm fruit in the target palm fruit cluster, calculating the proportion of red pixels within the polygon mask area, and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels include: Step S3001, inputting the enhanced feature map into a lightweight shared convolutional detection head, performing feature extraction and dimensionality compression on the enhanced feature map through the shared convolutional layer of the lightweight shared convolutional detection head, and outputting a fusion feature adapted to the target detection and segmentation task; Step S3002, based on the fusion feature, ... The segmentation branch of the lightweight shared convolutional detection head outputs a polygon mask corresponding to each target palm fruit, wherein the polygon mask is used to select the pixel region of the target palm fruit in the image to be detected; Step S3003: For the polygon mask region corresponding to each target palm fruit, count the number of red pixels, green pixels and blue pixels in the polygon mask region, and calculate the ratio of the number of red pixels to the total number of pixels in the polygon mask region to determine the proportion of red pixels; Step S3004: Determine the maturity category corresponding to each target palm fruit according to the proportion of red pixels to complete the detection of the maturity of palm fruits on the tree.

[0064] As described in steps S3001 to S3004, the shared convolutional layer of the lightweight shared convolutional detection head performs dual processing of enhanced feature map extraction and dimensionality compression. While retaining key features of palm fruit maturity (color and contour), it removes redundant information and outputs fused features adapted for both detection and segmentation tasks. This ensures feature purity and avoids redundant calculations, providing high-quality feature support for subsequent accurate segmentation and color analysis. The fused features simultaneously meet the localization requirements of object detection and the pixel-level accuracy requirements of instance segmentation, solving the problem of feature disconnect between detection and segmentation in traditional models. This ensures that the segmentation branch can quickly output accurate masks based on unified features, improving the coherence and efficiency of the overall judgment process.

[0065] Furthermore, by using polygonal masks output from segmented branches, precise pixel-level bounding boxes are achieved for individual palm fruits, completely separating the target palm fruit from the background environment such as branches, leaves, and sky. This completely avoids the color feature extraction distortion caused by interference pixels in traditional rectangular bounding box detection, ensuring the purity of subsequent red pixel statistics. The polygonal mask can accurately fit the densely clustered and mutually occluded growth pattern of palm fruits. Even when facing partially occluded fruits, it can still accurately delineate the outline of the visible area, ensuring the integrity of the segmentation results and providing a reliable regional basis for calculating the proportion of red pixels. This enables the model to achieve a segmentation and recognition rate of 96.7% in complex scenes.

[0066] Furthermore, by statistically analyzing the number of RGB three-channel pixels within the masked area and calculating the proportion of red pixels, the qualitative characteristic of palm fruit maturity is transformed into a quantitative indicator. This completely eliminates the subjectivity, reliance on experience, and fatigue-induced biases of manual visual inspection, ensuring consistency and repeatability in maturity assessment results. The calculation of the red pixel proportion is based solely on pixel statistics and ratio calculations within the masked area, requiring no complex algorithms. Combined with the efficient inference capabilities of the lightweight model, the average processing time for a single image is only 6 seconds, meeting the real-time batch inspection needs of large-scale plantations.

[0067] As can be seen from the above embodiments, compared with the prior art, this application addresses the problem that existing palm fruit clusters have the characteristics of dense fruit growth and severe occlusion between fruits and branches and leaves, making it easy for target detection models to suffer from missed detections, false detections, and inaccurate bounding boxes in such complex scenes. This application includes, but is not limited to, the following beneficial effects: First, traditional target detection models (such as YOLOv5) are prone to including interfering pixels such as branches, leaves, and sky in the detection box, which contaminates color feature extraction. This application achieves pixel-level separation between the target palm fruit and the background by outputting a polygon mask through an improved YOLOv8n-Seg model, completely eliminating irrelevant interference and ensuring the purity of color feature extraction.

[0068] Secondly, traditional target detection models are prone to missed detections and false detections in scenarios with dense clusters of palm fruits and severe occlusion. This application utilizes a star topology with one central processing branch and four parallel radial branches in the C2F-Star module to accurately capture local details and global contours of occluded areas. Combined with the DIOU loss function to optimize bounding box regression, it can still stably identify targets even in scenarios with intertwined branches and overlapping leaves, thus solving the detection failure problem caused by occlusion.

[0069] Third, traditional target detection models are not adaptable to changes in outdoor lighting and different planting terrains. This application constructs a dedicated dataset that covers multiple lighting conditions, including direct sunlight, overcast daytime scattering, and low evening light, as well as different terrains such as mountains and plains. Combined with data augmentation techniques such as transposition, sharpening, and shadow simulation, the model can still output accurate results stably in complex environments, significantly improving environmental robustness.

[0070] Fourth, traditional models are too complex to be deployed on computing-constrained devices such as agricultural robots. This application replaces the original backbone network with the StarNet framework and replaces the traditional Bottleneck with the C2F-Star module, reducing the model's GFLOPs to 4.5G, parameters to 1.37M, and model size to 2.85MB, which are only 56.0%, 46.0%, and 48.0% of the original YOLOv8 model, respectively, while maintaining high detection accuracy. It can be directly deployed on edge devices to meet the real-time detection needs in the field.

[0071] Fifth, compared with the original YOLOv8n-Seg model, the improved YOLOv8n-Seg model in this application has reduced the number of parameters and the model size to 1.37M and 2.85MB, respectively, and the computational complexity is significantly reduced.

[0072] Please refer to Figure 6, which describes a tree palm fruit ripening detection device based on target segmentation and colorimetric analysis, provided for one of the purposes of this application, comprising: The image acquisition module 1100 is configured to acquire images of palm fruits to be detected containing target palm fruit clusters; the model construction module 1200 is configured to update the backbone network of the first maturity detection model to the StarNet framework, update all C2F modules in the backbone network and neck network to C2F-Star modules, update the detection head network to a lightweight shared convolutional detection head, and update the original CIOU loss function to the DIOU loss function to construct a second maturity detection model, wherein the C2F-Star module is to update the Bottleneck structure in the original C2F module to the Star Blocks module; the maturity detection module 1300 is configured to input the image of the palm fruit to be detected into the second maturity detection model that has been trained to convergence, output the polygon mask corresponding to each target palm fruit in the target palm fruit cluster, count the proportion of red pixels in the polygon mask area, and determine the maturity category corresponding to each target palm fruit based on the proportion of red pixels to complete the detection of the maturity of palm fruits on the tree.

[0073] Based on any embodiment of this application, referring to Figure 7, another embodiment of this application also provides an electronic device, which can be implemented by a computer device. As shown in Figure 7, which is a schematic diagram of the internal structure of the computer device, the computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, the processor can implement a method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis. The processor of the computer device provides computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that the structure shown in Figure 7 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0074] In this embodiment, the processor executes the specific functions of each module in Figure 6, and the memory stores the program code and various types of data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the tree palm fruit maturity detection device based on target segmentation and color analysis of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0075] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the tree palm fruit ripening detection method based on target segmentation and color analysis as described in any embodiment of this application.

[0076] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the tree palm fruit maturity detection method based on target segmentation and color analysis as described in any embodiment of this application.

[0077] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0078] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis, characterized in that, include: Acquire images of palm fruits to be detected that contain clusters of target palm fruits; The backbone network of the first maturity detection model is updated to the StarNet framework, all C2F modules in the backbone network and neck network are updated to C2F-Star modules, the detection head network is updated to a lightweight shared convolutional detection head, and the original CIOU loss function is updated to the DIOU loss function to construct the second maturity detection model. The C2F-Star module is the original C2F module with the Bottleneck structure updated to the Star Blocks module. The image of the palm fruit to be detected is input into the second maturity detection model that has been trained to convergence. The model outputs a polygon mask corresponding to each target palm fruit in the target palm fruit string. The proportion of red pixels in the polygon mask area is counted. The maturity category corresponding to each target palm fruit is determined according to the proportion of red pixels to complete the detection of the maturity of palm fruits on the tree.

2. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to claim 1, characterized in that, The backbone network of the second maturity detection model includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit, a Star Blocks module, and an SPFF module connected in sequence. Each feature extraction unit is composed of a Conv module and a Star Blocks module connected in series. The Star Blocks module adopts a star topology with a central processing branch and multiple parallel radial branches. The neck network of the second maturity detection model includes a multi-scale feature adaptation module, a cross-scale feature fusion module, and a feature enhancement module. The cross-scale feature fusion module includes an Upsample module and a Concat concatenation module. The feature enhancement module includes multiple C2F-Star modules and an SPFF module.

3. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to claim 2, characterized in that, The palm fruit image to be detected is input into a second maturity detection model that has been trained to convergence, and the polygon mask corresponding to each target palm fruit in the target palm fruit cluster is output. This includes: in the backbone network of the second maturity detection model, the palm fruit image to be detected is sequentially extracted by the first feature extraction unit, the second feature extraction unit, the third feature extraction unit, and the fourth feature extraction unit; the Conv module in each feature extraction unit first performs channel adjustment and noise filtering on the input feature map, and then retains global features through the central processing branch and captures multi-scale local details through the radial branch in the Star Blocks module, and fuses multi-branch features through star operation to output multi-dimensional features containing palm fruit color, outline, and texture; after further enhancement of the features by the independent Star Blocks module, they are input into the first SPFF module for multi-scale pooling and feature integration, and the backbone feature map containing multi-scale contextual information of the palm fruit is output; the backbone feature map is input into the neck network and the lightweight shared convolutional detection head to output the polygon mask corresponding to each target palm fruit in the target palm fruit cluster.

4. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to claim 2, characterized in that, The image of the palm fruit to be detected is input into a second maturity detection model that has been trained to convergence, and the polygon mask corresponding to each target palm fruit in the target palm fruit string is output. This includes: obtaining the backbone feature map containing multi-scale contextual information of the palm fruit output by the backbone network of the second maturity detection model; inputting the backbone feature map into the neck network, and using the Conv module of the multi-scale feature adaptation module to unify the number of channels and filter background noise of the backbone feature maps at different levels, and outputting a dimension-matched adaptation feature map; and using the Upsample module of the cross-scale feature fusion module to adapt the feature maps at different levels. The feature map is matched for resolution, and then the Concat stitching module is used to stitch the cross-scale features into channels to output a fused feature map. Based on the feature enhancement module, multiple C2F-Star modules are used to perform secondary feature enhancement on the fused feature map, and then the SPFF module is used to perform multi-scale pooling and spatial context information integration to output an enhanced feature map. The enhanced feature map is input into a lightweight shared convolutional detection head, which outputs a polygon mask corresponding to each target palm fruit in the target palm fruit string. The proportion of red pixels in the polygon mask area is counted, and the maturity category corresponding to each target palm fruit is determined based on the proportion of red pixels.

5. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to claim 4, characterized in that, The steps of inputting the enhanced feature map into a lightweight shared convolutional detection head, outputting a polygon mask corresponding to each target palm fruit in the target palm fruit cluster, calculating the proportion of red pixels within the polygon mask region, and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels include: inputting the enhanced feature map into a lightweight shared convolutional detection head, performing feature extraction and dimensionality compression on the enhanced feature map through the shared convolutional layer of the lightweight shared convolutional detection head, and outputting fused features adapted to target detection and segmentation tasks; based on the fused features, outputting a polygon mask corresponding to each target palm fruit through the segmentation branch of the lightweight shared convolutional detection head, wherein the polygon mask is used to select the pixel region of the target palm fruit in the palm fruit image to be detected; for the polygon mask region corresponding to each target palm fruit, calculating the number of red pixels, green pixels, and blue pixels within the polygon mask region, calculating the ratio of the number of red pixels to the total number of pixels in the polygon mask region to determine the proportion of red pixels; and determining the maturity category corresponding to each target palm fruit based on the proportion of red pixels to complete the detection of palm fruit maturity on the tree.

6. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to claim 1, characterized in that, The steps for training the second maturity detection model include: acquiring a training dataset, wherein the training dataset includes multiple training samples and their corresponding sample labels, the training samples representing palm fruit image samples containing target palm fruit clusters, and the sample labels representing the sample annotation information of the bounding box of each target palm fruit in the target palm fruit cluster in the palm fruit image samples; inputting the multiple training samples into the second maturity detection model, calculating the loss value between the model prediction result and the sample annotation information based on the DIOU loss function, and updating the parameters of each module of the model through the backpropagation algorithm; after each round of training, inputting the validation dataset into the second maturity detection model in the current training state, calculating the model's detection precision, recall rate, and maturity classification accuracy; repeating the above steps until the model's loss value tends to stabilize and the detection precision, recall rate, and maturity classification accuracy corresponding to the validation dataset no longer improve for multiple consecutive rounds, determining that the model training has reached a convergence state, stopping training, and obtaining the trained second maturity detection model.

7. The method for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis according to any one of claims 1 to 6, characterized in that, The basic network architecture of the first maturity detection model is the original YOLOv8n-Seg model, and the basic network architecture of the second maturity detection model is the improved YOLOv8n-Seg model; the maturity categories include mature superior fruit bunches, mature first-grade fruit bunches, mature second-grade fruit bunches, and immature unqualified fruit bunches.

8. A device for detecting the maturity of palm fruits on trees based on target segmentation and colorimetric analysis, characterized in that, Includes: an image acquisition module, configured to acquire images of palm fruits to be detected that contain the target palm fruit cluster; The model building module is configured to update the backbone network of the first maturity detection model to the StarNet framework, update all C2F modules in the backbone network and neck network to C2F-Star modules, update the detection head network to a lightweight shared convolutional detection head, and update the original CIOU loss function to the DIOU loss function to construct the second maturity detection model. The C2F-Star module is configured to update the Bottleneck structure in the original C2F module to a Star Blocks module. The maturity detection module is configured to input the palm fruit image to be detected into the second maturity detection model that has been trained to convergence, output a polygon mask corresponding to each target palm fruit in the target palm fruit string, count the proportion of red pixels within the polygon mask area, and determine the maturity category corresponding to each target palm fruit based on the proportion of red pixels to complete the detection of palm fruit maturity on the tree.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images