Palm fruit string sorting control method, device, equipment and medium

Abstract

This application relates to a method, apparatus, equipment, and medium for controlling the sorting of palm fruit bunches. The method includes: inputting fused features into a neck multi-scale feature layer containing a CARAFE content-aware upsampling module; generating an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module; upsampling the fused features to retain the fine-grained maturity features of the palm fruit bunches to be sorted; inputting the upsampled fused features into a multi-class classification detection head to output the class label and corresponding confidence level of the palm fruit bunches to be sorted; generating a corresponding sorting control signal based on the class label corresponding to the highest confidence level and sending it to an automated sorting device to control the automated sorting device to sort the palm fruit bunches to be sorted to the target station. This application significantly improves the recognition accuracy of the nearest neighbor maturity and defect categories of palm fruit bunches, solving the pain points of low accuracy in manual grading and weak ability of traditional algorithms to distinguish subtle features.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a method for controlling the sorting of palm fruit bunches, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology

[0002] The quality grading of palm fruit bunches is a core link in improving the quality and efficiency of the oil palm industry. The accuracy of grading directly determines the oil extraction efficiency, product grade and economic benefits.

[0003] Currently, palm fruit bunch grading mainly relies on manual sorting and traditional deep learning algorithms. Existing technologies are insufficient to meet the industry's demands for automation and precision, and they mainly suffer from the following technical shortcomings:

[0004] Firstly, the challenge of distinguishing subtle features across multiple categories is significant: palm fruit bunches need to be graded into more than 8 categories (3 categories of maturity + 5 categories of morphology / defects). The visual features of neighboring grades (such as unripe fruit bunches and ripe fruit bunches, diseased fruit bunches and rotten fruit bunches) overlap significantly—unripe fruit bunches are mainly light green, while ripe fruit bunches are light orange-red, with only slight color differences between the two; the brown spots on diseased fruit bunches are similar in morphology to the initial blackening areas on rotten fruit bunches. The accuracy rate of manual grading is only about 75%, and traditional algorithms are prone to misclassification.

[0005] Secondly, the existing YOLO series algorithms lack adaptability: the current mainstream YOLO algorithm uses RGB single-channel image input by default, which can only capture the morphological and texture features of the fruit bunch. It has a weak ability to represent the subtle color differences (gradual process of green → light orange-red → orange-red → dark red) related to the maturity of palm fruit bunches, and cannot effectively distinguish the maturity levels of neighbors. In addition, the original backbone network and neck module of YOLOv11 were not designed for the characteristics of "dense fruit and complex background" of palm fruit bunches, resulting in poor feature extraction targeting and serious interference from redundant features.

[0006] Third, the effects of single image space and simple fusion schemes are limited: In existing technologies, the YOLOv11 algorithm using RGB single-space input is insufficient in capturing subtle color difference features of palm fruit bunches, and the maturity classification accuracy is only 88%; the scheme using LAB single-space input lacks texture feature support, and the recognition accuracy of abnormal shape and defect categories is low; a few fusion schemes that simply stitch together RGB and LAB images do not design a targeted feature fusion mechanism, but simply superimpose channel information, which cannot achieve feature complementarity, and still have problems such as misclassification of neighboring categories and insufficient extraction of features from dense fruit bunches.

[0007] Fourth, the imbalance between real-time performance and industrial deployment adaptability: existing improved YOLO algorithms mostly increase accuracy by increasing the number of network layers, resulting in a surge in model parameters and a decrease in inference speed (processing time for a single fruit string > 60ms), which cannot meet the requirements of high-speed conveyor belt sorting (requiring a processing time for a single fruit string ≤ 50ms); while lightweight models suffer from accuracy loss, making it difficult to balance "accurate classification" and "real-time inference".

[0008] Fifth, it does not adapt to the specific grading requirements of palm fruit bunches: Existing algorithms are mostly designed for fruits such as mangoes and citrus fruits, and do not take into account the national oil palm grading standard (SNI 01-2301-2008), enterprise harvesting standards and the grading experience of senior workers. They do not cover the specific categories of palm fruit bunches such as hedgehog fruit bunches and long-stemmed fruit bunches, resulting in poor industry adaptability.

[0009] In summary, given the existing technologies that suffer from problems such as the large variety of palm fruit clusters, small differences in features, insufficient adaptability of YOLO series algorithms, inefficient fusion schemes, and an imbalance between real-time performance and accuracy, the applicant has made corresponding explorations to address these issues. Summary of the Invention

[0010] The purpose of this application is to solve the above-mentioned problems by providing a palm fruit bunch sorting control method, corresponding device, electronic device and computer-readable storage medium.

[0011] To achieve the various objectives of this application, the following technical solution is adopted:

[0012] A palm fruit bunch sorting control method proposed for one of the purposes of this application includes:

[0013] Collect RGB images of the palm fruit bunches to be sorted in the target conveyor belt, and convert the RGB images into LAB color space images of the palm fruit bunches to be sorted.

[0014] The palm fruit cluster classification and detection model that has been trained to convergence is invoked. The RGB image and the LAB color space image are respectively input into the first branch backbone network and the second branch backbone network in the backbone network for feature extraction, so as to obtain the corresponding RGB texture morphology features and LAB color difference features.

[0015] The RGB texture morphology features and LAB color difference features are spliced ​​together in the channel dimension into 6-channel features and input to the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch area to be sorted.

[0016] The fused features are input into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module. An adaptive dynamic sampling kernel is generated based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module. The fused features are then upsampled to retain the fine-grained maturity features of the palm fruit bunches to be sorted.

[0017] The fused features after upsampling are input into the multi-category classification detection head to output the category label and corresponding confidence level of the palm fruit bunch to be sorted. The corresponding sorting control signal is generated according to the category label corresponding to the highest confidence level and sent to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station, so as to complete the sorting control of the palm fruit bunch.

[0018] Optionally, the basic network architecture of the palm fruit bunch classification and detection model is an improved YOLOv11 model, wherein the first branch backbone network and the second branch backbone network each contain a CBS module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, a CBS module, a C3K2 module and an SPPF module connected in sequence, and a C2PSA-6C cross-branch feature interaction module is deployed between the backbone network and the neck multi-scale feature layer;

[0019] The C2PSA-6C cross-branch feature interaction module includes a 6C-RGBLAB-CBS layer and a C2PSA structure. The 6C-RGBLAB-CBS layer is a CBS layer used for convolutional mapping, batch normalization, and non-linear activation of the 6-channel features after splicing RGB texture morphology features and LAB color difference features. The C2PSA structure is a C2f module that integrates a position-sensitive attention mechanism. The CBS layer includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0020] Optionally, the step of inputting the RGB image and the LAB color space image into the symmetrical first branch backbone network and second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features includes:

[0021] The RGB image of the palm fruit bunch to be sorted is input into the first branch backbone network. The feature extraction process is performed sequentially by each module in the first branch backbone network to obtain the RGB texture morphology features of the palm fruit bunch to be sorted.

[0022] Simultaneously, the LAB color space image of the palm fruit bunch to be sorted is input into the second branch backbone network. The feature extraction process is performed sequentially by each module in the second branch backbone network to obtain the LAB color difference features of the palm fruit bunch to be sorted.

[0023] Optionally, the RGB texture morphology features and LAB color difference features are concatenated into a 6-channel feature in the channel dimension and input into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. This step includes:

[0024] The RGB texture morphology features and the LAB color difference features are concatenated in the channel dimension to obtain 6-channel features, and the 6-channel features are input into the 6C-RGBLAB-CBS layer of the C2PSA-6C cross-branch feature interaction module.

[0025] After convolutional mapping, batch normalization, and activation function layers are sequentially connected in the 6C-RGBLAB-CBS layer to complete convolutional mapping, batch normalization, and nonlinear activation processing, the data is input to the C2PSA structure of the C2PSA-6C cross-branch feature interaction module. The C2PSA structure uses a fused position-sensitive attention mechanism to dynamically interact and fuse the RGB texture morphology features and LAB color difference features in the 6-channel features to obtain fused features.

[0026] Optionally, the step of inputting the fused features into a neck multi-scale feature layer containing a CARAFE content-aware upsampling module, generating an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and upsampling the fused features to retain the fine-grained maturity features of the palm fruit bunches to be sorted includes:

[0027] The fused features are input into the neck multi-scale feature layer of the palm fruit bunch classification and detection model. The CARAFE content-aware upsampling module in the neck multi-scale feature layer generates an adaptive dynamic sampling kernel that matches the features of the palm fruit bunch to be sorted through its built-in content-aware sampling kernel predictor.

[0028] The adaptive dynamic sampling kernel is used to perform upsampling processing on the fused features in order to retain the fine-grained maturity features of the palm fruit bunches to be sorted.

[0029] Optionally, the step of generating a corresponding sorting control signal based on the category label corresponding to the highest confidence level and sending it to the automated sorting device to control the automated sorting device to sort the palm fruit bunches to be sorted to the target workstation includes:

[0030] The category label corresponding to the highest confidence level is matched with the preset palm fruit bunch category-sorting station mapping table to determine the sorting matching result;

[0031] Based on the sorting matching result, a sorting control signal carrying target station information is generated and sent to the automated sorting device. The automated sorting device then sorts the palm fruit bunches to be sorted to the corresponding target stations according to the target station information in the sorting control signal.

[0032] Optionally, the RGB texture morphological features include the visual morphological features of the arrangement density of fruit clusters, the outline shape of fruit clusters, and the surface texture details of palm fruit clusters.

[0033] The LAB color difference feature characterizes the red-green color deviation and yellow-blue color deviation associated with the maturity of palm fruit bunches, and includes brightness channel features, red-green difference channel features, and yellow-blue difference channel features.

[0034] The category labels for the palm fruit bunches include unripe fruit bunches, underripe fruit bunches, overripe fruit bunches, long-stalked fruit bunches, hedgehog fruit bunches, diseased fruit bunches, rodent-damaged fruit bunches, and rotten fruit bunches.

[0035] A palm fruit bunch sorting control device provided for another purpose of this application includes:

[0036] The image acquisition module is configured to acquire RGB images of the palm fruit bunches to be sorted in the target conveyor belt and convert the RGB images into LAB color space images corresponding to the palm fruit bunches to be sorted.

[0037] The feature extraction module is configured to call the palm fruit cluster classification and detection model that has been trained to convergence, and input the RGB image and the LAB color space image into the first branch backbone network and the second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features.

[0038] The feature fusion module is configured to concatenate the RGB texture morphology features and LAB color difference features into a 6-channel feature in the channel dimension and input it into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to perform dynamic interaction and fusion of dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch area to be sorted.

[0039] The upsampling processing module is configured to input the fused features into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module, generate an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and perform upsampling processing on the fused features to retain the fine-grained maturity features of the palm fruit bunch to be sorted.

[0040] The sorting control module is configured to input the fused features after upsampling processing into the multi-category classification detection head to output the category label and corresponding confidence level of the palm fruit bunch to be sorted. Based on the category label corresponding to the highest confidence level, a corresponding sorting control signal is generated and sent to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station, thereby completing the sorting control of the palm fruit bunch.

[0041] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the palm fruit bunch sorting control method of this application.

[0042] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the palm fruit bunch sorting control method, which, when invoked by a computer, executes the steps included in the corresponding method.

[0043] Compared to existing technologies, this application addresses the problems of existing technologies, such as the large number of palm fruit cluster categories, small feature differences, insufficient adaptability to YOLO series algorithms, inefficient fusion schemes, and imbalance between real-time performance and accuracy. This application includes, but is not limited to, the following beneficial effects:

[0044] Firstly, this application constructs a symmetrical dual-branch backbone network to extract RGB texture morphological features and LAB color difference features respectively, while capturing subtle color difference features related to fruit bunch morphology and maturity. This effectively solves the problem of subtle color differences between unripe and ripe fruit bunches, between ripe and overripe fruit bunches, and the problem of misclassification caused by overlapping morphological features of diseased and rotten fruit. It significantly improves the recognition accuracy of neighboring maturity and defect categories, and solves the pain points of low accuracy of manual grading and weak ability of traditional algorithms to distinguish subtle features.

[0045] Secondly, the inefficient fusion method of simply splicing RGB texture features and LAB color difference features without effective interaction is replaced by the C2PSA-6C cross-branch feature interaction module, which dynamically fuses the 6-channel spliced ​​features. Relying on the position-sensitive attention mechanism, the weights of the spatial dimension (focusing on the main body of the fruit bunch) and the channel dimension (strengthening RGB texture and LAB red-green difference channels) are optimized simultaneously. At the same time, the red-green difference channel is assigned a preset multiple of initial weight to further strengthen the key features of maturity. It effectively suppresses the interference of redundant features in the background areas such as branches, leaves, and conveyor belts, improves the collaborative representation ability of dual spatial features, and solves the problem that simple fusion schemes cannot achieve feature complementarity.

[0046] Thirdly, the improved YOLOv11 model in this application replaces the static upsampling method of the original bilinear interpolation in the original YOLOv11. It generates an adaptive dynamic sampling kernel adapted to the fruit bunch features through the CARAFE content-aware upsampling module. This kernel performs fine sampling of the core region of the fruit bunch to preserve fine-grained features such as fruit edges and subtle color gradients, while sparsely sampling the background region to reduce noise, thus avoiding the "average blurring" problem of the original upsampling. This significantly improves the maturity feature retention rate in scenarios with small fruit pieces and dense fruit bunches, enhances the model's anti-interference ability in complex acquisition scenarios such as changes in lighting, fruit bunch stacking, and differences in shooting angles, and solves the pain point of insufficient feature extraction from dense fruit bunches in traditional algorithms.

[0047] Furthermore, this application replaces manual sorting and traditional semi-automatic grading methods, which not only avoids the problems of strong subjectivity, low efficiency and low accuracy of manual sorting, but also realizes the precise conversion of classification results into industrial hardware execution, helping the oil palm industry to achieve refined grading, improve oil extraction efficiency and overall economic benefits. Attached Figure Description

[0048] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0049] Figure 1 This is a flowchart illustrating the palm fruit bunch sorting control method in the embodiments of this application;

[0050] Figure 2 This is a flowchart of the palm fruit bunch sorting control method in the embodiments of this application;

[0051] Figure 3 This is an exemplary architecture of the improved YOLOv11 model in the embodiments of this application;

[0052] Figure 4 This is a diagram illustrating the effect of palm fruit classification and recognition in the embodiments of this application;

[0053] Figure 5 This is a schematic diagram illustrating the palm fruit enterprise maturity discrimination criteria in the embodiments of this application;

[0054] Figure 6 This is a schematic block diagram of the palm fruit bunch sorting control device in the embodiments of this application;

[0055] Figure 7 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0056] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0057] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0058] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0059] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDAs (Personal Digital Assistants) that may include radio frequency receivers, pagers, internet / intranet access, web browsers, notebooks, calendars, and / or GPS (Global Positioning System) receivers; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0060] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0061] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0062] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0063] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0064] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0065] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0066] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0067] Please see Figure 1 and Figure 2 In one embodiment of the palm fruit bunch sorting control method of this application, the method includes:

[0068] Step S10: Acquire the RGB image corresponding to the palm fruit bunch to be sorted in the target conveyor belt, and convert the RGB image into the LAB color space image corresponding to the palm fruit bunch to be sorted;

[0069] The palm fruit bunch sorting control system in the terminal equipment can acquire the RGB image corresponding to the palm fruit bunch to be sorted in the target conveyor belt, and convert the RGB image into the LAB color space image corresponding to the palm fruit bunch to be sorted;

[0070] Specifically, the original RGB images of palm fruit bunches can be captured by a preset image acquisition device, covering different lighting conditions (strong light / weak light / shade), fruit bunch stacking, and shooting angle scenarios, adapting to the dynamic acquisition needs of transport vehicles and conveyor belts.

[0071] The RGB images were standardized using the pre-trained model standard of the palm fruit cluster classification and detection model, with the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225] to scale the pixel values ​​to the [0,1] range. Simultaneously, data augmentation was performed (randomly cropped to 480×480 pixels, horizontal flip probability 0.5, brightness perturbation ±0.1, Gaussian noise variance 0.01) to avoid uneven sample distribution.

[0072] The OpenCV tool calls the function cv2.cvtColor(img, cv2.COLOR_RGB2LAB) to convert the RGB image to a LAB color space image. The LAB color space image contains a luminance channel (L channel), an RGB difference channel (A channel), and a B color difference channel (B channel). The LAB color space image is standardized independently. The luminance channel (L channel) (with values ​​ranging from 0 to 100) is divided by 100 and scaled to [0,1]. The A channel / B channel is first added by 128 and then divided by 255 and scaled to [0,1]. The data augmentation operation is completely synchronized with the RGB image to ensure the alignment of features in both color spaces.

[0073] Step S20: Call the palm fruit cluster classification and detection model that has been trained to convergence, and input the RGB image and the LAB color space image into the first branch backbone network and the second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features.

[0074] The system acquires RGB images of the palm fruit bunches to be sorted on the target conveyor belt, converts the RGB images into LAB color space images corresponding to the palm fruit bunches, and then calls a palm fruit bunch classification and detection model that has been trained to convergence. The RGB images and the LAB color space images are input into the first branch backbone network and the second branch backbone network of the backbone network, respectively, for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features. The RGB texture morphology features include the visual morphological features of the fruit bunch arrangement density, the outline shape of the fruit bunch, and the surface texture details of the palm fruit bunch. The LAB color difference features characterize the red-green color deviation and yellow-blue color deviation features related to the maturity of the palm fruit bunch, and include brightness channel features, red-green difference channel features, and yellow-blue difference channel features.

[0075] In some embodiments, please refer to Figure 3 The basic network architecture of the palm fruit bunch classification and detection model is an improved YOLOv11 model. The first and second branch backbone networks each contain a CBS module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, and an SPPF module connected in sequence. A C2PSA-6C cross-branch feature interaction module is deployed between the backbone network and the neck multi-scale feature layer. The C2PSA-6C cross-branch feature interaction module includes a 6C-RGBLAB-CBS layer and a C2PSA structure. The 6C-RGBLAB-CBS layer is a CBS layer used for convolutional mapping, batch normalization, and non-linear activation of the 6-channel features after splicing RGB texture morphology features and LAB color difference features. The C2PSA structure is a C2f module that integrates a position-sensitive attention mechanism. The CBS layer includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

[0076] In some embodiments, the step of inputting the RGB image and the LAB color space image into the symmetrical first branch backbone network and second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features includes:

[0077] Step S201: Input the RGB image of the palm fruit bunch to be sorted into the first branch backbone network. The feature extraction process is performed sequentially by each module in the first branch backbone network to obtain the RGB texture morphology features of the palm fruit bunch to be sorted.

[0078] Step S202: Simultaneously, the LAB color space image of the palm fruit bunch to be sorted is input into the second branch backbone network. The feature extraction processing is performed sequentially by each module in the second branch backbone network to obtain the LAB color difference features of the palm fruit bunch to be sorted.

[0079] Specifically, the basic network architecture of the palm fruit bunch classification and detection model in this application is an improved YOLOv11 model. Addressing the three core problems of the original YOLOv11 model in palm fruit bunch classification—insufficient single-space feature representation, inefficient dual-space feature interaction, and loss of fine-grained features during upsampling—this application sequentially designs several improved modules, including dual-space fusion input, C2PSA-6C cross-branch feature interaction, and CARAFE content-aware upsampling, to construct the improved YOLOv11 model. These modules are interconnected and interdependent, forming a complete feature processing chain from feature extraction and feature fusion to feature enhancement, gradually solving the pain points of palm fruit bunch maturity and category classification. Specifically, this includes:

[0080] Firstly, to address the shortcomings of RGB single-space only being able to capture morphological texture and LAB single-space lacking texture support, a dual-branch backbone network is constructed. The pre-processed RGB image and LAB color space image are respectively connected to the first branch backbone network and the second branch backbone network, which have a completely symmetrical structure. Both the first branch backbone network and the second branch backbone network adopt a standardized feature extraction process of CBS module, CBS module, C3K2 module, CBS module, C3K2 module, CBS module, C3K2 module and SPPF module in sequence.

[0081] The working principle of the dual-branch backbone network is as follows: The first branch backbone network specifically extracts the RGB texture morphological features of palm fruit bunches, which can characterize visual morphological features such as fruit arrangement density, bunch outline, and surface texture; the second branch backbone network specifically extracts precise LAB color difference features, among which the core red-green color difference channel (A channel) is highly sensitive to the subtle color gradation of "green → orange-red" related to the ripeness of palm fruit bunches. The first and second branch backbone networks extract features in parallel and independently, preserving the modality-specific features of the dual space to the greatest extent and avoiding the representational limitations of single-space features.

[0082] The core beneficial effect of this module in palm fruit maturity discrimination is that, compared with the original YOLOv11 model's RGB single-space input, it can simultaneously capture two-dimensional features including morphological texture and subtle color differences. This fundamentally solves the problem of distinguishing subtle features of the maturity levels of neighboring unripe fruit bunches (light green) and moderately ripe fruit bunches (light orange-red), as well as moderately ripe fruit bunches and overripe fruit bunches (dark orange-red), laying a high-quality dual-space feature foundation for subsequent feature fusion.

[0083] Secondly, to address the problem of simple channel splicing and lack of effective interactive fusion of RGB texture morphology features and LAB color difference features in existing technologies, a C2PSA-6C cross-branch feature interaction module is deployed after the SPPF output layer of the dual-branch backbone network to perform deep fusion of the features output by the dual branches.

[0084] The structural design and working principle of the C2PSA-6C cross-branch feature interaction module are as follows: First, the features extracted by the first branch backbone network and the second branch backbone network are concatenated into a 6-channel feature input in the channel dimension. After feature mapping is completed through the 6C-RGBLAB-CBS layer adapted to the 6 channels, it is connected to the C2PSA structure. At the same time, the red-green difference channel (A channel) in the LAB color space image can be assigned an initial weight of 1.3 times. Through the position-sensitive attention mechanism built into the C2PSA structure, the dual attention weights of the spatial dimension and the channel dimension are calculated simultaneously. The spatial dimension focuses on the main area of ​​the palm fruit bunch, and the channel dimension strengthens the RGB texture features and the red-green difference channel features, realizing the dynamic interaction and deep fusion of dual spatial features, while effectively suppressing the redundant feature interference of background areas such as branches, leaves, and conveyor belts.

[0085] The core beneficial effects of the C2PSA-6C cross-branch feature interaction module on palm fruit maturity discrimination are as follows: Compared with the simple channel splicing method, cross-branch attention fusion significantly improves the collaborative representation ability of RGB texture morphology features and LAB color difference features, further improving the recognition accuracy of neighboring ripe fruit clusters such as unripe and moderately ripe; and the module adopts a lightweight design, with an increase of ≤5% in the number of parameters, which will not cause a significant decrease in the model inference speed, balancing feature fusion effect and model real-time performance.

[0086] Third, to address the "average blurring" problem of the original bilinear interpolation upsampling in the original YOLOv11 model, the CARAFE content-aware upsampling module is deployed in the multi-scale feature layer at the neck of the model to replace the original static upsampling method, thereby achieving multi-scale enhancement and fine-grained preservation of fused features.

[0087] The CARAFE content-aware upsampling module's structure and working principle are as follows: The built-in content-aware sampling kernel predictor adaptively generates matching dynamic sampling kernels based on the content of the fused features. The kernel size can be adaptively adjusted to two different sizes: 3×3 and 5×5, i.e., a 3x3 sampling kernel and a 5x5 sampling kernel, depending on the complexity of the fruit bunch features. Fine sampling kernels are allocated to the main area of ​​the palm fruit bunch to accurately preserve fine-grained maturity features such as fruit edges and subtle color differences. Sparse sampling kernels are allocated to the background area to weaken background noise interference, achieving feature perception and accurate preservation during the upsampling process.

[0088] The core benefits of CARAFE's content-aware upsampling module for palm fruit maturity determination are: it completely avoids the problem of fine-grained feature loss caused by the original upsampling, improves the maturity feature retention rate by 15% in scenarios with small fruit pieces and dense fruit clusters, and effectively improves the classification accuracy in complex scenarios such as light interference; moreover, the module is highly lightweight, and while improving feature quality, it fully meets the real-time requirements of high-speed conveyor belt sorting.

[0089] In some embodiments, the training process of the improved YOLOv11 model of this application includes:

[0090] Dataset: 4293 palm fruit bunch labeled samples were compiled by 8 senior Indonesian oil palm grading workers, based on the SNI01-2301-2008 national oil palm grading standard and enterprise harvesting standards. The labels cover 8 categories (1-Immature fruit bunch, 2-Unripe fruit bunch, 3-Overripe fruit bunch, 4-Long-stalked fruit bunch, 5-Hedgehog fruit bunch, 6-Disease fruit bunch, 7-Rodent-damaged fruit bunch, 8-Rotten fruit bunch). The dataset is divided into a training set (3005 images), a validation set (859 images), and a test set (429 images) in a 7:2:1 ratio.

[0091] Training Environment: The hardware system is based on a computing platform, configured with an Intel Core i7-12700KF @ 3.6GHz CPU, an NVIDIA GeForce RTX 5060 Ti 16G GPU, two 16G DDR4 memory modules, and a high-speed solid-state drive. The software environment is developed based on the Ultralytics YOLO framework, compatible with Python 3.10.19, PyTorch 2.10.0.dev20251111, and CUDA 13.0, providing efficient computing power support for the training and inference of the palm fruit bunch classification model, meeting the needs of model iteration optimization and subsequent edge deployment, and adapting to the computational needs of real-time grading of palm fruit bunches in conveyor belt scenarios.

[0092] Training parameters: The optimizer used was AdamW, with an initial learning rate of 0.001, which decayed to 0.0001 and 0.00001 in rounds 50 and 80, respectively; the batch size was 16, and the number of iterations was 200; the loss function used was weighted multi-class cross-entropy loss (assigning 1.3 times the weight to the hedgehog fruit cluster and rodent damage fruit cluster categories with smaller sample sizes to balance the sample distribution).

[0093] Training process: The features output from the dual branches are input into the C2PSA-6C module. After fusion, upsampling, and multi-scale feature enhancement, they are input into the multi-class classification head. The weights of each improved module are optimized through backpropagation, and the iteration continues until the validation set loss is ≤0.08.

[0094] Model lightweighting module: PyTorch Quantization tool is used to perform INT8 quantization, converting the model weights from 32-bit floating-point to 8-bit integer, while performing channel pruning (pruning ratio of 30%) to remove redundant channels;

[0095] Inference Output Layer: The lightweight model is deployed on the NVIDIA Jetson Xavier NX embedded terminal and connected to the industrial camera and conveyor belt sorting device. During operation, the industrial camera acquires RGB images in real time, and simultaneously completes RGB2LAB conversion, preprocessing, and dual-branch input. After inputting the model for inference, the classification results (category labels and confidence scores) are output. The processing time for a single fruit string is ≤50ms.

[0096] Step S30: The RGB texture morphology features and LAB color difference features are spliced ​​into 6-channel features in the channel dimension and input to the C2PSA-6C cross-branch feature interaction module. The position-sensitive attention mechanism is used to perform dynamic interaction and fusion of dual-space features to determine the fusion features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to strengthen the key features of the palm fruit bunch area to be sorted.

[0097] The palm fruit bunch classification and detection model, which has been trained to convergence, is invoked. The RGB image and the LAB color space image are respectively input into the first branch backbone network and the second branch backbone network of the backbone network for feature extraction. After obtaining the corresponding RGB texture morphology features and LAB color difference features, the RGB texture morphology features and LAB color difference features are concatenated into 6-channel features in the channel dimension and input into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to strengthen the key features of the palm fruit bunch area to be sorted.

[0098] In some embodiments, the RGB texture morphology features and LAB color difference features are concatenated into a 6-channel feature in the channel dimension and input to the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. This includes the following steps:

[0099] Step S301: The RGB texture morphology features and the LAB color difference features are concatenated in the channel dimension to obtain 6-channel features, and the 6-channel features are input to the 6C-RGBLAB-CBS layer of the C2PSA-6C cross-branch feature interaction module;

[0100] Step S302: After the convolutional layer, batch normalization layer, and activation function layer connected in sequence in the 6C-RGBLAB-CBS layer complete the convolutional mapping, batch normalization, and nonlinear activation processing, the data is input to the C2PSA structure of the C2PSA-6C cross-branch feature interaction module. The C2PSA structure uses a fused position-sensitive attention mechanism to dynamically interact and fuse the RGB texture morphology features and LAB color difference features in the 6-channel features to obtain fused features.

[0101] As can be seen from steps S301 to S302 above, the 6C-RGBLAB-CBS layer performs convolution, normalization, and activation processing to specifically adapt the 6-channel features, achieving effective mapping and dimensional adaptation of the dual-space splicing features, laying a high-quality feature foundation for subsequent feature fusion. Relying on the position-sensitive attention mechanism of C2PSA structure fusion, the dynamic interaction and deep fusion of RGB texture morphology features and LAB color difference features are realized, strengthening the texture morphology features and color difference features of the core area of ​​the fruit bunch and effectively suppressing redundant feature interference from backgrounds such as branches, leaves, and conveyor belts. Compared with the simple channel splicing method, it greatly improves the collaborative representation ability of dual-space features, can more accurately distinguish the subtle feature differences of the maturity levels of neighboring palm fruit bunches, and significantly improves the recognition accuracy of unripe and moderately ripe fruit bunches.

[0102] Step S40: Input the fused features into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module, generate an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and perform upsampling processing on the fused features to retain the fine-grained maturity features of the palm fruit bunch to be sorted.

[0103] The RGB texture morphology features and LAB color difference features are concatenated into a 6-channel feature in the channel dimension and input into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch region to be sorted. Then, the fused features are input into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module. An adaptive dynamic sampling kernel is generated based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module to upsample the fused features to retain the fine-grained maturity features of the palm fruit bunch to be sorted.

[0104] In some embodiments, the step of inputting the fused features into a neck multi-scale feature layer containing a CARAFE content-aware upsampling module, generating an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and upsampling the fused features to retain the fine-grained maturity features of the palm fruit bunches to be sorted includes:

[0105] Step S401: Input the fused features into the neck multi-scale feature layer of the palm fruit bunch classification and detection model. The CARAFE content-aware upsampling module in the neck multi-scale feature layer generates an adaptive dynamic sampling kernel that matches the features of the palm fruit bunch to be sorted through its built-in content-aware sampling kernel predictor.

[0106] Step S402: The adaptive dynamic sampling kernel is used to perform upsampling processing on the fusion features to retain the fine-grained maturity features of the palm fruit bunches to be sorted.

[0107] As described in steps S401 to S402 above, the content-aware sampling kernel predictor generates an adaptive dynamic sampling kernel that matches the features of the palm fruit bunch. This kernel size can be flexibly matched to the complexity of the fruit bunch region features, enabling differentiated processing of fine sampling in the core area and sparse sampling in the background area, accurately matching the characteristics of dense fruit and complex background in palm fruit bunches. Upsampling of the fused features based on the adaptive dynamic sampling kernel effectively avoids the "averaging blurring" problem of the original bilinear interpolation upsampling, significantly improving the retention rate of fine-grained maturity-related features in scenarios such as small fruit pieces and dense fruit bunches. Simultaneously, it accurately preserves key maturity features such as fruit edges and subtle color gradients. This significantly enhances the model's anti-interference ability in complex collection scenarios such as lighting changes and fruit bunch stacking, effectively improving the classification accuracy of the nearest neighbor maturity levels of palm fruit bunches, and only slightly increasing the model inference time; the processing time for a single fruit bunch still meets the real-time requirements of high-speed sorting on industrial conveyor belts. Multi-scale feature enhancement is performed on the fused RGB texture morphology features and LAB color difference features, which further enhances the semantic expression capability of the dual-space fusion features. This provides a high-quality, high-resolution feature foundation for the accurate discrimination of subsequent multi-class classification detection heads, and helps to improve the overall classification accuracy.

[0108] Step S50: Input the fused features after upsampling processing into the multi-category classification detection head to output the category label and corresponding confidence level of the palm fruit bunch to be sorted. Generate the corresponding sorting control signal according to the category label corresponding to the highest confidence level and send it to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station to complete the sorting control of the palm fruit bunch.

[0109] In some embodiments, please refer to Figure 4 and Figure 5 The palm fruit bunch category labels include unripe bunches, underripe bunches, overripe bunches, long-stalked bunches, hedgehog bunches, diseased bunches, rodent-damaged bunches, and rotten bunches. The maturity criteria for palm fruit enterprises are as follows: Unripe bunches have no signs of fruit detachment, the bunch structure is intact, the peel is dark black and dull, and the orange-red proportion is less than 10%; Underripe bunches have a small number of detached fruits (1 to 10 fruits / bundle), the overall color is light orange-red, and the dark black base color has not completely faded; Overripe bunches have a large number of detached fruits (50%-90% / bundle), the main body of the bunch is exposed, and the remaining fruits are dark brown and dull; Long-stalked bunches have protruding remaining stalks at the base of the bunch, with a length significantly exceeding that of normal bunches (labeled pixel length > 120px); Hedgehog bunches are... Single-core fruit characteristics: >50% of fruit bunches have hedgehog-like protrusions on the surface, irregular shape, >20 protrusions per fruit and protrusion height >3mm; diseased fruit bunches have internal rot and brown paste, irregularity of outline >0.4, and abnormal reflectance area in near-infrared band (905mm) >15%; rodent-damaged fruit bunches have obvious nibbling marks on the surface of the fruit, nibbled fruit quotient >10% and defect depth >1mm; rotten fruit bunches have mold and yellowing on the surface of the fruit, with some areas showing dark green mold spots, yellowing / mold spot area quotient >20% and spot texture roughness >0.8.

[0110] In some embodiments, the step of generating a corresponding sorting control signal based on the category label corresponding to the highest confidence level and sending it to the automated sorting device to control the automated sorting device to sort the palm fruit bunches to be sorted to the target workstation includes:

[0111] Step S501: Match the category label corresponding to the highest confidence level with the preset palm fruit bunch category-sorting station mapping table to determine the sorting matching result;

[0112] Step S502: Generate a sorting control signal carrying target station information based on the sorting matching result, and send the sorting control signal to the automated sorting device. The automated sorting device sorts the palm fruit bunches to be sorted to the corresponding target station according to the target station information in the sorting control signal.

[0113] As can be seen from steps S501 to S502 above, using the category label corresponding to the highest confidence level as the sole matching criterion avoids the sorting judgment confusion caused by the overlap of multiple label confidence levels, ensuring the uniqueness and accuracy of the sorting matching results and reducing the probability of missorting from the source. By pre-setting the mapping relationship table between palm fruit bunch categories and sorting stations, the abstract algorithm classification labels are directly transformed into specific sorting station information, realizing the accurate adaptation of classification results with industry sorting processes, national oil palm grading standards, and enterprise harvesting requirements, thus improving the industrial applicability of the algorithm. Standardized sorting control signals carrying target station information are generated, providing clear and identifiable execution instructions for automated sorting devices, realizing seamless connection between algorithm inference results and hardware execution actions, and ensuring the automated and intelligent advancement of the entire "classification-sorting" process. The automated sorting device completes precise sorting based on the target station information in the control signal, enabling palm fruit bunches of different maturity, shape and defect types to be quickly diverted to the corresponding processing station, which meets the needs of the oil palm industry for refined processing and helps to improve the efficiency of oil extraction and the overall economic benefits of the industry.

[0114] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems existing in the prior art, such as the large number of palm fruit bunch categories, small feature differences, insufficient adaptability of YOLO series algorithms, inefficient fusion schemes, and imbalance between real-time performance and accuracy. This application has, but is not limited to, the following beneficial effects:

[0115] Firstly, this application constructs a symmetrical dual-branch backbone network to extract RGB texture morphological features and LAB color difference features respectively, while capturing subtle color difference features related to fruit bunch morphology and maturity. This effectively solves the problem of subtle color differences between unripe and ripe fruit bunches, between ripe and overripe fruit bunches, and the problem of misclassification caused by overlapping morphological features of diseased and rotten fruit. It significantly improves the recognition accuracy of neighboring maturity and defect categories, and solves the pain points of low accuracy of manual grading and weak ability of traditional algorithms to distinguish subtle features.

[0116] Secondly, the inefficient fusion method of simply splicing RGB texture features and LAB color difference features without effective interaction is replaced by the C2PSA-6C cross-branch feature interaction module, which dynamically fuses the 6-channel spliced ​​features. Relying on the position-sensitive attention mechanism, the weights of the spatial dimension (focusing on the main body of the fruit bunch) and the channel dimension (strengthening RGB texture and LAB red-green difference channels) are optimized simultaneously. At the same time, the red-green difference channel is assigned a preset multiple of initial weight to further strengthen the key features of maturity. It effectively suppresses the interference of redundant features in the background areas such as branches, leaves, and conveyor belts, improves the collaborative representation ability of dual spatial features, and solves the problem that simple fusion schemes cannot achieve feature complementarity.

[0117] Thirdly, the improved YOLOv11 model in this application replaces the static upsampling method of the original bilinear interpolation in the original YOLOv11. It generates an adaptive dynamic sampling kernel adapted to the fruit bunch features through the CARAFE content-aware upsampling module. This kernel performs fine sampling of the core region of the fruit bunch to preserve fine-grained features such as fruit edges and subtle color gradients, while sparsely sampling the background region to reduce noise, thus avoiding the "average blurring" problem of the original upsampling. This significantly improves the maturity feature retention rate in scenarios with small fruit pieces and dense fruit bunches, enhances the model's anti-interference ability in complex acquisition scenarios such as changes in lighting, fruit bunch stacking, and differences in shooting angles, and solves the pain point of insufficient feature extraction from dense fruit bunches in traditional algorithms.

[0118] Furthermore, this application replaces manual sorting and traditional semi-automatic grading methods, which not only avoids the problems of strong subjectivity, low efficiency and low accuracy of manual sorting, but also realizes the precise conversion of classification results into industrial hardware execution, helping the oil palm industry to achieve refined grading, improve oil extraction efficiency and overall economic benefits.

[0119] Please see Figure 6A palm fruit bunch sorting control device provided for one of the purposes of this application includes an image acquisition module 1100, a feature extraction module 1200, a feature fusion module 1300, an upsampling processing module 1400, and a sorting control module 1500. The image acquisition module 1100 is configured to acquire RGB images of the palm fruit bunches to be sorted in the target conveyor belt and convert the RGB images into LAB color space images corresponding to the palm fruit bunches to be sorted. The feature extraction module 1200 is configured to call a palm fruit bunch classification and detection model that has been trained to convergence, and input the RGB images and the LAB color space images into the symmetrical first branch backbone network and the second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features. The feature fusion module 1300 is configured to concatenate the RGB texture morphology features and LAB color difference features into 6-channel features in the channel dimension and input them into the C2PSA-6C cross-branch feature interaction module, and use a position-sensitive attention mechanism to perform dynamic interaction and fusion of dual-space features to determine the fused features. At the same time, during the fusion process, the LAB color space image is analyzed. The red-green difference channel is assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch region to be sorted; the upsampling processing module 1400 is configured to input the fused features into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module, generate an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and perform upsampling processing on the fused features to retain the fine-grained maturity features of the palm fruit bunch to be sorted; the sorting control module 1500 is configured to input the upsampled fused features into the multi-class classification detection head to output the class label and corresponding confidence of the palm fruit bunch to be sorted, generate the corresponding sorting control signal according to the class label corresponding to the highest confidence and send it to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station to complete the sorting control of the palm fruit bunch.

[0120] Based on any embodiment of this application, please refer to Figure 7 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 7The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, they enable the processor to implement a palm fruit bunch sorting control method. The processor of the computer device provides computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When these computer-readable instructions are executed by the processor, they enable the processor to execute the palm fruit bunch sorting control method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0121] In this embodiment, the processor is used to execute... Figure 6 The specific functions of each module are defined within the device, and the memory stores the program code and various data required to execute these modules. A network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the palm fruit bunch sorting control device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0122] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the palm fruit bunch sorting control method described in any embodiment of this application.

[0123] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the palm fruit bunch sorting control method described in any embodiment of this application.

[0124] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0125] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for controlling the sorting of palm fruit bunches, characterized in that, include: Collect RGB images of the palm fruit bunches to be sorted in the target conveyor belt, and convert the RGB images into LAB color space images of the palm fruit bunches to be sorted. The palm fruit cluster classification and detection model that has been trained to convergence is invoked. The RGB image and the LAB color space image are respectively input into the first branch backbone network and the second branch backbone network in the backbone network for feature extraction, so as to obtain the corresponding RGB texture morphology features and LAB color difference features. The RGB texture morphology features and LAB color difference features are spliced ​​together in the channel dimension into 6-channel features and input to the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch area to be sorted. The fused features are input into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module. An adaptive dynamic sampling kernel is generated based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module. The fused features are then upsampled to retain the fine-grained maturity features of the palm fruit bunches to be sorted. The fused features after upsampling are input into the multi-category classification detection head to output the category label and corresponding confidence level of the palm fruit bunch to be sorted. The corresponding sorting control signal is generated according to the category label corresponding to the highest confidence level and sent to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station, so as to complete the sorting control of the palm fruit bunch.

2. The palm fruit bunch sorting control method according to claim 1, characterized in that, The basic network architecture of the palm fruit bunch classification and detection model is an improved YOLOv11 model. The first branch backbone network and the second branch backbone network each contain a CBS module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, a CBS module, a C3K2 module, and an SPPF module connected in sequence. A C2PSA-6C cross-branch feature interaction module is deployed between the backbone network and the neck multi-scale feature layer. The C2PSA-6C cross-branch feature interaction module includes a 6C-RGBLAB-CBS layer and a C2PSA structure. The 6C-RGBLAB-CBS layer is a CBS layer used for convolutional mapping, batch normalization, and non-linear activation of the 6-channel features after splicing RGB texture morphology features and LAB color difference features. The C2PSA structure is a C2f module that integrates a position-sensitive attention mechanism. The CBS layer includes a convolutional layer, a batch normalization layer, and an activation function layer connected in sequence.

3. The palm fruit bunch sorting control method according to claim 2, characterized in that, The steps of inputting the RGB image and the LAB color space image into the symmetrical first branch backbone network and second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features include: The RGB image of the palm fruit bunch to be sorted is input into the first branch backbone network. The feature extraction process is performed sequentially by each module in the first branch backbone network to obtain the RGB texture morphology features of the palm fruit bunch to be sorted. Simultaneously, the LAB color space image of the palm fruit bunch to be sorted is input into the second branch backbone network. The feature extraction process is performed sequentially by each module in the second branch backbone network to obtain the LAB color difference features of the palm fruit bunch to be sorted.

4. The palm fruit bunch sorting control method according to claim 2, characterized in that, The RGB texture morphology features and LAB color difference features are concatenated into a 6-channel feature in the channel dimension and input into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to dynamically interact and fuse the dual-space features to determine the fused features. The steps include: The RGB texture morphology features and the LAB color difference features are concatenated in the channel dimension to obtain 6-channel features, and the 6-channel features are input into the 6C-RGBLAB-CBS layer of the C2PSA-6C cross-branch feature interaction module. After convolutional mapping, batch normalization, and activation function layers are sequentially connected in the 6C-RGBLAB-CBS layer to complete convolutional mapping, batch normalization, and nonlinear activation processing, the data is input to the C2PSA structure of the C2PSA-6C cross-branch feature interaction module. The C2PSA structure uses a fused position-sensitive attention mechanism to dynamically interact and fuse the RGB texture morphology features and LAB color difference features in the 6-channel features to obtain fused features.

5. The palm fruit bunch sorting control method according to claim 1, characterized in that, The steps of inputting the fused features into a neck multi-scale feature layer containing a CARAFE content-aware upsampling module, generating an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and upsampling the fused features to retain the fine-grained maturity features of the palm fruit bunches to be sorted include: The fused features are input into the neck multi-scale feature layer of the palm fruit bunch classification and detection model. The CARAFE content-aware upsampling module in the neck multi-scale feature layer generates an adaptive dynamic sampling kernel that matches the features of the palm fruit bunch to be sorted through its built-in content-aware sampling kernel predictor. The adaptive dynamic sampling kernel is used to perform upsampling processing on the fused features in order to retain the fine-grained maturity features of the palm fruit bunches to be sorted.

6. The palm fruit bunch sorting control method according to any one of claims 1 to 5, characterized in that, The steps of generating a corresponding sorting control signal based on the category label corresponding to the highest confidence level and sending it to the automated sorting device to control the automated sorting device to sort the palm fruit bunches to the target workstation include: The category label corresponding to the highest confidence level is matched with the preset palm fruit bunch category-sorting station mapping table to determine the sorting matching result; Based on the sorting matching result, a sorting control signal carrying target station information is generated and sent to the automated sorting device. The automated sorting device then sorts the palm fruit bunches to be sorted to the corresponding target stations according to the target station information in the sorting control signal.

7. The palm fruit bunch sorting control method according to any one of claims 1 to 5, characterized in that, The RGB texture morphological features include the visual morphological features of the arrangement density of fruit clusters, the outline shape of fruit clusters, and the surface texture details of palm fruit clusters. The LAB color difference feature characterizes the red-green color deviation and yellow-blue color deviation associated with the maturity of palm fruit bunches, and includes brightness channel features, red-green difference channel features, and yellow-blue difference channel features. The category labels for the palm fruit bunches include unripe fruit bunches, underripe fruit bunches, overripe fruit bunches, long-stalked fruit bunches, hedgehog fruit bunches, diseased fruit bunches, rodent-damaged fruit bunches, and rotten fruit bunches.

8. A palm fruit bunch sorting control device, characterized in that, include: The image acquisition module is configured to acquire RGB images of the palm fruit bunches to be sorted in the target conveyor belt and convert the RGB images into LAB color space images corresponding to the palm fruit bunches to be sorted. The feature extraction module is configured to call the palm fruit cluster classification and detection model that has been trained to convergence, and input the RGB image and the LAB color space image into the first branch backbone network and the second branch backbone network of the backbone network respectively for feature extraction to obtain the corresponding RGB texture morphology features and LAB color difference features. The feature fusion module is configured to concatenate the RGB texture morphology features and LAB color difference features into a 6-channel feature in the channel dimension and input it into the C2PSA-6C cross-branch feature interaction module. A position-sensitive attention mechanism is used to perform dynamic interaction and fusion of dual-space features to determine the fused features. At the same time, during the fusion process, the red and green difference channels in the LAB color space image are assigned an initial weight of a preset multiple to enhance the key features of the palm fruit bunch area to be sorted. The upsampling processing module is configured to input the fused features into the neck multi-scale feature layer containing the CARAFE content-aware upsampling module, generate an adaptive dynamic sampling kernel based on the content-aware sampling kernel predictor in the CARAFE content-aware upsampling module, and perform upsampling processing on the fused features to retain the fine-grained maturity features of the palm fruit bunch to be sorted. The sorting control module is configured to input the fused features after upsampling processing into the multi-category classification detection head to output the category label and corresponding confidence level of the palm fruit bunch to be sorted. Based on the category label corresponding to the highest confidence level, a corresponding sorting control signal is generated and sent to the automated sorting device to control the automated sorting device to sort the palm fruit bunch to be sorted to the target station, thereby completing the sorting control of the palm fruit bunch.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images