Fusion detection system for panicle tomato maturity identification and picking point positioning

Abstract

The invention discloses a fusion detection system for panicle tomato maturity identification and picking point positioning. The fusion detection system comprises a data processing module, a DBS-YOLOv5s target detection model, a Mask R-CNN segmentation model and a fusion calculation module. The generalization ability of the model is improved through data enhancement, a double-layer routing attention mechanism and a dynamic up-sampler are introduced into the DBS-YOLOv5s model, and high-precision identification of the maturity of the panicle tomatoes is realized in combination with an improved Shape-IoU loss function; a Mask R-CNN model optimizes a segmentation label strategy, a stalk mask region is generated, and the segmentation accuracy is ensured. And the fusion calculation module positions the optimal picking point through mathematical operation, so that the problems of large positioning error and low detection efficiency in the prior art are solved, the model is light in weight and wide in applicability, and the intelligent level of automatic identification and picking of the panicle tomatoes is remarkably improved.

Description

Technical Field

[0001] The invention relates to the technical field of agricultural automation, and in particular to a fusion detection system for identifying the maturity of ear tomatoes and locating picking points. Background Art

[0002] Panicle tomatoes are an important economic crop. Their fruits grow in clusters and have multiple levels of maturity, that is, the same cluster of fruits may contain green, semi-ripe, ripe, and fully ripe fruits. Panicle tomatoes need to be picked after the fruits reach a certain maturity. Picking them at different times not only affects the quality of the fruit, but also increases the complexity of the picking operation. Accurately identifying the maturity of panicle tomatoes and accurately locating the picking point is a key technical issue in agricultural automation.

[0003] In the prior art, in order to realize the maturity identification and picking point positioning of ear tomatoes, the following methods are usually adopted:

[0004] 1. Manual picking: This is the main picking method in traditional agriculture. The ripeness of the tomatoes is judged by the naked eye and the picking position is manually selected. Although manual picking has a certain degree of flexibility, it is inefficient and easily affected by human judgment errors, resulting in fruit damage or wrong picking. Especially in large-scale planting environments, the cost and efficiency of manual picking are difficult to meet actual needs.

[0005] 2. Automated methods based on target detection: For example, the YOLO series of models are widely used for target detection of fruits, and the maturity of fruits is classified by color features. However, existing target detection technologies are difficult to accurately determine the maturity ratio of fruits, especially in the color distribution analysis of whole bunches of tomatoes. In addition, the target detection model alone cannot accurately locate the picking point.

[0006] 3. Improved solution based on instance segmentation: Some studies in the industry have attempted to use the Mask R-CNN segmentation model to segment the stems of tomato clusters. However, due to the complex geometric boundaries between the stems and fruits, traditional segmentation methods are easily affected by background interference, resulting in incomplete stem segmentation, which further affects the positioning accuracy of the picking point.

[0007] Although these methods have been improved, the target detection technology has not performed well in grading the multi-level maturity of tomato bunches, especially in calculating the color ratio of the whole bunch of fruit, where it is difficult to achieve high-precision recognition, affecting the accuracy of picking decisions; traditional instance segmentation models cannot reliably handle the complexity of the stem and fruit boundaries, and the segmentation results are easily affected by noise and occlusion, resulting in inaccurate positioning of the picking point; although the multi-sensor fusion solution can improve detection accuracy and robustness, it is difficult to be widely used in resource-constrained agricultural environments due to the high equipment cost and strong deployment complexity.

[0008] Therefore, how to achieve accurate identification of the maturity of ear tomatoes and accurate positioning of the picking point has become a technical problem to be solved by the present invention. Summary of the invention

[0009] The technical problem solved by the present invention is to provide a fusion detection system for tomato ear maturity identification and picking point positioning in response to the defects existing in the above-mentioned prior art, so as to solve the problems of insufficient maturity identification accuracy, large picking point positioning error, adaptability and cost limitations of the prior art mentioned in the above-mentioned background technology.

[0010] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0011] A fusion detection system for identifying the maturity of tomato ears and locating the picking point, comprising:

[0012] A data processing module, used for collecting tomato images and generating a diversified training data set through a data enhancement method, wherein the data enhancement method includes symmetrical cropping, translation, random brightness adjustment, noise addition, random rotation and their combination changes;

[0013] The DBS-YOLOv5s target detection model is used to identify the maturity of the tomato image. The DBS-YOLOv5s target detection model includes:

[0014] The backbone network uses the BiFormer module based on a two-layer routing attention mechanism to extract key features through dynamic sparse attention;

[0015] The neck network uses a dynamic upsampler DySample instead of the default nearest neighbor interpolation method to improve image resolution and the ability to capture semantic information;

[0016] The improved Shape-IoU loss function is used to pass the formula L = α·IoU+β·Scale factor The bounding box shape and scale are comprehensively considered to optimize the target detection accuracy, where α and β are weight coefficients;

[0017] The Mask R-CNN segmentation model is used to perform instance segmentation on the stalk of the panicle tomato, wherein the mask region generated by the Mask R-CNN can mark the position of the stalk of the panicle tomato, and the mask region contains the upper and lower boundary coordinate information. The Mask R-CNN segmentation model ensures the integrity of the stalk curve by optimizing the labeling strategy, avoids the occlusion of the main trunk and branch connection, and avoids the coverage of the fruit area to the greatest extent during the segmentation process;

[0018] The fusion calculation module is used to locate the best picking point of the tomato cluster based on the following mathematical calculation method:

[0019] Extract the upper and lower boundary coordinate points a1 and a2 of the mask area segmented by Mask R-CNN, and calculate the midpoint a3 of the connecting line segment L;

[0020] Calculate the slope k1 of line segment L, and construct the perpendicular line L2 through point a3 according to the formula k2=-1 / k1;

[0021] Determine the intersection O of L2 and the nearest point within the mask range as the picking point.

[0022] As a further solution of the present invention, the backbone network of the DBS-YOLOv5s target detection model introduces a double-layer routing attention mechanism so that each query only focuses on key tags related to the tomato features to avoid distraction of irrelevant features and improve detection accuracy and computational efficiency.

[0023] As a further solution of the present invention, the Mask R-CNN segmentation model maintains the integrity of the stem segmentation results by limiting the fruit area within the label coverage.

[0024] As a further solution of the present invention, the fusion calculation module further determines the final picking point based on the distance relationship between the top midpoint of the tomato maturity detection label box output by DBS-YOLOv5s and the picking point.

[0025] As a further solution of the present invention, the data enhancement method includes performing random color jittering and random affine transformation on the image to further improve the diversity of the data set and the generalization ability of the model.

[0026] As a further solution of the present invention, in the improved Shape-IoU loss function, the weight coefficient of the bounding box shape is calculated by the formula w h =h t / H and w w =w t / W dynamic adjustment, where h t and w t are the height and width of the target box, H and W are the total height and width of the image, respectively.

[0027] As a further solution of the present invention, the mask region in the Mask R-CNN segmentation model enhances the efficiency of candidate region generation by introducing a region generation network, and the screening criteria of the candidate region are based on the shape characteristics and curvature information of the panicle tomato stem.

[0028] As a further solution of the present invention, when generating the mask area, the Mask R-CNN segmentation model performs weighted processing on the pixel values ​​of the boundaries between the fruit and the stem to reduce the segmentation error caused by boundary blur.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The DBS-YOLOv5s target detection model not only improves the feature extraction capability and resolution of target detection by introducing a two-layer routing attention mechanism (BiFormer module), a dynamic upsampler (DySample), and an improved Shape-IoU loss function, but also optimizes the maturity recognition accuracy by comprehensively considering the bounding box shape and scale. It is significantly better than traditional target detection models and meets the needs of high-precision agriculture.

[0031] 2. The Mask R-CNN model generates high-quality mask areas by optimizing the labeling strategy, effectively avoiding occlusion of the fruit area while maintaining the integrity of the stem curve, providing a solid foundation for the precise positioning of the subsequent picking points. In addition, the fusion calculation module realizes the automatic positioning of the picking points through mathematical calculation methods (such as boundary point extraction, line segment slope calculation, and vertical line intersection positioning), significantly reducing the subjective errors of human judgment.

[0032] 3. By combining the maturity recognition output of the DBS-YOLOv5s model with the mask segmentation results of the Mask R-CNN model, this application proposes a unique multi-task collaboration method, which successfully achieves the full process optimization of tomato from maturity detection to picking point positioning. This fusion method takes into account both detection speed and positioning accuracy, significantly improving the intelligence level of the system.

[0033] 4. The data enhancement module combines symmetric cropping, random brightness adjustment, and affine transformation to significantly improve data diversity and model generalization, enabling the system to adapt to complex and changing planting environments. At the same time, the solution is optimized for the unlimited growth characteristics of panicle tomatoes, and can be further promoted and applied to other fruit and vegetable picking tasks, with strong industrial value.

[0034] 5. Dynamic sparse attention (BiFormer) is used to reduce model computational redundancy, and a lightweight dynamic upsampler (DySample) is used to significantly reduce computing resource requirements while improving model performance. This makes it suitable for deployment on agricultural automation equipment with limited resources, and is both efficient and economical.

[0035] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is the overall framework diagram of the system of the present invention.

[0038] Figure 2 This is a schematic diagram of picking point positioning according to the present invention.

[0039] Figure 3 This is a detailed diagram of the DBS-YOLOv5s model improvement of the present invention.

[0040] Figure 4 This is a mathematical model diagram of the fusion calculation module of the present invention. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] See also Figure 1 —4. In an embodiment of the present invention, a fusion detection system for identifying the maturity of ear tomatoes and locating the picking point includes:

[0043] A data processing module, used for collecting tomato images and generating a diversified training data set through a data enhancement method, wherein the data enhancement method includes symmetrical cropping, translation, random brightness adjustment, noise addition, random rotation and their combination changes;

[0044] The DBS-YOLOv5s target detection model is used to identify the maturity of the tomato image. The DBS-YOLOv5s target detection model includes:

[0045] The backbone network uses the BiFormer module based on a two-layer routing attention mechanism to extract key features through dynamic sparse attention;

[0046] The neck network uses a dynamic upsampler DySample instead of the default nearest neighbor interpolation method to improve image resolution and the ability to capture semantic information;

[0047] The improved Shape-IoU loss function is used to pass the formula L = α·IoU+β·Scalefactor The bounding box shape and scale are comprehensively considered to optimize the target detection accuracy, where α and β are weight coefficients;

[0048] The Mask R-CNN segmentation model is used to perform instance segmentation on the stalk of the panicle tomato, wherein the mask region generated by the Mask R-CNN can mark the position of the stalk of the panicle tomato, and the mask region contains the upper and lower boundary coordinate information. The Mask R-CNN segmentation model ensures the integrity of the stalk curve by optimizing the labeling strategy, avoids the occlusion of the main trunk and branch connection, and avoids the coverage of the fruit area to the greatest extent during the segmentation process;

[0049] The fusion calculation module is used to locate the best picking point of the tomato cluster based on the following mathematical calculation method:

[0050] Extract the upper and lower boundary coordinate points a1 and a2 of the mask area segmented by Mask R-CNN, and calculate the midpoint a3 of the connecting line segment L;

[0051] Calculate the slope k1 of line segment L, and construct the perpendicular line L2 through point a3 according to the formula k2=-1 / k1;

[0052] Determine the intersection O of L2 and the nearest point within the mask range as the picking point.

[0053] As a further solution of the present invention, the backbone network of the DBS-YOLOv5s target detection model introduces a two-layer routing attention mechanism so that each query only focuses on key tags related to the ear tomato features to avoid distraction of attention by irrelevant features and improve detection accuracy and computational efficiency; specifically, the mechanism uses dynamic sparse attention to screen out key feature areas related to the ear tomato target in the image, suppresses background noise and irrelevant information, and thus avoids distraction. Its working mode includes two layers of routing: the first layer preliminarily screens potential target areas on a global scale to ensure the integrity of target coverage; the second layer performs refined extraction of target features in the preliminarily screened area to extract maturity features such as the shape and color of the fruit. Through this method, not only the detection accuracy is significantly improved, the problems of false detection and missed detection are reduced, but also the model's waste of computing resources on irrelevant features is reduced, thereby achieving higher detection efficiency and reliability in complex agricultural scenarios.

[0054] The Mask R-CNN segmentation model limits the fruit area within the label coverage to maintain the integrity of the stem segmentation results; the Mask R-CNN segmentation model optimizes the label strategy to set a clear pixel boundary between the fruit and stem areas to avoid excessive coverage of the fruit area affecting the accuracy of stem segmentation. Specifically, the model effectively limits the fruit area within the label coverage to ensure a clear geometric boundary between the fruit and the stem, thereby maintaining the integrity of the stem segmentation results. In addition, by optimizing the mask generation of the stem area, background interference and segmentation errors are minimized, providing accurate data support for subsequent picking point positioning.

[0055] The fusion calculation module further determines the final picking point based on the distance relationship between the top midpoint of the tomato maturity detection label frame output by DBS-YOLOv5s and the picking point; the data enhancement method includes random color jittering and random affine transformation of the image to further improve the diversity of the data set and the generalization ability of the model. The fusion calculation module determines the optimal picking point position through geometric calculation based on the maturity detection label output by the DBS-YOLOv5s model. Specifically, the module extracts the coordinates of the top midpoint of the label frame, combines the boundary information in the mask area, calculates the distance between the midpoint and the candidate picking point, and accurately identifies the optimal picking point through mathematical formulas (such as slope calculation and vertical intersection positioning). In addition, the data enhancement method significantly improves the diversity of the data set by introducing technologies such as random color jittering and random affine transformation, thereby improving the generalization ability of the model for complex scenes, enabling it to adapt to changing lighting conditions and background noise, and improving the stability and robustness of overall detection and positioning.

[0056] As a further solution of the present invention, in the improved Shape-IoU loss function, the weight coefficient of the bounding box shape is calculated by the formula w t =h t / H and w w =w t / W dynamic adjustment, where h t and w t are the height and width of the target box, H and W are the total height and width of the image, respectively.

[0057] The mask region in the Mask R-CNN segmentation model enhances the efficiency of candidate region generation by introducing a region generation network. The screening criteria of the candidate region are based on the shape characteristics and curvature information of the tomato stem. When generating the mask region, the Mask R-CNN segmentation model performs weighted processing on the pixel values ​​of the boundary between the fruit and the stem to reduce the segmentation error caused by the blurred boundary. The Mask R-CNN segmentation model significantly improves the efficiency of candidate region generation by introducing a region generation network (RPN). The network selects high-quality candidate regions based on the shape characteristics and curvature information of the tomato stem to ensure more accurate mask generation. At the same time, during the mask region generation process, the model performs weighted processing on the pixel values ​​of the boundary between the fruit and the stem. By strengthening the resolution of the boundary region, the segmentation error that may be caused by the blurred boundary is reduced. This optimization strategy not only improves the integrity of the stem region segmentation, but also reduces the risk of interference in the fruit region, providing more reliable data support for the subsequent picking point positioning.

[0058] Embodiment 1:

[0059] In modern agriculture, in order to improve the economic value and picking efficiency of ear tomatoes, it is necessary to accurately identify the maturity of ear tomatoes and accurately locate the picking points during the automated picking process. However, the traditional manual picking method has the problems of low efficiency and large subjective errors, and the existing automated system is unable to simultaneously take into account the high-precision identification of maturity and the accurate positioning of the picking points due to the limitations of technical means. Against the above background, this embodiment provides a fusion detection system, which combines an improved target detection model and an instance segmentation model to realize the intelligent picking of ear tomatoes in an agricultural planting environment.

[0060] The system is mainly composed of a data processing module, a DBS-YOLOv5s target detection model, a Mask R-CNN segmentation model, and a fusion calculation module. The system installs a high-resolution RGB camera in the greenhouse for growing tomato clusters to collect image data of tomato clusters in real time. The collected image data is enhanced by the data processing module, including symmetrical cropping, random brightness adjustment, noise addition, and random rotation to increase data diversity and model robustness. The enhanced image data is used to train the DBS-YOLOv5s target detection model and the Mask R-CNN segmentation model, thereby improving the system's recognition ability under complex lighting and background interference.

[0061] The system first detects and identifies the tomato cluster targets in the image using the DBS-YOLOv5s target detection model. The model uses the dynamic sparse attention mechanism (BiFormer) of the backbone network to improve the accuracy of extracting the features of different maturity levels of tomato clusters, enabling the system to accurately classify green, semi-ripe, ripe, and fully ripe fruits in the same bunch of fruits; the dynamic upsampling technology (DySample) of the neck network enhances the ability to analyze the color ratio of the entire bunch of tomatoes, providing high-precision support for the calculation of the maturity of the entire bunch of fruits; the regression accuracy of the detection model is optimized by combining the shape and scale of the target bounding box through the Shape-IoU loss function. Through the above improvements, the system can accurately classify the maturity of tomato clusters in complex planting environments and generate annotation box information to distinguish fruits at different maturity stages.

[0062] After identifying the maturity, the system passes the image to the Mask R-CNN segmentation model, which performs instance segmentation on the stem area of ​​the panicle tomato through an optimized labeling strategy. During the segmentation process, label optimization ensures the integrity of the stem curve while avoiding coverage of the fruit area. The mask area generated by the segmentation model accurately marks the stem position and contains the boundary coordinate information of the stem, providing basic data for subsequent picking point positioning. The segmentation technology effectively reduces the interference of lighting changes and complex backgrounds on stem identification, ensuring the integrity and accuracy of the segmentation results.

[0063] The system's fusion calculation module combines the maturity annotation box information output by the DBS-YOLOv5s model and the stalk mask area generated by the Mask R-CNN model, and determines the picking point based on the following mathematical calculation method: extract the upper and lower boundary coordinate points a1 and a2 of the mask area, calculate the midpoint a3 of the connecting line segment L; calculate the slope k1 of the line segment L, and construct the vertical line L2 through point a3 according to the formula k2=-1 / k1; determine the intersection O of L2 and the nearest point within the mask range as the picking point. Finally, the system outputs the picking point information of the mature tomato ears, and performs precise picking through the robotic arm.

[0064] Through this embodiment, the system shows the following significant advantages in real planting scenarios: the improved DBS-YOLOv5s model significantly improves the accuracy of calculating the maturity of the whole bunch of fruit in complex lighting conditions and multi-level maturity scenarios; combined with the mask segmentation and mathematical operation methods of the Mask R-CNN model, the picking point positioning error is controlled at the millimeter level, solving the deviation problem caused by incomplete segmentation in traditional methods; through data enhancement and optimization of the dynamic sparse attention mechanism, the system adapts to changes in different lighting and complex backgrounds, and can operate stably in both greenhouse environments and open-air planting environments; the full process optimization of tomato bunches from maturity recognition to picking point positioning is realized, significantly reducing the need for manual participation.

[0065] Embodiment 2:

[0066] In the process of planting unlimited-growth panicle tomatoes, due to the long growth cycle of the fruit and the large difference in maturity, the traditional picking method and simple automatic recognition system are difficult to meet the requirements of accurate fruit picking. In view of this problem, this embodiment was practiced in a certain agricultural science and technology demonstration area, and the improved DBS-YOLOv5s target detection model and Mask R-CNN segmentation model were used to realize efficient maturity recognition and picking point positioning of panicle tomatoes, and on this basis, the agricultural production process was optimized.

[0067] In the experiment, an unlimited growth type panicle tomato planting base was selected as the research object. First, 674 panicle tomato images with a resolution of 658×492 were collected. In order to improve the generalization ability of the model, the data processing module used data enhancement techniques such as symmetric cropping, random brightness adjustment, and adding noise to expand the images to 6740. The data set was divided into training set, validation set, and test set in a ratio of 8:1:1, and the panicle tomato fruits and stems in the image were annotated through a strict labeling strategy. Among them, the color ratio of the fruit is used to distinguish different maturity stages (green ripe, semi-ripe, ripe and fully ripe), and the stem area ensures that the label completely covers the curve contour to avoid blocking the main trunk.

[0068] In the training and application of the model, the improved DBS-YOLOv5s target detection model improves the feature extraction capability through the dynamic sparse attention mechanism (BiFormer), enhances the analysis accuracy of the color ratio of the whole bunch of tomatoes through the dynamic upsampler (DySample), and optimizes the shape and scale of the bounding box in combination with the Shape-IoU loss function. When the system is running, the DBS-YOLOv5s model can output the maturity grading results of the bunch of tomatoes in real time, and provide accurate maturity label box information for the whole bunch of fruits.

[0069] On this basis, the Mask R-CNN model is used to segment the stem area. Through the optimized segmentation label strategy, the system can fully retain the curve characteristics of the stem while avoiding interference with the coverage of the fruit area. The stem mask area generated by the model contains the key coordinates of the upper and lower boundaries of the stem, providing accurate data support for the subsequent picking point positioning.

[0070] Through this embodiment, the system shows the following advantages in real agricultural scenarios:

[0071] 1. High adaptability to unlimited-growth panicle tomatoes: Through strict labeling strategies and grading standards, the system can fully cover the fruit and stem characteristics of panicle tomatoes at different growth stages.

[0072] 2. High maturity recognition accuracy: The improved DBS-YOLOv5s model can accurately calculate the color ratio of a bunch of tomatoes to meet actual production needs.

[0073] 3. Accurate picking point positioning: Combining mathematical calculation methods with the mask data of the segmentation model, the error of picking point positioning is controlled at the millimeter level, effectively improving the picking efficiency.

[0074] 4. Lightweight model and efficient operation: Through the introduction of dynamic upsampling technology and sparse attention mechanism, the system achieves high precision and real-time performance while reducing computing resource usage.

[0075] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "screw-on" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0076] Embodiment 3:

[0077] See also Figure 2 ,In the figure, a1: the lower boundary coordinate point of the stem mask area. a2: the upper boundary coordinate point of the stem mask area.

[0078] exist Figure 2 In FIG. 1 , the straight line segment connecting points a1 and a2 forms L. a3 is the midpoint of line segment L. The perpendicular line through midpoint a3 is L2. O is the final picking point, and L2 is the intersection of the perpendicular line L2 and the mask boundary.

[0079] This embodiment further illustrates how the system locates the best picking point in practical applications based on mathematical calculation methods, combines the maturity annotation box information output by the DBS-YOLOv5s target detection model with the stalk mask area generated by the Mask R-CNN segmentation model, and uses geometric calculation methods to accurately determine the picking point position of the tomato ear.

[0080] 1. Mathematical calculation method: First, the Mask R-CNN segmentation model is used to extract the upper and lower boundary coordinate points a1 and a2 of the mask area of ​​the tomato stalk, such as Figure 2 shown.

[0081] Calculate the line segment L connecting a1 and a2, and determine the midpoint a3. Based on the slope k1 of the line segment L, construct a perpendicular line L2 passing through the midpoint a3, whose slope k2 satisfies

[0082] The intersection of the vertical line L2 and the nearest boundary point in the stalk mask area is determined as the optimal picking point. The position of the picking point O will be used as the system output result, and the robot arm will achieve accurate positioning and picking.

[0083] In actual tests, this embodiment shows the following advantages in the maturity detection and picking point positioning of tomato panicles: the error of picking point O is controlled at the millimeter level, ensuring the accuracy of the robotic arm when picking; by combining the model with mathematical operations, the system adapts to different lighting conditions and complex backgrounds to achieve stable detection and positioning; through optimized mathematical model calculations, the system can output picking point information in real time, greatly improving the efficiency of picking operations.

[0084] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A fusion detection system for identifying the maturity of ear tomatoes and locating the picking point, characterized in that: include: A data processing module, used for collecting tomato images and generating a diversified training data set through a data enhancement method, wherein the data enhancement method includes symmetrical cropping, translation, random brightness adjustment, noise addition, random rotation and their combination changes; The DBS-YOLOv5s target detection model is used to identify the maturity of the tomato image. The DBS-YOLOv5s target detection model includes: The backbone network uses the BiFormer module based on a two-layer routing attention mechanism to extract key features through dynamic sparse attention; The neck network uses a dynamic upsampler DySample instead of the default nearest neighbor interpolation method to improve image resolution and the ability to capture semantic information; The improved Shape-IoU loss function is used to pass the formula L = α·IoU+β·Scale factor The bounding box shape and scale are comprehensively considered to optimize the target detection accuracy, where α and β are weight coefficients; The Mask R-CNN segmentation model is used to perform instance segmentation on the tomato stalk, wherein the mask region generated by the Mask R-CNN can mark the position of the tomato stalk, and the mask region contains the upper and lower boundary coordinate information. The Mask R-CNN segmentation model ensures the integrity of the stalk curve by optimizing the labeling strategy; The fusion calculation module is used to locate the best picking point of the tomato cluster based on the following mathematical calculation method: Extract the upper and lower boundary coordinate points a1 and a2 of the mask area segmented by Mask R-CNN, and calculate the midpoint a3 of the connecting line segment L; Calculate the slope k1 of line segment L, and construct the perpendicular line L2 through point a3 according to the formula k2=-1 / k1; Determine the intersection O of L2 and the nearest point within the mask range as the picking point.

2. A fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: The backbone network of the DBS-YOLOv5s target detection model introduces a two-layer routing attention mechanism so that each query only focuses on key tags related to the features of the tomato.

3. A fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: The Mask R-CNN segmentation model constrains the fruit area within the label coverage.

4. A fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: The fusion calculation module further determines the final picking point based on the distance relationship between the top midpoint of the tomato maturity detection label box output by DBS-YOLOv5s and the picking point.

5. The fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: Data augmentation methods include random color jittering and random affine transformation of images.

6. The fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: In the improved Shape-IoU loss function, the weight coefficient of the bounding box shape is calculated by the formula w h =h t / H and w w =w t / W dynamic adjustment, where h t and w t are the height and width of the target box, H and W are the total height and width of the image, respectively.

7. The fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: The mask region in the Mask R-CNN segmentation model enhances the efficiency of candidate region generation by introducing a region generation network, and the screening criteria of the candidate region are based on the shape characteristics and curvature information of the panicle tomato stem.

8. The fusion detection system for identifying the maturity of ear tomatoes and locating the picking point according to claim 1, characterized in that: When generating the mask area, the Mask R-CNN segmentation model weights the pixel values ​​at the boundaries of fruits and stems to reduce the segmentation error caused by blurred boundaries.

Images