Method, system and equipment for accurately erasing complex grape buds based on two-stage visual localization and medium

Abstract

The invention discloses a two-stage visual positioning grape complex bud accurate erasing method, system and equipment and a medium, and relates to the technical field of fruit and vegetable bud management. The method comprises the following steps: driving a working platform to move along grape branches through a crawler-type chassis, and detecting result branches in a first visual field by using a depth camera and a global visual recognition algorithm. And if not, continuing to move, if so, stopping and calculating the relative space relationship between the execution mechanism and the target coordinate, adjusting the industrial camera to a second visual field, and judging the complex bud condition by using a local visual recognition algorithm. If no complex bud exists and one or more complex buds exist, the pixel position relation is calculated to enable a laser point to be aligned with a target bud, a laser module is triggered to complete single operation, repeated detection is conducted till the buds are removed, and after an execution mechanism is reset, the movement continues to be conducted for next round of operation till all bud picking is completed. According to the method, accurate recognition and positioning of complex buds can be realized, and high quality, high yield and standardized management of grapes are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of fruit and vegetable sprout management, and in particular to a method, system, equipment and medium for accurately erasing complex grape sprouts using dual-stage visual positioning. Background Art

[0002] During their growth, grapes often form complex buds, including hidden buds, weak buds, and secondary buds within concurrent buds. These buds present varying degrees of difficulty in identification, depending on their location, nutrient distribution, or structural characteristics. Hidden buds are difficult to spot due to their proximity to older branches or obscuration. Weak buds, due to their poor development, small size, and weak growth potential, struggle to become fruiting branches after germination. Secondary buds, growing alongside primary buds and small in size, can easily lead to double-ended branches, hindering ventilation, light transmission, and nutrient utilization. If these complex buds are not removed promptly, they can impact the overall branch load, fruit quality, and garden management efficiency, making them key targets for identification in current precision bud removal.

[0003] Existing grape bud removal methods mainly include three forms: manual bud removal, mechanical bud removal, and spraying bud removal. Manual bud removal relies on the experience of fruit farmers. It visually identifies the bud type and removes them one by one manually. Although it has a certain degree of recognition accuracy and operational flexibility and can deal with different buds in a targeted manner, it is labor-intensive and inefficient. It relies heavily on the operator's experience and judgment, making it difficult to achieve standardization and large-scale production. Mechanical bud removal devices mostly use rotary cutting or dial structures, which can quickly complete large-scale operations and significantly reduce labor costs. However, due to the lack of visual recognition capabilities, they cannot distinguish between bud types and are very likely to accidentally damage the main buds, affecting the later growth of grapes and the quality of fruit bearing. Spraying bud removal uses chemical means to inhibit bud growth. It has a wide coverage range and is easy to operate. However, because it cannot accurately identify the buds, there is a risk of damage to the main buds, pesticide damage, and environmental pollution. It does not meet the current development requirements of green agriculture. In recent years, some intelligent mechanical bud removal equipment has tried to introduce visual recognition technology, such as using RGB-D cameras combined with deep learning algorithms to identify secondary buds, and using robotic arms to control laser modules to achieve fixed-point erasure. This has improved the level of automation and recognition accuracy to a certain extent. However, it currently focuses on secondary bud recognition, and its recognition ability for complex buds such as hidden buds and weak buds is still limited. In addition, the equipment has a complex structure, high manufacturing cost, and slow operating speed, making it difficult to meet large-scale and high-intensity operation needs.

[0004] The existing bud removal method is limited by the recognition ability and operation method, and it is difficult to accurately remove complex buds such as hidden buds and weak buds. It is easy to cause damage to the main buds or retention of invalid buds, affecting the grape branch load, ventilation and light transmission, and nutrient distribution, thereby reducing fruit quality and yield. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, system, equipment and medium for accurately erasing complex grape buds with dual-stage visual positioning, aiming to solve or improve at least one of the above-mentioned technical problems.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A method for accurately erasing complex grape buds using dual-stage visual positioning, comprising:

[0008] The crawler chassis drives the work platform to move along the direction of the grape branches to be tested. The instance segmentation model in the main controller is activated, the depth camera is aligned with the first field of view, and the global visual recognition algorithm is used to determine the recognition results in the first field of view:

[0009] If the recognition result in the first field of view is that no fruiting branch is detected, continue moving; if the recognition result in the first field of view is that a fruiting branch is detected, stop moving and calculate the relative spatial relationship between the target coordinates and the actuator;

[0010] Determine the horizontal rotation angle and pitch rotation angle of the actuator based on the relative spatial relationship, align the industrial camera with the second field of view, and use the local visual recognition algorithm to determine the recognition result in the second field of view:

[0011] If the recognition result in the second field of view is that there is no complex bud, continue moving; if the recognition result in the second field of view is that there is a single complex bud, calculate the pixel position relationship between the target bud coordinates and the laser action point, and align the laser point with the target bud based on the pixel position relationship; if the recognition result in the second field of view is that there are multiple complex buds, perform path planning, determine the processing order, and align the laser point with the target bud based on the pixel position relationship between the target bud coordinates and the laser action point; the complex buds include hidden buds, weak buds, and accessory buds;

[0012] The laser module is triggered to complete a single bud removal operation, and the main controller is instructed to re-run the instance segmentation model for detection until all buds in the first field of view and the second field of view have been removed. The actuator is reset, and the crawler chassis is used to drive the working platform to continue moving along the direction of the grape branch to be tested, and a new round of bud removal operation is carried out until all bud removal operations are completed.

[0013] Optionally, the first field of view is 50 to 60 cm in front of the grape vine; and the second field of view is 50 to 60 cm in front of the grape vine magnified 5 times.

[0014] Optionally, the global visual recognition algorithm adopts the RT-DETR target detection network, and the specific detection process is:

[0015] Inputting the image of the first field of view into the RT-DETR target detection network for recognition, determining whether there is a fruiting branch, and when a fruiting branch is detected, determining whether there is a bud, if a bud exists and is located on the fruiting branch, extracting the two-dimensional coordinates of the fruiting branch detection frame; if a bud exists and is not located on the fruiting branch, extracting the two-dimensional coordinates of the bud detection frame;

[0016] Using the image registration method, the three-dimensional coordinates in the camera coordinate system are determined according to the depth information of the extracted detection frame center, and the three-dimensional coordinates in the world coordinate system are converted using the coordinate conversion formula. The three-dimensional coordinates in the world coordinate system are defined as the target coordinates, and the relative spatial relationship between the target coordinates and the actuator is calculated.

[0017] Optionally, the local visual recognition algorithm adopts a SAM model, and the specific detection process is as follows:

[0018] The image of the second field of view is input into the SAM model for recognition. First, old branches, single buds and secondary buds are distinguished. Then, the spatial relationship analysis and feature judgment of old branches and single buds are performed: if the mask of the single bud and the old branch overlap or the distance is less than the set threshold, it is determined to be a hidden bud; if the area of the remaining single bud is less than 30% of the average value, it is determined to be a weak bud;

[0019] When any of the hidden buds, weak buds, and accessory buds exists, it is determined that a single complex bud exists. The two-dimensional coordinates of the corresponding bud are extracted and defined as the target bud coordinates. The pixel position relationship between the target bud coordinates and the laser action point is calculated, and the laser point is aligned with the target bud.

[0020] When the number of latent buds, weak buds and accessory buds is greater than or equal to two, it is determined that there are multiple complex buds. The two-dimensional coordinates of each bud are extracted respectively, and each two-dimensional coordinate is defined as the target bud coordinate. The pixel position relationship between each target bud coordinate and the laser action point is calculated, and the path planning is performed according to the pixel position relationship. The processing order is determined, and the laser point is aligned with the target bud according to the processing order and the pixel position relationship between the target bud coordinate and the laser action point.

[0021] Optionally, the specific process of aligning the laser point with the target bud is:

[0022] According to the pixel position relationship between the target bud coordinates and the laser action point, the pixel difference is calculated, and the fine-tuning angle of the laser module turntable is calculated according to the pixel difference to achieve the alignment of the laser point with the bud.

[0023] The present invention also provides a dual-stage visual positioning system for precisely erasing complex grape buds, which uses the above method and includes: a main controller, a depth camera, an industrial camera, a crawler chassis, an actuator, and a laser module;

[0024] The main controller is respectively connected to the depth camera, the industrial camera, the crawler chassis, the actuator and the laser module, and the industrial camera is also connected to the actuator; the working process of the system is as follows:

[0025] The crawler chassis drives the work platform to move along the direction of the grape branches to be tested. The instance segmentation model in the main controller is activated, the depth camera is aligned with the first field of view, and the global visual recognition algorithm is used to determine the recognition results in the first field of view:

[0026] If the recognition result in the first field of view is that no fruiting branch is detected, continue moving; if the recognition result in the first field of view is that a fruiting branch is detected, stop moving and calculate the relative spatial relationship between the target coordinates and the actuator;

[0027] Determine the horizontal rotation angle and pitch rotation angle of the actuator based on the relative spatial relationship, align the industrial camera with the second field of view, and use the local visual recognition algorithm to determine the recognition result in the second field of view:

[0028] If the recognition result in the second field of view is that there is no complex bud, continue moving; if the recognition result in the second field of view is that there is a single complex bud, calculate the pixel position relationship between the target bud coordinates and the laser action point, and align the laser point with the target bud based on the pixel position relationship; if the recognition result in the second field of view is that there are multiple complex buds, perform path planning, determine the processing order, and align the laser point with the target bud based on the pixel position relationship between the target bud coordinates and the laser action point; the complex buds include hidden buds, weak buds, and accessory buds;

[0029] The laser module is triggered to complete a single bud removal operation, and the main controller is instructed to re-run the instance segmentation model for detection until all buds in the first field of view and the second field of view have been removed. The actuator is reset, and the crawler chassis is used to drive the working platform to continue moving along the direction of the grape branch to be tested, and a new round of bud removal operation is carried out until all bud removal operations are completed.

[0030] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for accurately erasing complex grape buds based on the above-mentioned dual-stage visual positioning.

[0031] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for accurately erasing complex grape buds using dual-stage visual positioning.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] The present invention discloses a method, system, equipment and medium for accurately erasing complex grape buds with dual-stage visual positioning. The method comprises the following steps: driving a working platform along the grape branches by a crawler chassis, and detecting the fruiting branches in the first field of view using a depth camera and a global visual recognition algorithm. If no branches are detected, the movement continues; if detected, the movement stops and the relative spatial relationship between the actuator and the target coordinates is calculated, the industrial camera is adjusted to the second field of view, and the complex bud situation is judged using a local visual recognition algorithm. If there are no complex buds, the movement continues; if there are single or multiple complex buds, the pixel position relationship is calculated to align the laser point with the target bud, the laser module is triggered to complete a single operation, and the detection is repeated until the buds are erased. After the actuator is reset, the movement continues for the next round of operation until all buds are removed. The present invention can realize the accurate identification and positioning of complex buds, which contributes to the high-quality, high-yield and standardized management of grapes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flowchart of global visual positioning in this embodiment;

[0036] Figure 2 Schematic diagram of the global bud image in this embodiment;

[0037] Figure 3 This is a flowchart of local fine segmentation and positioning in this embodiment;

[0038] Figure 4 This is a schematic diagram of the image of the fruiting branches and buds in this embodiment;

[0039] Figure 5 Schematic diagram of the overall process in this embodiment. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] The purpose of the present invention is to provide a method, system, equipment and medium for accurately erasing complex grape buds with dual-stage visual positioning, aiming to solve or improve at least one of the above-mentioned technical problems.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1-Figure 5 As shown, the present invention provides a method for accurately erasing complex grape buds using dual-stage visual positioning, comprising:

[0044] Step 100: Use the crawler chassis to drive the work platform to move along the direction of the grape branches to be tested, start the instance segmentation model in the main controller, aim the depth camera at the first field of view, and use the global visual recognition algorithm to determine the recognition result in the first field of view:

[0045] If the recognition result in the first field of view is that no fruiting branch is detected, the movement continues; if the recognition result in the first field of view is that a fruiting branch is detected, the movement is stopped and the relative spatial relationship between the target coordinates and the actuator is calculated.

[0046] Step 200: Determine the horizontal rotation angle and the pitch rotation angle of the actuator based on the relative spatial relationship, align the industrial camera with the second field of view, and use the local visual recognition algorithm to determine the recognition result in the second field of view:

[0047] If the recognition result in the second field of view is that there is no complex bud, continue moving; if the recognition result in the second field of view is that there is a single complex bud, calculate the pixel position relationship between the target bud coordinates and the laser action point, and aim the laser point at the target bud according to the pixel position relationship; if the recognition result in the second field of view is that there are multiple complex buds, perform path planning, determine the processing order and aim the laser point at the target bud according to the pixel position relationship between the target bud coordinates and the laser action point; the complex buds include hidden buds, weak buds and accessory buds.

[0048] Step 300: Trigger the laser module to complete a single bud removal operation, and instruct the main controller to re-run the instance segmentation model for detection until all buds in the first field of view and the second field of view have been removed, reset the actuator, and use the crawler chassis to drive the working platform to continue moving along the direction of the grape branch to be tested, and perform a new round of bud removal operation until all bud removal operations are completed.

[0049] As a specific implementation, the visual recognition algorithm used above is described in detail:

[0050] For global visual recognition algorithms:

[0051] like Figure 1 As shown in the figure, a depth camera is used to collect the global bud RGB image 50 to 60 cm in front of the grape vine ( Figure 2 ); use image annotation software to finely annotate the image data, and the labels are divided into fruiting branches and buds, and the labels are divided into training set, test set and validation set; use RT-DETR target detection network to train and optimize the data set to obtain a fruiting branch and bud detection network with good results; in the actual detection process, first determine whether the bud is located on the fruiting branch, if so, extract the two-dimensional coordinates of the fruiting branch detection frame; if not, extract the two-dimensional coordinates of the bud detection frame; use RGB image and depth image registration, extract the three-dimensional coordinates in the camera coordinate system according to the depth information of the detection frame center, and then convert it into a three-dimensional position in the world coordinate system.

[0052] For local visual recognition algorithms:

[0053] like Figure 3 As shown, an industrial camera was used to collect RGB images of fruiting branches and buds at a magnification of 5 times at a distance of 50 to 60 cm from the front of the grape vine ( Figure 4 ), construct a training data set labeled with old branches, single buds, and auxiliary buds, and divide it into training set, test set, and validation set; use SAM model for instance segmentation training and optimization; after RGB image input, the system first segments and processes old branches, single buds, and auxiliary buds; then performs spatial relationship analysis and feature judgment on old branches and buds: if the single bud overlaps with the old branch mask or the distance is less than the set threshold, it is determined to be a hidden bud and removed; if the area of the remaining single bud is less than 30% of the average value, it is determined to be a weak bud and also removed; only the main bud is retained among the concurrent buds, and the auxiliary buds are removed; the system effectively distinguishes the retained buds, hidden buds, weak buds, and auxiliary buds; outputs the two-dimensional coordinates of hidden buds, weak buds, and auxiliary buds to achieve accurate identification, screening, and positioning of grape buds.

[0054] As a specific implementation method, Figure 5 The overall process shown is explained as follows:

[0055] S1: The program starts and the system loads core components such as the main controller, depth camera, industrial camera, crawler chassis, actuator, and laser module in sequence;

[0056] S2: After the system is started, the crawler chassis drives the work platform to slowly move forward along the direction of the grape branches, while the global vision system is activated for preliminary environmental perception and positioning;

[0057] S3: The system determines whether there are fruiting branches in the current field of view. If no fruiting branches are detected, the vehicle continues to move forward and performs global visual positioning. If a fruiting branch is detected, the vehicle controls the chassis to stop moving.

[0058] S4: Obtain the three-dimensional coordinates of the fruiting branch or outer bud through global visual positioning. Formula (1) is the conversion of pixel coordinate system coordinates into three-dimensional coordinates in the world coordinate system:

[0059]

[0060] in, It is the camera internal parameter, which describes the internal structure of the camera imaging. It is the camera external parameter, which defines the position and orientation of the camera.

[0061] Where u, v are the pixel coordinates on the image (unit: pixel); Xw, Yw, Zw are three-dimensional points in the world coordinate system; R is the 3×3 rotation matrix; T is the 3×1 translation vector; Zc is the depth information; and f is the camera focal length.

[0062] The system further identifies the position of the fruiting branch or bud in the image and determines the relative spatial relationship between it and the actuator;

[0063] S5: Based on the identified spatial information, the system calculates the horizontal rotation angle and the pitch rotation angle of the actuator. The calculation formula is shown in formula (2):

[0064]

[0065] Where θ is the horizontal rotation angle, is the pitch rotation angle; ||P|| is the Euclidean norm of the target point P = (Xw, Yw, Zw).

[0066] S6: The actuator rotates rapidly according to the calculated rotation angle, so that the industrial camera installed thereon is aimed at the target (i.e., a specific fruiting branch or bud position);

[0067] S7: Entering the local visual recognition stage, using high-precision image segmentation algorithms to further process and locate the target area;

[0068] S8: The system determines whether there are complex buds (hidden buds, weak buds and accessory buds) in the recognition area;

[0069] S9: If the recognition result indicates that there is no complex bud, the actuator is controlled to directly turn to the next fruiting branch or outer bud; if there is a complex bud, the pixel difference is calculated based on the relative pixel position relationship between the laser action point and the target bud, and the fine-tuning angle of the turntable is calculated based on the pixel difference to achieve the laser point aligning with the bud. The formula is as follows:

[0070]

[0071] where x laser ,y laseris the pixel coordinate of the laser point in the camera field of view; where x bud ,y bud is the pixel coordinate of the bud in the camera field of view; Δx, Δy is the pixel difference between the bud and the laser point; Δθ, is the U-shaped turntable for fine-tuning the angle; f is the focal length of the camera.

[0072] S10: The system determines whether there are multiple buds to be erased (i.e., multiple target points) in the current area;

[0073] S11: If there are multiple target points, the system plans a reasonable processing sequence and path based on the recognition results and performs laser operations in sequence; if there is only a single target point, the actuator angle is adjusted to align with the target;

[0074] S12: The system determines whether the laser module on the current actuator has been accurately aligned with the target bud;

[0075] S13: If the alignment is not precise, the actuator is controlled to further rotate and adjust so that the laser point is aligned with the target bud; if the alignment is correct, the laser is triggered to emit, completing a single bud removal operation;

[0076] S14: After the laser operation is completed, the main controller re-runs the instance segmentation model and re-analyzes the RGB image obtained by the industrial camera to determine whether all buds in the area have been completely erased;

[0077] S15: If all target buds have not been erased, re-judge whether there are still multiple buds that need to be erased in the current field of view, and continue to execute the corresponding processing flow; if the erasing operation of all buds has been completed, control the actuator to rotate so that the industrial camera installed on it is aimed at the next fruiting branch, repeat the local fine segmentation and positioning operations, and continue the bud erasing operation.

[0078] S16: When all fruiting branches or external buds within the field of view of the depth camera are erased, the control actuator is reset.

[0079] S17: After the bud erasing program is finished, the system resources are released and the system enters the standby state;

[0080] S18: The crawler chassis continues to move forward slowly, and the system enters the next position and repeats the above process to perform a new round of bud removal operation.

[0081] This process uses a five-stage processing method of global positioning - local refinement - intelligent judgment - multi-point planning - laser execution, integrating visual recognition, spatial modeling, path planning and mechanical execution, to achieve automatic detection, judgment and precise laser operation of complex grape buds, significantly improving operational efficiency and intelligence.

[0082] Therefore, the present invention has the following beneficial effects:

[0083] (1) Through global and local two-level visual recognition and positioning technology, the accurate segmentation and positioning of latent buds, weak buds and secondary buds in the spring grape bud removal operation is achieved, which improves the efficiency, accuracy and bud removal ability of the bud removal device. (2) Each system module functions independently and operates in coordination, which is convenient for functional expansion or software and hardware upgrades according to different crops or application scenarios in the future. (3) The system can automatically plan the operation path according to the number and distribution of buds, support multi-target sequential processing, effectively optimize the execution sequence, and improve overall operation efficiency. (4) By integrating visual recognition, path planning and execution control, the system realizes full-process automated operation, significantly reduces manual dependence, reduces labor intensity, and can achieve accurate removal of different types of buds (hidden buds, weak buds and secondary buds), improve operation efficiency and bud removal accuracy, and meet the needs of modern vineyard intelligent management.

[0084] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0085] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for accurately erasing complex grape buds using dual-stage visual positioning, characterized in that: include: The crawler chassis drives the work platform to move along the direction of the grape branches to be tested. The instance segmentation model in the main controller is activated, the depth camera is aligned with the first field of view, and the global visual recognition algorithm is used to determine the recognition results in the first field of view: If the recognition result in the first field of view is that no fruiting branch is detected, continue moving; if the recognition result in the first field of view is that a fruiting branch is detected, stop moving and calculate the relative spatial relationship between the target coordinates and the actuator; Determine the horizontal rotation angle and pitch rotation angle of the actuator based on the relative spatial relationship, align the industrial camera with the second field of view, and use the local visual recognition algorithm to determine the recognition result in the second field of view: If the recognition result in the second field of view is that there is no complex bud, continue moving; if the recognition result in the second field of view is that there is a single complex bud, calculate the pixel position relationship between the target bud coordinates and the laser action point, and align the laser point with the target bud based on the pixel position relationship; if the recognition result in the second field of view is that there are multiple complex buds, perform path planning, determine the processing order, and align the laser point with the target bud based on the pixel position relationship between the target bud coordinates and the laser action point; the complex buds include hidden buds, weak buds, and accessory buds; The laser module is triggered to complete a single bud removal operation, and the main controller is instructed to re-run the instance segmentation model for detection until all buds in the first field of view and the second field of view have been removed. The actuator is reset, and the crawler chassis is used to drive the working platform to continue moving along the direction of the grape branch to be tested, and a new round of bud removal operation is carried out until all bud removal operations are completed.

2. The method for accurately erasing complex grape buds using dual-stage visual positioning according to claim 1 is characterized in that: The first field of view is 50 to 60 cm in front of the grape vine; the second field of view is 50 to 60 cm in front of the grape vine, magnified 5 times.

3. The method for accurately erasing complex grape buds using dual-stage visual positioning according to claim 1 is characterized in that: The global visual recognition algorithm adopts the RT-DETR target detection network, and the specific detection process is as follows: Inputting the image of the first field of view into the RT-DETR target detection network for recognition, determining whether there is a fruiting branch, and when a fruiting branch is detected, determining whether there is a bud, if a bud exists and is located on the fruiting branch, extracting the two-dimensional coordinates of the fruiting branch detection frame; if a bud exists and is not located on the fruiting branch, extracting the two-dimensional coordinates of the bud detection frame; Using the image registration method, the three-dimensional coordinates in the camera coordinate system are determined according to the depth information of the extracted detection frame center, and the three-dimensional coordinates in the world coordinate system are converted using the coordinate conversion formula. The three-dimensional coordinates in the world coordinate system are defined as the target coordinates, and the relative spatial relationship between the target coordinates and the actuator is calculated.

4. The method for accurately erasing complex grape buds using dual-stage visual positioning according to claim 1 is characterized in that: The local visual recognition algorithm adopts the SAM model, and the specific detection process is as follows: The image of the second field of view is input into the SAM model for recognition. First, old branches, single buds and secondary buds are distinguished. Then, the spatial relationship analysis and feature judgment of old branches and single buds are performed: if the mask of the single bud and the old branch overlap or the distance is less than the set threshold, it is determined to be a hidden bud; if the area of the remaining single bud is less than 30% of the average value, it is determined to be a weak bud; When there is only one hidden bud, weak bud or accessory bud, it is determined that there is a single complex bud. The two-dimensional coordinates of the corresponding bud are extracted and defined as the target bud coordinates. The pixel position relationship between the target bud coordinates and the laser action point is calculated, and the laser point is aligned with the target bud. When the number of latent buds, weak buds and accessory buds is greater than or equal to two, it is determined that there are multiple complex buds. The two-dimensional coordinates of each bud are extracted respectively, and each two-dimensional coordinate is defined as the target bud coordinate. The pixel position relationship between each target bud coordinate and the laser action point is calculated, and the path planning is performed according to the pixel position relationship. The processing order is determined, and the laser point is aligned with the target bud according to the processing order and the pixel position relationship between the target bud coordinate and the laser action point.

5. The method for accurately erasing complex grape buds using dual-stage visual positioning according to claim 1 is characterized in that: The specific process of aligning the laser point with the target bud is as follows: According to the pixel position relationship between the target bud coordinates and the laser action point, the pixel difference is calculated, and the fine-tuning angle of the laser module turntable is calculated according to the pixel difference to achieve the alignment of the laser point with the bud.

6. A system for accurately erasing complex grape buds using dual-stage visual positioning, employing the method according to any one of claims 1 to 5, characterized in that: include: Main controller, depth camera, industrial camera, crawler chassis, actuator and laser module; The main controller is respectively connected to the depth camera, the industrial camera, the crawler chassis, the actuator and the laser module, and the industrial camera is also connected to the actuator; the working process of the system is as follows: The crawler chassis drives the work platform to move along the direction of the grape branches to be tested. The instance segmentation model in the main controller is activated, the depth camera is aligned with the first field of view, and the global visual recognition algorithm is used to determine the recognition results in the first field of view: If the recognition result in the first field of view is that no fruiting branch is detected, continue moving; if the recognition result in the first field of view is that a fruiting branch is detected, stop moving and calculate the relative spatial relationship between the target coordinates and the actuator; Determine the horizontal rotation angle and pitch rotation angle of the actuator based on the relative spatial relationship, align the industrial camera with the second field of view, and use the local visual recognition algorithm to determine the recognition result in the second field of view: If the recognition result in the second field of view is that there is no complex bud, continue moving; if the recognition result in the second field of view is that there is a single complex bud, calculate the pixel position relationship between the target bud coordinates and the laser action point, and align the laser point with the target bud based on the pixel position relationship; if the recognition result in the second field of view is that there are multiple complex buds, perform path planning, determine the processing order, and align the laser point with the target bud based on the pixel position relationship between the target bud coordinates and the laser action point; the complex buds include hidden buds, weak buds, and accessory buds; The laser module is triggered to complete a single bud removal operation, and the main controller is instructed to re-run the instance segmentation model for detection until all buds in the first field of view and the second field of view have been removed. The actuator is reset, and the crawler chassis is used to drive the working platform to continue moving along the direction of the grape branch to be tested, and a new round of bud removal operation is carried out until all bud removal operations are completed.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for accurately erasing complex grape buds with dual-stage visual positioning according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The device stores a computer program, which, when executed by a processor, implements the method for accurately erasing complex grape buds using dual-stage visual positioning as described in any one of claims 1 to 5.

Images