A method and system for cooperative scanning of double robot arms in a vegetation-shielded environment

Abstract

The present application relates to the technical field of mechanical arm control, and particularly relates to a method and system for cooperative scanning of double mechanical arms in a vegetation-shielded environment. The present application collects images and a first depth point cloud of a local vegetation area when the double arms are stationary, generates an optimal clearing vector, and controls the first mechanical arm to complete the barrier clearing. A second depth point cloud, a real-time observation angle of the second mechanical arm, a reachable range, and a distance between the two mechanical arm ends are collected in real time, and a real-time incremental visible band is obtained in combination with a spatial residual map corresponding to the two sets of depth point clouds. Whether the scanning permission condition is met is determined according to the visible band, the observation angle of the second mechanical arm, the reachable range, and the distance between the two mechanical arm ends. When the condition is met, the second mechanical arm performs scanning work on the real-time incremental visible band until the scanning work of the entire local vegetation area is completed. The present application can effectively realize cooperative scanning of the double arms.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, and in particular to a dual-robotic arm collaborative scanning method and system in a vegetation-occupied environment. Background Technology

[0002] In agricultural informatization and intelligent operation technologies, vegetation target acquisition, structural observation, and local scanning are important means of assessing crop growth status and extracting phenotypic features. However, in actual operations, target vegetation is often obstructed by surrounding weeds, branches, leaves, or other local shading objects, resulting in a limited camera field of view and thus affecting the complete observation and information extraction of the target area. Vegetation scanning refers to the refined three-dimensional perception and phenotypic data acquisition operation for living crops in the field. It differs from two-dimensional acquisition methods such as ordinary panoramic photography and macroscopic aerial imaging, and is a close-range, high-precision three-dimensional scanning operation adapted to the extraction of microscopic growth features of crops.

[0003] Current crop vegetation observation generally employs a single robotic arm scanning mode with a pre-set fixed trajectory. This approach can only complete standardized scanning and basic modeling work according to a predetermined path, and its operational logic is simple and highly versatile. However, when faced with complex scenarios such as densely overlapping vegetation and partial shading by branches and leaves in the field, the single robotic arm has limited structural freedom and cannot actively adjust the observation angle or actively remove obstructions to create an effective observation window. The passive scanning method cannot avoid the problem of vegetation shading. Therefore, this approach is prone to generating large-area visual blind spots, ultimately leading to holes in the scanned point cloud and missing local structural information of the crop, failing to meet the requirements for high-precision, full-coverage crop phenotypic observation.

[0004] To address the limitations of single-arm scanning in actively avoiding obstacles and clearing occlusions, some research has attempted to transfer mature dual-arm collaborative robot technology to agricultural observation scenarios. However, current research and application of agricultural dual-arm collaborative systems focus on fruit and vegetable harvesting scenarios, lacking dedicated dual-arm collaborative solutions designed for crop occlusion scanning and phenotypic observation tasks. Traditional harvesting dual-arm systems employ a rigid "grab first, then cut" operational logic, lacking any compliant control strategies throughout, resulting in a single and highly rigid operational mode. Forcibly migrating this harvesting dual-arm system to the narrow-spaced, high-density crop scanning scenario in the field exposes serious adaptability problems: on the one hand, the compact growth space and minimal operational redundancy in the field make the rigid movements of the arms highly susceptible to physical interference and collisions that damage crops; on the other hand, the harvesting arms lack a collaborative operational logic of "obstacle removal assistance + precise scanning," failing to achieve efficient spatiotemporal coordination of one arm compliantly removing obstacles while the other scans synchronously, thus failing to fundamentally solve the problem of missing scanning information caused by vegetation occlusion and making it difficult to adapt to precise crop phenotypic observation tasks. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of existing dual-robotic arm collaborative scanning in the complex narrow-spacing environment of agriculture, where physical interference easily occurs between the two arms, and the inability to achieve efficient coordination between obstacle removal and scanning tasks in time and space, resulting in incomplete collection of vegetation observation information.

[0006] To address the aforementioned technical problems, this invention provides a dual-robotic arm collaborative scanning method for environments with vegetation obstruction, comprising: Based on the depth camera at the end of the dual robotic arms, the image of the local vegetation area and the first depth point cloud of the dual robotic arms in a stationary state are acquired, and the optimal clearing vector of the local vegetation area is generated, thereby controlling the first robotic arm to perform obstacle clearing operation on the local vegetation area. The second robotic arm is used to acquire the second depth point cloud of the local vegetation area in real time, the real-time observation angle and real-time reachable range of the second robotic arm, and the real-time distance between the end of the second robotic arm and the end of the first robotic arm. Based on the spatial residual map corresponding to the first depth point cloud and the second depth point cloud, the real-time incremental visual band is obtained. Based on the real-time incremental visual zone, the real-time observation angle and reachable range of the second robotic arm, and the real-time distance between the ends of the second and first robotic arms, it is determined whether the scanning permission conditions are met. If they are met, the second robotic arm is used to scan the real-time incremental visual zone. After scanning, the scanning quality of the real-time incremental visual zone is scored. Based on the score, the pose of the first and / or second robotic arms is adjusted until the score reaches the preset scoring threshold, and then the next local vegetation area is scanned.

[0007] Preferably, the method for scoring the scanning quality of the real-time incremental visual strip after scanning, and adjusting the pose of the first robotic arm and / or the second robotic arm based on the score, includes: If the score does not reach the preset score threshold, it is determined whether the resistance experienced by the first robotic arm exceeds the set safety threshold. If the score exceeds the limit, the first robotic arm's pose remains unchanged, and the second robotic arm's pose is adjusted using the next best viewpoint replanning mechanism, thereby supplementing the real-time incremental visual band with additional scanning until the score reaches the preset score threshold. If the deviation is not exceeded, the pose of the first robotic arm is adjusted according to the local micro-push offset, thereby controlling the first robotic arm to perform obstacle removal operation on the local vegetation area and reacquire the real-time incremental visual strip.

[0008] Preferably, the method for generating the optimal clearing vector for a local vegetation area includes: The first depth point cloud corresponding to the two robotic arms is preprocessed and registered respectively to obtain the registered first depth point cloud, and the effective point cloud belonging to the target vegetation and its occlusions is retained. Instance segmentation is performed on the image of a local vegetation area in a stationary state of dual robotic arms to obtain the target vegetation area and the occlusion area; The target vegetation area and the occlusion area are mapped to the effective point cloud, the category of each effective point cloud is obtained, and then the occlusion density, the visible density of the first target vegetation and the target density are statistically obtained and a first spatial residual map is constructed. Based on the first spatial residual map, the optimal clearing vector of the local vegetation area is generated.

[0009] Preferably, the method for generating the optimal clearing vector for a local vegetation area based on the first spatial residual map includes: Based on the valid point cloud belonging to the occluder, the main extension direction of the occluder is determined; based on the gradient of the first spatial residual map at the center of the occluder, the residual descent direction is obtained; based on the main extension direction of the occluder and the residual descent direction, the obstacle removal guidance direction is generated. The optimal clearing amplitude is solved with the optimization objective of maximizing the increase in visibility of the target vegetation and minimizing the collision risk cost and the target vegetation damage cost. The optimal clearing vector of the local vegetation area is constructed based on the optimal clearing amplitude and the obstacle clearing guidance direction.

[0010] Preferably, the collision risk cost formula is as follows: , in, As a consequence of collision risk, , , These are the weighting coefficients. To obtain the maximum value, and To preset a safe distance threshold, This indicates the overlap rate between the first robotic arm sweep envelope corresponding to the candidate deflection amplitude and the target vegetation and obstructions. For the target vegetation point set, For the occlusion point set, The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the target vegetation. The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the obstruction.

[0011] Preferably, the formula for the cost of damage to the target vegetation is: , in, The cost of damage to the target vegetation The first robotic arm end contact force corresponding to the amplitude of the candidate is used to separate the candidate. The local displacement of the target vegetation. To allow displacement threshold, , , These are the weighting coefficients. To allow contact force threshold, To determine the overlap rate of target vegetation areas corresponding to the amplitude values ​​of the candidates. This indicates taking the maximum value. It is an absolute value.

[0012] Preferably, the method for obtaining the real-time incremental viewband based on the spatial residual map corresponding to the first depth point cloud and the second depth point cloud includes: Based on the difference between the spatial residual maps corresponding to the first depth point cloud and the second depth point cloud, the residual increment map is obtained. The set of spatial locations in a local vegetation area where the residual increment is greater than a set residual threshold, is not scanned and collected by the second robotic arm, and is at a distance greater than a set safe distance threshold from the space occupied by the first robotic arm, is used as the real-time incremental visual zone.

[0013] Preferably, the method for determining whether the scanning permission conditions are met based on the real-time incremental visual band, the real-time observation angle and real-time reachable range of the second robotic arm, and the real-time distance between the ends of the second and first robotic arms includes: If the width of the real-time incremental visual band is greater than or equal to the width threshold, then it is determined whether the real-time observation angle of the second robotic arm does not exceed its maximum allowable range. If it does not exceed the maximum allowable range, then it is determined whether the real-time distance between the end of the second robotic arm and the end of the first robotic arm is greater than or equal to the set distance threshold. If it is greater than or equal to the set distance threshold, then it is determined whether the real-time incremental visual band is within the real-time reachable range of the second robotic arm. If it is, then it is determined that the permission conditions are met.

[0014] Preferably, the method for generating the optimal clearing vector for a local vegetation area, thereby controlling the first robotic arm to perform obstacle clearing operations on the local vegetation area, includes: The first robotic arm uses impedance control and compliant control technology to perform obstacle removal operations on local vegetation areas.

[0015] This invention also provides a dual-robotic arm collaborative scanning system for environments with vegetation obstruction, comprising: Mobile base; A first robotic arm and a second robotic arm are respectively mounted on a mobile base, and a depth camera is installed at the end of each robotic arm. The central controller is communicatively connected to the depth camera, obstacle-clearing actuator, first robotic arm, and second robotic arm, and is used to implement the steps of the above-mentioned dual-robotic arm collaborative scanning method in a vegetation-occupied environment.

[0016] Compared with the prior art, the above-described technical solution of the present invention has the following advantages: The present invention discloses a dual-robotic arm collaborative scanning method and system for vegetation-occupied environments. By comparing the first and second depth point clouds before and after obstacle removal, a corresponding spatial residual map is constructed, and the real-time incremental visible band is accurately solved. This allows for precise location of newly exposed and effectively observable vegetation areas after obstacle removal by the first robotic arm. It abandons the traditional blind full-area scanning mode, achieving dynamic incremental expansion of the observation area, fundamentally reducing vegetation-occupied blind spots, and solving the problem of incomplete vegetation information collection. Simultaneously, based on the real-time incremental visible band, and combined with the real-time observation angle, real-time reachable range, and real-time distance of the second robotic arm's end effector, the present invention comprehensively determines scanning permission conditions, initiating scanning only when spatial location, observation conditions, and safety distance all meet the requirements. This invention limits the scanning target to the local incremental area just released by the left arm, thereby reducing invalid scanning and repeated coverage, and improving the efficiency of data incremental acquisition in occluded scenarios. This mechanism can constrain the working space of the two arms in real time in complex agricultural operation scenarios with narrow spacing, effectively avoiding physical interference problems in the operation of the two robotic arms. At the same time, it constructs a spatiotemporal matching logic for obstacle removal and field of view expansion and permission for rescanning, realizing precise spatiotemporal coordination between obstacle removal and scanning tasks, and greatly improving the integrity and operational safety of vegetation observation data collection in complex farmland environments.

[0017] To address the issue that existing solutions often rely solely on single observation results to determine the obstacle removal direction and amplitude, neglecting to consider spatial distribution, robotic arm movement risks, and vegetation damage, problems arise in densely vegetated, narrowly spaced agricultural settings, such as unreasonable obstacle removal directions, robotic arm collisions, vegetation compression damage, or limited improvement in visibility after obstacle removal. This invention first combines point cloud registration and image instance segmentation to accurately distinguish target vegetation from obstructions. It then constructs a spatial residual map based on statistically obtained density parameters. Next, it determines the main extension direction by combining the obstruction's shape and the residual gradient to determine the residual descent direction, fusing both to obtain the obstacle removal guidance direction. This ensures the obstacle removal direction aligns with the obstruction's distribution characteristics, maximizing the release of the observation area. The visible density change corresponding to different candidate obstacle removal amplitudes quantifies the improvement in visibility. Simultaneously, a collision risk cost is constructed using a specialized calculation method, comprehensively considering the overlap and minimum spacing between the robotic arm's sweeping area and the target vegetation / obstructions, and applying corresponding weights for constraint, effectively avoiding physical collisions during operation. In addition, a separate target vegetation damage cost is constructed, which is combined with the contact force at the end of the robotic arm and the local displacement of the vegetation. Calculation constraints are carried out with reference to preset allowable thresholds for displacement and contact force to strictly control the deformation and crushing damage to the vegetation caused by the obstacle removal action. The optimal removal amplitude is solved with the comprehensive optimization objectives of improving visibility, reducing collision risk, and reducing vegetation damage. Then, the final vector is synthesized by combining the obstacle removal guidance direction, realizing dual fine-grained control of obstacle removal direction and amplitude, taking into account observation gain, operational safety, and vegetation protection. Attached Figure Description

[0018] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0019] Figure 1 This is a flowchart illustrating a dual-robotic arm collaborative scanning method for a vegetation-occupied environment according to the present invention.

[0020] Figure 2 This is a flowchart for obtaining the optimal deflection vector. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0022] Reference Figure 1 As shown, this embodiment provides a dual-robotic arm collaborative scanning method in a vegetation-covered environment, including: Based on the depth camera at the end of the dual robotic arms, the image of the local vegetation area and the first depth point cloud of the dual robotic arms in a stationary state are acquired, and the optimal clearing vector of the local vegetation area is generated, thereby controlling the first robotic arm to perform obstacle clearing operation on the local vegetation area. All operational steps in this invention are performed under a unified robot working coordinate system. After sensor initialization and coordinate registration are completed, depth cameras installed at the ends of the left and second robotic arms conduct synchronous observation of the target vegetation area. Since the center distance between the bases of the two robotic arms is 559mm, the two sets of end cameras can form complementary observation perspectives, effectively improving the perception ability of vegetation-covered areas. This step focuses on identifying the occlusion status and converting the occlusion information into the basis for obstacle removal action control, thus building a unified perception foundation for the subsequent obstacle removal operation of the first robotic arm and the scanning operation of the second robotic arm.

[0023] This invention employs a dual-robotic arm end-effector depth camera synchronous sampling method to simultaneously acquire depth maps and image information of the target vegetation area, thereby solving the problem of blind spots caused by occlusion in single-view observation. Synchronous sampling allows the left and right views to complement each other, thus improving the completeness of occlusion recognition.

[0024] In this embodiment, optionally, the first robotic arm is the left robotic arm and the second robotic arm is the right robotic arm.

[0025] In this embodiment, specifically, in the initial static state where the two robotic arms have not started obstacle removal and scanning operations, color images and depth maps of local vegetation areas are acquired by depth cameras at the ends of the two robotic arms, and the acquired depth maps are converted into a first depth point cloud to complete the acquisition of raw perception data.

[0026] like Figure 2 As shown, Figure 2 A flowchart for obtaining the optimal deflection vector.

[0027] In this embodiment, preferably, the method for generating the optimal clearing vector for a local vegetation area includes:

[0028] The first depth point cloud corresponding to the two robotic arms is preprocessed and registered respectively to obtain the registered first depth point cloud, and the effective point cloud belonging to the target vegetation and its occlusions is retained.

[0029] In this embodiment, specifically, the preprocessing of the first depth point cloud corresponding to the dual robotic arms includes: performing bidirectional filtering and dynamic denoising on the first depth point cloud to effectively filter out high-frequency noise and spatial outliers caused by slight leaf shaking and sensor acquisition errors in the field operation environment, optimize the overall quality of the point cloud, stabilize the local structural features and boundary information of vegetation and occlusions, and avoid noise interference with the accuracy of subsequent occlusion recognition and vector solution.

[0030] Next, coordinate transformation and multi-point cloud registration are performed. Using pre-calibrated camera parameters, the multi-view point cloud data collected by the depth camera at the end of the second robotic arm are uniformly transformed and mapped to the same robot base coordinate system. This completes the accurate registration of multi-source point clouds, ensuring that all subsequent calculation processes, such as occlusion relationship judgment, spatial residual calculation, and deflection vector solution, are based on a unified spatial reference benchmark, thus eliminating calculation errors caused by multi-camera viewpoint deviations.

[0031] Finally, the target environment was accurately extracted. The RANSAC algorithm was used to iteratively remove ground plane points and irrelevant background points from the point cloud data, completely removing non-target interference areas and retaining only the target vegetation area and the corresponding occlusions as effective point clouds. This accurately focused on the target area and provided clean and effective point cloud data support for subsequent instance segmentation, spatial density statistics, spatial residual map construction, and accurate generation of the optimal clearing vector.

[0032] Instance segmentation is performed on the image of a local vegetation area in a stationary state of dual robotic arms to obtain the target vegetation area and the occlusion area;

[0033] This invention uses SAM-2 to segment the target vegetation and the obstructing weeds, obtaining pixel-level or point-level separation results of the target vegetation area and the obstructing area. These segmentation results are used for subsequent residual analysis and obstacle removal decisions, rather than just for target recognition itself.

[0034] The target vegetation area and the shading area are mapped to the effective point cloud, the category of each effective point cloud is obtained, and then the shading density, the visible density of the first target vegetation and the target density are statistically obtained and a spatial residual map is constructed. Based on the spatial residual map, the optimal clearing vector of the local vegetation area is generated.

[0035] After separating the target vegetation from the blocking weeds, the present invention further calculates the spatial location of the blocking objects and constructs a spatial residual map to characterize areas in the target region that have not yet been fully exposed.

[0036] Based on the valid point cloud belonging to the occlusion, determine the main extension direction of the occlusion;

[0037] The set of valid point clouds belonging to the occlusion is taken as the occlusion point set. ,in, Then the spatial centroid of the obstruction can be represented as:

[0038] ,

[0039] in, The spatial centroid of the obstruction, The total number of valid point clouds within the occlusion point set. For the occlusion point concentration A valid point cloud's three-dimensional coordinate vector. For the occlusion point concentration The x-coordinate of a valid point cloud. For the occlusion point concentration The y-coordinate of a valid point cloud. For the occlusion point concentration The z-coordinate of a valid point cloud. Indicates transpose. For effective point cloud indexing, .

[0040] To obtain the main extension direction of the occlusion, the covariance matrix of the occlusion point set is further calculated:

[0041] ,

[0042] in, Let be the covariance matrix of the occluded object point set.

[0043] Pick The eigenvector corresponding to the largest eigenvalue After normalization, it is used as the extension direction of the occluding object, and the formula is: , To obscure the direction of the owner's extension, for The modulus length, this main direction can reflect the overall extension trend of the obstruction, and provide a geometric basis for the subsequent solution of the obstacle removal direction.

[0044] Simultaneously, the fused observation results from the two perspectives are projected onto a unified plane or a unified voxel space, defining the visible density of the first target vegetation as... The target density of the target vegetation is The density of the obstruction is Then the first space residual diagram It can be defined as: , is the weighting coefficient. This residual map is used to characterize areas where "the target should be visible but is not currently sufficiently visible." The higher the residual value, the more severe the occlusion of the local vegetation area, and the more necessary it is to remove obstacles to release the viewport.

[0045] Based on the gradient of the first spatial residual map at the occlusion matter center, the direction of residual descent is obtained. The formula for the gradient of the first spatial residual map at the occlusion matter center is: , in, This represents the gradient of the first-space residual map at the occluded matter center. This is the first space residual plot. , , These are the first-order partial derivatives of the first spatial residual plot along the X, Y, and Z axes, respectively.

[0046] The direction of the gradient at the occlusion center in the first-space residual plot represents the direction of residual increase, which is also the direction of most concentrated occlusion and the direction of residual decrease. The calculation formula is: , for The model length; combined with the extension direction of the occluded object. This can then form a more stable spatial judgment basis for obstacle-pushing actions.

[0047] Based on the extension direction of the occluded object and the descent direction of the residual, a barrier-clearing guidance direction is generated. To make the obstacle-clearing action calculable and executable, the obstacle-clearing guidance direction is... It is expressed as a weighted combination of the occlusion property extension direction and the residual descent direction, and the formula is: , in, , For directional fusion weights, and satisfying .

[0048] With the optimization objective of maximizing the increase in target vegetation visibility and minimizing the collision risk and target vegetation damage costs, the optimal clearing amplitude is solved. The optimal clearing vector for the local vegetation area is then constructed using the optimal clearing amplitude and the obstacle clearing guidance direction, as shown in the formula: , The optimal clearing vector for a local vegetated area. The optimal deflection amplitude.

[0049] The optimization objective is: , in, Indicates the candidate clearance amplitude. To remove the lower limit of the amplitude, To remove the upper limit of amplitude, , , These are the weighting coefficients. The amount by which the visibility of the target vegetation is increased. As a consequence of collision risk, The cost of damage to the target vegetation.

[0050] The improvement in the visibility of target vegetation is usually not measured directly, but is obtained through candidate pose simulation evaluation. The controller translates the occluders along the candidate directions in the virtual environment. Then, the visibility of the target vegetation (any one of visible points, visible area, or visible voxels) is recalculated, and the difference between the recalculated and original visibility is used to obtain the increase in the visibility of the target vegetation.

[0051] Collision risk cost is constructed based on the minimum distance and overlap rate between the first robotic arm sweep envelope corresponding to the candidate clearance amplitude and the target vegetation and obstructions, as shown in the formula: , in, As a consequence of collision risk, , , These are the weighting coefficients. To obtain the maximum value, and To preset a safe distance threshold, This indicates the overlap rate between the first robotic arm sweep envelope corresponding to the candidate deflection amplitude and the target vegetation and obstructions. For the target vegetation point set, For the occlusion point set, The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the target vegetation. The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the obstruction. .

[0052] The first robotic arm sweep envelope refers to the sweep envelope formed by the end effector and connecting rods of the first robotic arm, and the minimum distance between the first robotic arm sweep envelope corresponding to the candidate sweep amplitude and the target vegetation. , To separate the candidate values ​​from the first robotic arm sweep envelope corresponding to their amplitude, The coordinates of the first robotic arm sweep envelope corresponding to the amplitude of the candidate are determined. Given the coordinates of the target vegetation, the minimum distance between the first robotic arm sweep envelope corresponding to the candidate clearing amplitude and the obstruction. , The coordinates of the obstructing object.

[0053] The collision risk cost is calculated by combining the current point cloud model, the end-effector pose of the first robotic arm, the robotic arm envelope model, and the preset safety distance. This allows for the screening of candidate actions before obstacle removal, preventing the first robotic arm from entering areas that may cause contact, collision, or compression.

[0054] The damage cost of the target vegetation is constructed based on the overlap rate of the target vegetation area corresponding to the candidate clearing amplitude, the local displacement of the target vegetation, and the contact force at the end of the first robotic arm, as shown in the formula: , in, The cost of damage to the target vegetation The first robotic arm's end contact force corresponds to the candidate clearing amplitude. When the first robotic arm performs obstacle clearing operations on a local vegetation area using impedance control and compliant control technology, then... It can also be used to estimate the force for impedance control. The local displacement of the target vegetation. To allow displacement threshold, , , These are the weighting coefficients. To allow contact force threshold, This indicates taking the maximum value. For absolute values, The overlap rate of the target vegetation area corresponding to the candidate obstacle removal amplitude is defined as the spatial area traversed by the robotic arm's obstacle removal action at that amplitude. With the main area of ​​the target vegetation The ratio of the overlapping area to the area of ​​the spatial region traversed by the robotic arm's obstacle-clearing action at that amplitude.

[0055] The local displacement of the target vegetation refers to the change in geometric position of the local area of ​​the target vegetation caused by the first robotic arm at the candidate obstacle removal distance s. This displacement is not the overall movement of the entire vegetation, but rather the amplitude of the change in spatial position of local point sets such as main stems, branches or leaf clusters in a certain neighborhood near the obstacle removal contact point before and after obstacle removal.

[0056] Let the local points before and after the obstacle removal be respectively and Then the local displacement of the target vegetation can be expressed as: M represents the total number of local points.

[0057] The damage cost to the target vegetation is calculated using a combination of target overlap and force / displacement constraints. The main area of ​​the target vegetation (main stem, branches, or leaves) is defined as segmented from instances, denoted as... The contact or action area corresponding to the candidate deflection amplitude is The damage cost is determined by the target vegetation point cloud skeleton, branch and leaf distribution, barrier contact position, expected displacement, and end contact force or impedance control feedback. If the candidate action leads to increased overlap of the main area of ​​the target vegetation, excessive local displacement, or excessive contact force, the corresponding damage cost increases, thus enabling the priority selection of barrier amplitude and direction that cause less disturbance to the target vegetation.

[0058] Through the above solution method, the present invention does not simply provide an empirical obstacle-clearing direction, but incorporates "occlusion severity", "occlusion geometry", "target visibility improvement" and "safety cost" into the calculation to obtain the optimal obstacle-clearing vector pose that can directly guide the execution of the first robotic arm.

[0059] In this embodiment, preferably, the method for generating the optimal clearing vector for a local vegetation area, thereby controlling the first robotic arm to perform obstacle clearing operations on the local vegetation area, includes: The first robotic arm uses impedance control and compliant control technology to perform obstacle removal operations on local vegetation areas.

[0060] This invention outputs the optimal clearing vector and target contact point information of the local vegetation area to the impedance control module of the first robotic arm. The first robotic arm then executes a compliant and controllable obstacle clearing action according to the instruction, creating an observable window for subsequent scanning by the second robotic arm.

[0061] After completing multi-view perception and occlusion reasoning, this invention drives the first and second robotic arms to perform asymmetric cooperative control based on the output optimal obstacle-clearing vector. Since the two robotic arms are mounted on the same movable base, and the center distance between their bases is 559mm, there is a significant overlap in their workspace. Using ordinary independent control methods can easily lead to problems such as motion interference, window occlusion, or scanning path conflicts. Therefore, this invention sets the first robotic arm as an obstacle-clearing unit and the second robotic arm as a scanning unit, achieving continuous coordination of obstacle-clearing and scanning tasks through a locally recursive "obstacle-clearing—scanning—evaluation—re-obstacle-clearing" approach.

[0062] The collaborative process of this invention does not pre-calculate the complete global obstacle-clearing sequence and then execute it uniformly. Instead, it first calculates the optimal clearing vector for the current local obstruction block and executes a segment of local obstacle clearing. After the segment is completed, the next segment is recalculated based on the updated residual map and incremental visible band. That is, the entire control process adopts a recursive method of segment-by-segment update and segment-by-segment decision-making, thereby adapting to the characteristics of vegetation obstruction deformation, rebound, and dynamic changes in local viewpoints.

[0063] After receiving the current optimal removal vector, the first robotic arm enters the impedance control mode. This part can employ existing impedance control and force feedback compliant control technology. The basic principle is that when the robotic arm contacts obstructing weeds or branches, it does not use rigid force, but rather yields within a small range based on contact force, displacement error, and preset compliance parameters, keeping the obstacle removal process smooth, continuous, and controllable. Specifically, the first robotic arm first approaches the current local obstruction at a low speed; when the end effector detects a change in contact or force, the controller adjusts the end effector's attitude and force based on feedback signals, gradually removing the obstruction and forming a local visible window. The goal of this step is not to remove all obstructions at once, but to complete compliant, limited, and directional obstacle removal only in the current local area, creating conditions for subsequent local scanning by the second robotic arm. After completing this obstacle removal segment, the local residual map and visible boundary are immediately updated, and the next recursive calculation begins.

[0064] The second robotic arm, acting as a scanning unit, does not employ a fixed, pre-set scanning trajectory during the obstacle avoidance process of the first robotic arm. Instead, it performs scanning pose safely based on the current real-time second depth point cloud, the end-effector pose of the first robotic arm, the real-time incremental visual zone, and safety distance requirements. This part can be implemented using existing robotic arm obstacle avoidance control methods, such as collision detection based on distance thresholds, local trajectory fine-tuning, speed limiting, standby pose switching, or short-term retraction, to ensure that the second robotic arm does not interfere with the first robotic arm, obstructions, or other obstacles.

[0065] Specifically, the scanning trajectory of the second robotic arm is divided into several continuous path segments, and each path segment is judged segment by segment in relation to the space currently occupied by the first robotic arm and potential interference areas. When a certain trajectory segment is close to or has a risk of conflict with the first robotic arm's range of motion, the controller only corrects that local segment, including adjusting the position of the intermediate transition point, laterally shifting the scanning pose, fine-tuning the end effector posture, and redistributing the motion speed, so that the second robotic arm always performs scanning within the currently permitted visual window. It should be noted that the safe movement of the second robotic arm is only a guarantee at the execution layer; its actual scanning entry timing and scanning object are determined by the incremental visual zone permission mechanism in the next step.

[0066] The second robotic arm is used to acquire the second depth point cloud of the local vegetation area in real time, the real-time observation angle and real-time reachable range of the second robotic arm, and the real-time distance between the end of the second robotic arm and the end of the first robotic arm. Based on the spatial residual map corresponding to the first depth point cloud and the second depth point cloud, the real-time incremental visual band is obtained. In this embodiment, preferably, the method for obtaining the real-time incremental viewband based on the spatial residual map corresponding to the first depth point cloud and the second depth point cloud includes: Based on the difference between the spatial residual maps corresponding to the first depth point cloud and the second depth point cloud, the residual increment map is obtained, and the formula is: ,in, This is a residual increment plot. This is the spatial residual map corresponding to the first depth point cloud. This is the spatial residual map corresponding to the second depth point cloud. It represents the spatial position in a unified robot coordinate system.

[0067] The set of spatial locations in a local vegetation area where the residual increment is greater than a set residual threshold, is not scanned and collected by the second robotic arm, and is at a distance greater than a set safe distance threshold from the space occupied by the first robotic arm, is used as the real-time incremental visual zone.

[0068] The larger the value, the stronger the increased visibility gained at that location after the obstacle removal. Further, the set of currently uncollected points is defined as... The first robotic arm occupies space of The real-time incremental visual band can then be represented as: , in, This is a localized vegetated area. The residual threshold, This is the safety distance threshold for the first robotic arm. This definition indicates that only areas that simultaneously meet the criteria of "significantly reduced residuals, not yet acquired, and maintaining a safe distance from the first robotic arm" are considered valid incremental view zones.

[0069] In terms of information sources, incremental visual bands are typically determined by the following four types of information: Changes in the visible boundaries of point clouds before and after obstacle removal; The region in the spatial residual plot where the residual decreases most significantly; The new visible area formed by the expansion of the target vegetation boundary after instance segmentation; The missing point cloud area that has not yet been captured from the current viewpoint of the second robotic arm.

[0070] In this invention, whether the second robotic arm enters the scanning process is not determined by a fixed trajectory or simple obstacle avoidance results, but by the incremental visible zone formed after the first robotic arm removes the obstacle. Here, the incremental visible zone refers to the newly exposed, yet-to-be-collected, observable area of ​​the target vegetation after one obstacle removal action.

[0071] Step S3: Based on the real-time incremental visual band, the real-time observation angle and reachable range of the second robotic arm, and the real-time distance between the ends of the second and first robotic arms, determine whether the scanning permission conditions are met. If not, the second robotic arm remains in a waiting or returns to a safe position, and the first robotic arm continues to fine-tune to further expand the visual window. If the conditions are met, the second robotic arm is used to scan the real-time incremental visual band. After scanning, the scanning quality of the real-time incremental visual band is scored. Based on the score, the positions of the first and / or second robotic arms are adjusted until the score reaches a preset scoring threshold. The next local vegetation area is then scanned until the entire vegetation area is scanned.

[0072] In this embodiment, specifically, after obtaining the scan pose corresponding to the second robotic arm, the second robotic arm is driven to move to the spatial coordinates corresponding to the pose, and the depth camera installed on the second robotic arm is adjusted to the angle corresponding to the pose according to the scan target pose, and the scanning operation is performed after it is in place.

[0073] After each obstacle removal, differential analysis is performed on the point cloud and residual map before and after the obstacle removal to identify the newly released visible area, which is then mapped to an incremental visible band in a unified robot coordinate system. Subsequently, based on the visible band width, visible angle, safe distance from the end effector of the first robotic arm, local residual value, and the accessibility of the second robotic arm, it is determined whether the area meets the scanning permission conditions.

[0074] Determine if the width of the real-time incremental viewband is greater than or equal to the width threshold. If the value is greater than or equal to the value, then determine whether the real-time observation angle of the second robotic arm has not exceeded its maximum allowable range. If the distance is not exceeded, then determine whether the real-time distance between the end effector of the second robotic arm and the end effector of the first robotic arm is greater than or equal to the set distance threshold. If it is greater than or equal to, then determine whether the real-time incremental visual strip is within the real-time reachable range of the second robotic arm. If it is located, then the permission conditions are met; in, The width of the real-time incremental viewband. For width threshold, This is the real-time observation angle for the second robotic arm. This is the minimum value of the maximum allowable range of the observation angle. The maximum value of the maximum allowable range of the observation angle. This represents the real-time distance between the end effector of the second robotic arm and the end effector of the first robotic arm. To set a distance threshold, This indicates whether the real-time incremental visual strip is within the real-time reachable range of the second robotic arm. If it is equal to 1, it means that it is within the range; if it is 0, it means that it is not within the range.

[0075] Currently, most data collection follows an open-loop model, lacking a real-time scanning quality assessment mechanism. If missed scans or data loss due to occlusion occur, automatic identification and triggering of re-scanning and replanning are not possible, making it difficult to guarantee the completeness of vegetation growth index quantification.

[0076] The asymmetric cooperative control of this invention further constitutes a closed loop: a segment is opened, a segment is scanned, a segment is evaluated, and then the next segment is determined. Its principle is as follows: First, the first robotic arm performs local obstacle removal according to the optimal obstacle removal vector and releases the current visible window after a brief pause; the controller simultaneously updates the residual map and the incremental visible strip. Let the output state of the first robotic arm's current obstacle removal action be... Then the real-time incremental visual band An update can be represented as: , in, This represents a mapping function that updates the real-time incremental visual strip based on the obstacle removal results. This is the spatial residual map corresponding to the first depth point cloud. This is the spatial residual map corresponding to the second depth point cloud.

[0077] Secondly, the second robotic arm only performs local scanning on the permitted real-time incremental visual strip. During the scanning process, existing point cloud sampling and trajectory tracking methods can be used to acquire local geometric information in real time. Let the real-time second depth point cloud obtained by the second robotic arm be... The reference point cloud or expected point cloud for local vegetation areas is The scanning quality of the real-time incremental visual strip can be evaluated using the following metrics: Point cloud coverage : , This indicates the degree of coverage of the target reference area by the collected point cloud.

[0078] Point cloud density : , This indicates the effective sampling density within a local vegetation area.

[0079] Reconstruction integrity : , in, This represents the set of points that are still missing. The larger the value, the more complete the reconstruction.

[0080] Information entropy : , in, The percentage of point cloud in the w-th spatial distribution unit is used to reflect the richness of spatial information in the current scan results. This represents the total number of spatially distributed units.

[0081] Scan quality score of real-time incremental visual strip for: , in, For weighting coefficients. When At that time, it is considered that the real-time incremental visual band has been fully collected; when This indicates that the real-time incremental visual band still has local missing parts, residual occlusion, or the visible boundary has not been fully expanded.

[0082] In this embodiment, specifically, the method for scoring the scanning quality of the real-time incremental visual band after scanning, and adjusting the pose of the first robotic arm and / or the second robotic arm based on the score, includes: If the score does not reach the preset score threshold, it is determined whether the resistance experienced by the first robotic arm exceeds the set safety threshold. If the score exceeds the threshold, the first robotic arm's pose remains unchanged, and the second robotic arm's pose is adjusted using the Next Best View (NBV) replanning mechanism to supplement the real-time incremental visual band until the score reaches the preset score threshold. If the deviation is not exceeded, the pose of the first robotic arm is adjusted according to the local micro-push offset, thereby controlling the first robotic arm to perform obstacle removal operation on the local vegetation area and reacquire the real-time incremental visual strip; the local micro-push offset is a set value.

[0083] If the scan quality score does not reach the preset score threshold, indicating that there are still local missing parts, residual occlusion, or the visible boundary has not been fully expanded, the first robotic arm is fed back to perform further fine-tuning or attitude adjustment to expand the next segment of the real-time incremental view. If the second robotic arm finds that there are still blind spots in the real-time incremental view, it performs local supplementary scanning instead of a full rescan. The next control strategy can be written as follows: , in, The next step is to develop a control strategy for the first robotic arm. This indicates the local micro-adjustment control amount for the next segment of the first robotic arm. This indicates the control amount for the next stage of supplementary sweeping by the second robotic arm. This indicates the end control variable for completing the task in the current local area.

[0084] Therefore, this invention does not perform a unified supplementary scan after a one-time scan of the entire area, but instead couples the obstacle-clearing action with the scanning action into a local incremental closed-loop process. The core innovation of this process lies in the fact that the obstacle-clearing action of the first robotic arm directly determines the permitted scanning area of ​​the second robotic arm, and the scanning result of the second robotic arm in turn drives the next round of obstacle-clearing decisions, thereby achieving continuous collaboration oriented towards the viewing window.

[0085] Through the aforementioned asymmetric cooperative control, this invention enables stable cooperation between one robotic arm for obstacle removal and the other for scanning, within a compact structure with a center-to-center distance of only 559 mm between the bases of the two robotic arms. This mechanism not only reduces motion interference between the robotic arms but also allows obstacle removal and scanning to form a continuous linkage, improving observation efficiency, scanning completeness, and operational safety in occluded environments.

[0086] After the first robotic arm removes obstacles and establishes an effective observation window, the second robotic arm performs a precise scan of the target vegetation. The second robotic arm moves along a preset scanning trajectory or a locally adaptive trajectory, continuously acquiring point cloud data at a second depth, and constructing an incremental 3D point cloud model in real time during the scanning process. The purpose of this process is to obtain the geometric morphology information of the target vegetation as completely as possible after the occlusion is partially released, providing basic data for subsequent phenotypic extraction and quantitative analysis.

[0087] Because vegetation scenes are characterized by complex occlusion relationships, irregular local structures, and strong viewpoint dependence, a single scan often cannot completely cover all areas. Therefore, this invention further introduces a closed-loop scanning mechanism based on quality feedback. After the second robotic arm completes a scan, it evaluates the quality of the current scan results to determine whether there are visual holes, missing point clouds, or incomplete local reconstructions.

[0088] Specifically, the following indicators can be comprehensively evaluated: point cloud coverage, point cloud density, reconstruction completeness, and information entropy value. Point cloud coverage reflects the proportion of the currently scanned area that has been effectively acquired; point cloud density measures whether the local point cloud is sufficiently dense; reconstruction completeness determines whether the main target vegetation has been fully reconstructed; and entropy value characterizes whether the information distribution in the current observation results is balanced and whether there are still obvious unobserved areas. When any one or more of the above indicators fails to reach a preset threshold, the current scan is determined to be incomplete, and the supplementary scanning phase begins.

[0089] During the rescanning phase, this invention employs a next-best-viewpoint replanning mechanism. Based on the current incremental 3D point cloud model, occlusion status, hole locations, and the reachability of the robotic arm, the next observation viewpoint or local rescanning pose is recalculated. The core purpose of NBV is not simply to repeat the scan, but to select a viewpoint that maximizes the addition of new information and minimizes redundant coverage, allowing the second robotic arm to prioritize supplementing areas that have not yet been observed. In implementation, the NBV solution process can comprehensively evaluate the missing areas, visible areas, and candidate viewpoint sets in the current point cloud model, selecting the viewpoint with the highest information gain as the next scan position. If there is a linkage requirement between the rescanning area and the obstacle removal window of the first robotic arm, a pose fine-tuning command can also be sent to the first robotic arm simultaneously to readjust the position of the occluder, further expanding the effective observation window of the second robotic arm. During the rescanning process, a closed-loop control flow of "scanning—evaluation—replanning—rescanning—re-evaluation" is formed. That is, the second robotic arm does not immediately end after completing a scan, but first determines whether there is any omission; if there is an omission, NBV recalculation is automatically triggered and rescanning is performed until the scan quality meets the set requirements. This can significantly reduce point cloud holes caused by occlusion, limited viewing angle, or local missed scans, and improve the integrity and reliability of the target vegetation reconstruction results.

[0090] This invention incorporates this process into a global state machine. When insufficient scan coverage, inadequate reconstruction completeness, or excessively large point cloud holes are detected, the state machine transitions from the "scanning and reconstruction" state to the "NBV supplementary scan" state. Once the supplementary scan is completed and the quality assessment meets the standards, it returns to the data output state, entering the subsequent vegetation index calculation and result fusion stage. Through this closed-loop scanning and supplementary scan mechanism, this invention can continuously improve scan coverage and data completeness under conditions of complex vegetation occlusion and limited observation windows, avoiding common problems in open-loop scanning such as missed scans, holes, and local distortions, thereby ensuring the accuracy of subsequent vegetation structure analysis and quantitative assessment.

[0091] Existing dual-robotic arm vegetation collaborative scanning technology suffers from several technical shortcomings that urgently need to be addressed. Firstly, its occlusion recognition capability is weak, making it difficult to accurately determine the complex spatial distribution and occlusion relationships between target vegetation and obstructing weeds, and failing to effectively distinguish between effective observation areas and blind spots. Secondly, obstacle removal and scanning tasks are disconnected, resulting in poor spatiotemporal coordination. Vegetation observation cannot be completed simultaneously during obstacle removal operations, and in complex working environments with narrow, overlapping spaces in farmland, the collaborative constraint capability of the dual robotic arms is insufficient, easily leading to motion conflicts and physical interference problems. Thirdly, the system... The mobile base is prone to pitch and high-frequency vibration when traveling on uneven field surfaces. This error, amplified by the robotic arm structure, will seriously reduce the point cloud acquisition accuracy and spatial registration effect of the end camera. Fourth, the scanning operation lacks a real-time quality assessment and closed-loop correction mechanism, and cannot dynamically detect and supplement the scanning blind area, point cloud holes and insufficiently unfolded visible boundaries. Ultimately, the vegetation 3D reconstruction results have a large number of missing parts and insufficient integrity, which makes it difficult to meet the operational needs of high-precision vegetation observation in complex farmland shading environments.

[0092] The present invention aims to solve the problems in the prior art such as inaccurate identification of vegetation obstruction, asynchronous obstacle removal and scanning actions, unsmooth coordination of the two arms, incomplete scanning results, and the impact of the movement of the base on the scanning accuracy.

[0093] During the collaborative scanning process, this invention first reads the base IMU and odometry data in real time to identify vibration disturbances in the pitch, roll, and yaw directions; then, it calculates the end-effector compensation amount through inverse kinematics algorithm to correct the pose in real time to counteract the base sway; subsequently, it performs multimodal fusion on the compensated point cloud to construct voxelized spatial-spectral features, and finally calculates quantitative data such as vegetation cover (FVC), plant height, and canopy volume.

[0094] This invention enables highly adaptable asymmetric operations, allowing for a division of labor where one arm is responsible for smooth obstacle removal and the other for precise scanning. It supports removing obstacles before scanning or removing obstacles while scanning, significantly improving operational efficiency and smoothness in overlapping and occluded environments.

[0095] This invention enables high-completeness point cloud reconstruction: based on spatial relationship reasoning and NBV quality feedback supplementation mechanism, it effectively eliminates visual blind spots and scanning holes, and significantly improves the observation completeness and scanning coverage of target vegetation.

[0096] This invention enables high-precision mobile stabilization scanning: it innovatively introduces a base disturbance recognition and feedforward compensation closed loop, which effectively counteracts attitude drift caused by driving on uneven ground and ensures the accuracy of point cloud acquisition in motion.

[0097] This invention enables precise quantification of vegetation phenotypes: through multi-source heterogeneous data fusion and multi-layer geometric analysis, it can stably output high-resolution results of vegetation cover and growth indicators.

[0098] This second embodiment provides a dual-robotic arm collaborative scanning system for environments with vegetation obstruction, including: Mobile base; A first robotic arm and a second robotic arm are respectively mounted on a mobile base, and a depth camera is installed at the end of each robotic arm. The central controller, which is communicatively connected to the depth camera, the first robotic arm, and the second robotic arm, is used to implement the steps of the above-mentioned dual-robotic arm collaborative scanning method in a vegetation-occupied environment.

[0099] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0101] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0103] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A dual-robotic arm collaborative scanning method for vegetation-occluded environments, characterized in that, include: Based on the depth camera at the end of the dual robotic arms, the image of the local vegetation area and the first depth point cloud of the dual robotic arms in a stationary state are acquired, and the optimal clearing vector of the local vegetation area is generated, thereby controlling the first robotic arm to perform obstacle clearing operation on the local vegetation area. The second robotic arm is used to acquire the second depth point cloud of the local vegetation area in real time, the real-time observation angle and real-time reachable range of the second robotic arm, and the real-time distance between the end of the second robotic arm and the end of the first robotic arm. Based on the spatial residual map corresponding to the first depth point cloud and the second depth point cloud, the real-time incremental visual band is obtained. Based on the real-time incremental visual zone, the real-time observation angle and reachable range of the second robotic arm, and the real-time distance between the ends of the second and first robotic arms, it is determined whether the scanning permission conditions are met. If they are met, the second robotic arm is used to scan the real-time incremental visual zone. After scanning, the scanning quality of the real-time incremental visual zone is scored. Based on the score, the pose of the first and / or second robotic arms is adjusted until the score reaches the preset scoring threshold, and then the next local vegetation area is scanned.

2. The dual-robotic arm collaborative scanning method under vegetation-occupied environment according to claim 1, characterized in that, The method for scoring the scanning quality of the real-time incremental visual strip after scanning, and adjusting the pose of the first robotic arm and / or the second robotic arm based on the score, includes: If the score does not reach the preset score threshold, it is determined whether the resistance experienced by the first robotic arm exceeds the set safety threshold. If the score exceeds the limit, the first robotic arm's pose remains unchanged, and the second robotic arm's pose is adjusted using the next best viewpoint replanning mechanism, thereby supplementing the real-time incremental visual band with additional scanning until the score reaches the preset score threshold. If the deviation is not exceeded, the pose of the first robotic arm is adjusted according to the local micro-push offset, thereby controlling the first robotic arm to perform obstacle removal operation on the local vegetation area and reacquire the real-time incremental visual strip.

3. The dual-robotic arm collaborative scanning method under vegetation occlusion environment according to claim 1, characterized in that, Methods for generating the optimal clearing vector for local vegetation areas include: The first depth point cloud corresponding to the two robotic arms is preprocessed and registered respectively to obtain the registered first depth point cloud, and the effective point cloud belonging to the target vegetation and its occlusions is retained. Instance segmentation is performed on the image of a local vegetation area in a stationary state of dual robotic arms to obtain the target vegetation area and the occlusion area; The target vegetation area and the occlusion area are mapped to the effective point cloud, the category of each effective point cloud is obtained, and then the occlusion density, the visible density of the first target vegetation and the target density are statistically obtained and a first spatial residual map is constructed. Based on the first spatial residual map, the optimal clearing vector of the local vegetation area is generated.

4. The dual-robotic arm collaborative scanning method under vegetation obstruction environment according to claim 3, characterized in that, Methods for generating the optimal clearing vector for local vegetation regions based on the first spatial residual map include: Based on the valid point cloud belonging to the occluder, the main extension direction of the occluder is determined; based on the gradient of the first spatial residual map at the center of the occluder, the residual descent direction is obtained; based on the main extension direction of the occluder and the residual descent direction, the obstacle removal guidance direction is generated. The optimal clearing amplitude is solved with the optimization objective of maximizing the increase in visibility of the target vegetation and minimizing the collision risk cost and the target vegetation damage cost. The optimal clearing vector of the local vegetation area is constructed based on the optimal clearing amplitude and the obstacle clearing guidance direction.

5. The dual-robotic arm collaborative scanning method under vegetation occlusion environment according to claim 4, characterized in that, The formula for the cost of collision risk is: ， in, As a consequence of collision risk, , , These are the weighting coefficients. To obtain the maximum value, and To preset a safe distance threshold, This indicates the overlap rate between the first robotic arm sweep envelope corresponding to the candidate deflection amplitude and the target vegetation and obstructions. For the target vegetation point set, For the occlusion point set, The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the target vegetation. The minimum distance between the first robotic arm sweep envelope corresponding to the amplitude of the candidate and the obstruction.

6. The dual-robotic arm collaborative scanning method under vegetation-occupied environment according to claim 4, characterized in that, The formula for calculating the cost of damage to the target vegetation is: ， in, The cost of damage to the target vegetation The first robotic arm end contact force corresponding to the amplitude of the candidate is used to separate the candidate. The local displacement of the target vegetation. To allow displacement threshold, , , These are the weighting coefficients. To allow contact force threshold, To determine the overlap rate of target vegetation areas corresponding to the amplitude values ​​of the candidates. This indicates taking the maximum value. It is an absolute value.

7. The dual-robotic arm collaborative scanning method under vegetation-occupied environment according to claim 1, characterized in that, Methods for obtaining real-time incremental viewbands based on the spatial residual maps corresponding to the first depth point cloud and the second depth point cloud include: Based on the difference between the spatial residual maps corresponding to the first depth point cloud and the second depth point cloud, the residual increment map is obtained. The set of spatial locations in a local vegetation area where the residual increment is greater than a set residual threshold, is not scanned and collected by the second robotic arm, and is at a distance greater than a set safe distance threshold from the space occupied by the first robotic arm, is used as the real-time incremental visual zone.

8. The dual-robotic arm collaborative scanning method under vegetation-occupied environment according to claim 1, characterized in that, The methods for determining whether scanning permission conditions are met, based on the real-time incremental visual strip, the real-time observation angle and real-time reachable range of the second robotic arm, and the real-time distance between the ends of the second and first robotic arms, include: If the width of the real-time incremental visual band is greater than or equal to the width threshold, then it is determined whether the real-time observation angle of the second robotic arm does not exceed its maximum allowable range. If it does not exceed the maximum allowable range, then it is determined whether the real-time distance between the end of the second robotic arm and the end of the first robotic arm is greater than or equal to the set distance threshold. If it is greater than or equal to the set distance threshold, then it is determined whether the real-time incremental visual band is within the real-time reachable range of the second robotic arm. If it is, then it is determined that the permission conditions are met.

9. The dual-robotic arm collaborative scanning method under vegetation-occupied environment according to claim 1, characterized in that, The method for generating the optimal clearing vector for a local vegetated area, thereby controlling the first robotic arm to perform obstacle clearing operations on the local vegetated area, includes: The first robotic arm uses impedance control and compliant control technology to perform obstacle removal operations on local vegetation areas.

10. A dual-robotic arm collaborative scanning system for vegetation-shaded environments, characterized in that, include: Mobile base; A first robotic arm and a second robotic arm are respectively mounted on a mobile base, and a depth camera is installed at the end of each robotic arm. A central controller, which is communicatively connected to a depth camera, a first robotic arm, and a second robotic arm, is used to implement the steps of a dual-robotic arm collaborative scanning method in a vegetation-occupied environment as described in any one of claims 1-9.

Images