Impedance model-based edelweiss picking force potential hybrid soft adjustment method and system

By employing a hybrid and compliant adjustment method based on impedance model for picking feathers from bird's nests, the deformation characteristics and geometric features of feathers in bird's nests are obtained, initial impedance parameters are constructed, and damping parameters are updated by combining microscopic image detection. This solves the problems of unstable feather removal and insufficient protection of bird's nest strands in existing technologies, and achieves a more stable feather picking process.

CN123069327APending Publication Date: 2026-11-10北京和益源生物技术有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611445351.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-17
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing automatic feather removal methods struggle to balance stable removal of stray feathers with protection of bird's nest strands. Fixed impedance parameters are ill-suited to the differences between the geometric information of stray feathers and the local mechanical state of the bird's nest, leading to unstable generation of control parameters.

Method used

The method for force-position hybrid compliance adjustment based on impedance model of bird's nest feather picking constructs initial impedance parameters by acquiring the deformation characteristics, geometric characteristics of miscellaneous feathers and equivalent stiffness distribution of bird's nest, and updates the damping parameters by combining microscopic image detection, so as to achieve force-position hybrid compliance control.

Benefits of technology

It improves the stability of feather removal and the protection of bird's nest silk, reduces the risk of bird's nest silk being pulled or damaged, and optimizes the generation of control parameters for the feather removal process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123069327A_ABST
    Figure CN123069327A_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of robot compliance control and bird's nest intelligent processing, and discloses a bird's nest hair picking force-position hybrid compliance adjustment method and system based on an impedance model, which comprises the following steps: obtaining a color image, a depth point cloud, a structured light pattern before and after micro-displacement loading and a force value of a bird's nest area to be picked, and calculating a local deformation field; analyzing an embedded position, an extension direction, a visible length and an overlapping degree of a target hair, and inverting a local equivalent stiffness; taking the hair as a graph node to generate initial impedance parameters; performing force-position hybrid picking based on the extension direction of the hair, and dynamically updating damping parameters according to the separation state of the target hair relative to the bird's nest matrix and the pulling state of the silk. The present application can improve the stability of hair removal and reduce the risk of synchronous pulling or damage of the silk.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of robot compliance control and intelligent bird's nest processing technology, specifically to a method and system for adjusting the mixed compliance of bird's nest picking force based on an impedance model. Background Technology

[0002] During the manual or automatic feather removal process of bird's nest raw materials, it is necessary to identify and remove fine impurities from the interwoven network of bird's nest strands. Impurities are usually attached to, cross, or partially embedded in the surrounding bird's nest strands. The direction of extension, visible length, degree of overlap, and density of bird's nest strands in different areas of impurities vary, resulting in significant variations in the force and displacement adjustment required during the removal process.

[0003] Existing automatic feather removal methods typically rely on preset clamping positions, movement trajectories, or fixed control parameters to perform the removal. When the target feathers overlap deeply with the bird's nest strands or the local strand structure is relatively dense, fixed impedance parameters are insufficient to simultaneously ensure stable feather removal and bird's nest strand protection. Furthermore, relying solely on visual position or single force feedback for control makes it difficult to continuously transmit the geometric characteristics of the target feathers, the local mechanical state of the bird's nest, and the dynamic removal state during the removal process to the control parameters.

[0004] Therefore, this application provides a method and system for adjusting the mixing and softening of the force used to pick feathers in bird's nest based on an impedance model to solve the above-mentioned technical problems. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for adjusting the force and flexibility of bird's nest picking based on an impedance model, in order to solve the technical problem in the prior art where the picking control parameters are fixed and the geometric information of the impurities and the local mechanical state of the bird's nest are difficult to participate in the generation of control parameters, resulting in the inability to simultaneously ensure the stability of impurity removal and the protection of bird's nest silk.

[0006] To address the aforementioned technical problems, this invention provides a method for adjusting the force and smoothness of bird's nest picking based on an impedance model, comprising: Acquire color images, depth point clouds, and structured light patterns and force values ​​of the area to be cleaned in the bird's nest before and after micro-displacement loading, and calculate the deformation characteristics of the bird's nest. Instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow hairs; The deformation features and the force values ​​are input into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and the local equivalent stiffness values ​​of the target hair embedding positions are extracted. Using stray hairs as graph nodes, node features are constructed based on the geometric features, the degree of overlap, and the equivalent stiffness distribution. Initial impedance parameters are generated through a graph neural network. A task coordinate system is established based on the extension direction of the target stray hairs. The position correction amount of the picking direction is calculated using the initial impedance parameters, and the lateral position servo is executed. A microscopic image of the area where the target stray hair overlaps with the surrounding filaments is acquired. Based on the microscopic image, the initiation event of the target stray hair extraction and the filament traction state are detected. According to the detection results, the rate of change of the picking force error, the optical flow characteristics of the microscopic image, the filament traction probability, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction.

[0007] In some specific embodiments, acquiring color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, and calculating the deformation characteristics of the bird's nest, further includes: Acquire color images and depth point clouds of the area to be cleaned from the bird's nest, as well as structured light patterns and force values ​​before and after applying a preset micro-displacement to the surface of the bird's nest. The micro-displacement loading head is placed against the surface of the bird's nest in the vicinity of the target hairy area and the preset micro-displacement is applied. A speckle pattern is projected onto the surface of the bird's nest, and the speckle images before and after the micro-displacement is applied, as well as the force value when the preset micro-displacement is applied, are recorded. A pixel-by-pixel matching calculation is performed on the speckle image to obtain the local displacement field of the bird's nest surface, and the local deformation field of the bird's nest is calculated based on the local displacement field, and the local deformation field of the bird's nest is used as the deformation feature.

[0008] In some specific embodiments, instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow's filaments, further including: The color image and the depth point cloud are input into the stray hair instance segmentation network, and the stray hair instance segmentation network outputs stray hair mask, swallow hair mask and embedding position in the color image based on the stray hair instance segmentation network. A three-dimensional point set is extracted from the depth point cloud that coincides with the hair mask. A principal direction analysis is performed on the three-dimensional point set to obtain the extension direction vector of the target hair in three-dimensional space. The visible length is calculated based on the projection range of the three-dimensional point set along the extension direction vector. The degree of overlap is calculated based on the ratio of the number of overlapping pixels between the stray hair mask and the swallow silk mask to the number of pixels in the stray hair mask; The geometric features include the embedding location, the extension direction vector, and the visible length.

[0009] In some specific embodiments, the deformation features and the force values ​​are input into a stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and the local equivalent stiffness values ​​at the target hair embedding locations are extracted, further including: The local deformation field of the bird's nest and the force value are input into the stiffness inversion network to obtain a spatial equivalent stiffness distribution map; The two-dimensional image of the local deformation field of the bird's nest is spliced ​​with the force value to form a multi-channel tensor, which is then input into the stiffness inversion network of the encoder and decoder structure to generate a spatial equivalent stiffness distribution map of the same size as the two-dimensional image of the local deformation field of the bird's nest. Read the effective pixel values ​​of the embedding position and the preset neighborhood of the embedding position in the spatial equivalent stiffness distribution map, and determine the local equivalent stiffness value based on the effective pixel values; The equivalent stiffness distribution includes the spatial equivalent stiffness distribution map, and the deformation characteristics include the local deformation field of the bird's nest.

[0010] In some specific embodiments, using stray hairs as graph nodes, node features are constructed based on the geometric features, the degree of overlap, and the equivalent stiffness distribution. Initial impedance parameters are generated through a graph neural network, further including: Using stray hairs as graph nodes, and the extension direction vector, the local equivalent stiffness value, the visible length, and the degree of overlap as node features, initial impedance parameters are generated through message passing based on a graph neural network. Triangulation is performed based on the embedding position of each stray hair, and connecting edges with a length greater than a preset threshold are deleted. The remaining connecting edges form the connection relationship between graph nodes. The Euclidean distance between graph nodes and the change in the density of the swallow's thread on the connecting path are used as edge features. The message weight is calculated using the node features and the edge features, and the geometric and mechanical information of neighboring graph nodes is aggregated. The desired damping and desired stiffness in the picking direction are output, as well as the tangential compliance center stiffness and tangential damping as the compliance center adjustment parameters perpendicular to the picking direction. The initial impedance parameters include the desired damping and desired stiffness in the picking direction, as well as the tangential compliance center stiffness and tangential damping.

[0011] In some specific embodiments, a task coordinate system is established based on the extension direction of the target stray hairs, the position correction amount of the picking direction is calculated using the initial impedance parameters, and lateral position servoing is performed, further including: Move to the target stray hairs and establish a task coordinate system based on the extended direction vector; Establish the picking direction coordinate axis with the extended direction vector, obtain the actual force in the picking direction, and calculate the picking force error between the actual force and the expected picking force; substitute the fixed expected inertia, the expected damping and expected stiffness in the picking direction in the initial impedance parameters into the time-varying impedance relationship, calculate the position correction amount according to the backward Euler recursion, and superimpose the position correction amount onto the expected position of the robot position loop; The lateral force in the plane perpendicular to the picking direction is obtained. An admittance relationship is established with the tangential compliance center stiffness and tangential damping in the compliance center adjustment parameters. The lateral position correction is calculated based on the lateral force and input into the robot position loop. The lateral position correction is updated in each position control cycle.

[0012] In some specific embodiments, the extraction initiation event of the target stray hairs and the filament pulling state are detected based on the microscopic image. According to the detection results, the rate of change of the picking force error, the optical flow characteristics of the microscopic image, the filament pulling probability, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction. This further includes: A microscopic image of the area where the target feathers overlap with the surrounding bird's nest fibers is acquired. The microscopic image is then input into a separation state detection network to detect the separation initiation event of the target feathers relative to the bird's nest matrix and the traction state of the bird's nest fibers. When the separation initiation event or the traction state of the bird's nest fibers is detected, the picking force error change rate, the optical flow characteristics of the microscopic image, the traction probability of the bird's nest fibers, and the damping value in the initial impedance parameters are input into a separation adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction. The extraction state detection network outputs the extraction probability, the filament pulling probability, and the direction of movement. When the extraction probability is greater than a set extraction judgment threshold and continues for a preset number of frames, the extraction start event is determined. When the filament pulling probability is greater than a set pulling judgment threshold, the filament pulling state is determined. After the damping increment is subjected to amplitude limiting processing, it is superimposed on the current damping parameter. When the swallow wire pulling state is established or the picking force error exceeds the safety envelope, the stiffness parameter in the picking direction is reduced by a preset ratio.

[0013] In some specific embodiments, the extraction state detection network outputs the extraction probability, the filament pulling probability, and the direction of movement, and determines the extraction initiation event and the filament pulling state, further including: Inter-frame optical flow is extracted from the target hairy area and the adjacent swallow silk area from the continuous microscopic images, and the inter-frame optical flow is aggregated in time order to form extraction time sequence features and swallow silk traction time sequence features; The extraction probability and the target hair movement direction are calculated based on the extraction timing characteristics. When the extraction probability is continuously higher than the set extraction judgment threshold, the extraction start event is output. The swallow silk pulling probability is calculated based on the swallow silk pulling timing characteristics. When the swallow silk pulling probability is higher than the set pulling judgment threshold, the swallow silk pulling state is output. When the extraction probability is lower than the extraction recovery threshold, the filament pulling probability is lower than the set pulling judgment threshold, and the picking force error is within the safety envelope, the damping parameter is gradually restored to the damping value in the initial impedance parameter.

[0014] In some specific embodiments, it also includes: Record the characteristics of stray hairs, initial impedance parameters, force and torque curves, microscopic image frames, extraction trigger time, filament tension state, damping update history, and removal results for each hair removal process; Positive examples are selected from the records where stray hairs are completely removed, the probability of tufting of swallow silk is lower than the set tufting judgment threshold, and the force curve is smooth. Based on the positive examples and the remaining records, the stiffness inversion network and the graph neural network are periodically fine-tuned.

[0015] Based on the same concept, this invention also provides a bird's nest picking force-position mixing and smoothing adjustment system based on an impedance model, comprising: The data acquisition module is configured to acquire color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, as well as before and after micro-displacement loading, and to calculate the deformation characteristics of the bird's nest. The stray hair analysis module is configured to perform instance segmentation based on the color image and the depth point cloud to obtain the geometric features of the target stray hair and the degree of overlap between the target stray hair and the swallow hair; The stiffness inversion module is configured to input the deformation features and the force values ​​into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest and extract the local equivalent stiffness values ​​at the target hair embedding location. The parameter generation module is configured to use stray hairs as graph nodes, construct node features based on the geometric features, the degree of overlap and the equivalent stiffness distribution, and generate initial impedance parameters through a graph neural network. The compliance control module is configured to establish a task coordinate system based on the extension direction of the target stray hairs, calculate the position correction amount of the picking direction using the initial impedance parameters, and perform lateral position servoing. The damping reconstruction module is configured to acquire a microscopic image of the area where the target stray hair overlaps with the surrounding filaments, detect the initial event of target stray hair removal and the filament traction state based on the microscopic image, and input the picking force error change rate, microscopic image optical flow characteristics, filament traction probability and damping value in the initial impedance parameters into the removal adaptation network according to the detection results, obtain the damping increment and update the damping parameters, and input the updated damping parameters into the impedance relationship of the picking direction.

[0016] Compared with existing technologies, its advantages are as follows: By calculating the local deformation field of the bird's nest using the structured light pattern before and after the micro-displacement loading, and combining it with the spatial equivalent stiffness distribution inverted by the loading force, the local equivalent stiffness of the target hair embedding position can be included in the initial impedance parameter generation process, so that the control parameters have the input basis of the local mechanical state of the bird's nest.

[0017] The extension direction vector, local equivalent stiffness value, visible length and overlap degree of the target stray hairs are used to form the graph node features. The geometric and mechanical information of the neighboring stray hairs and swallow nest network is aggregated through message passing between graph nodes, so that the generated initial impedance parameters simultaneously characterize the characteristics of the target stray hairs and the local swallow nest structure state.

[0018] A task coordinate system is established based on the direction of the target stray hair extension. The position correction is calculated in the picking direction based on the measured force, the expected picking force, the fixed expected inertia, the expected damping, and the expected stiffness. The lateral position correction is calculated in the plane perpendicular to the picking direction based on the lateral force, so that the hair picking actuator can perform force-position hybrid compliant control along the direction of target stray hair removal.

[0019] The extraction initiation event of target hairs relative to the bird's nest matrix and the traction state of bird's nest fibers are detected by microscopic images. The rate of change of picking force error, optical flow characteristics and traction probability of bird's nest fibers are input into the extraction adaptation network to update the damping parameters. When bird's nest fibers are pulled or the picking force exceeds the safe envelope, the stiffness in the picking direction is reduced, thereby reducing the risk of bird's nest fibers being pulled or damaged synchronously.

[0020] Record the characteristics of stray hairs, initial impedance parameters, force and torque curves, microscopic image frames, extraction trigger time, filament pulling state, damping update history, and removal results. Then, use the selected positive examples to periodically fine-tune the stiffness inversion network and graph neural network to enable subsequent hair removal operations to use historical effective picking data to update parameter generation capabilities. Attached Figure Description

[0021] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1This is a flowchart illustrating some specific embodiments of the bird's nest picking force and position mixing and smoothing adjustment method based on impedance model of the present invention; Figure 2 This is a schematic diagram of the structure of the bird's nest picking force and position mixing and smoothing adjustment system based on the impedance model of the present invention in some specific embodiments. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application are also intended to include the plural forms, unless the context clearly indicates otherwise, and "multiple" generally includes at least two.

[0024] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0025] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0026] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0027] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0028] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0029] Reference Figure 1 A method for adjusting the force and smoothness of bird's nest picking based on an impedance model includes: S101, acquire color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, and calculate the deformation characteristics of the bird's nest. S102, Instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow's hairs; S103, input the deformation features and the force values ​​into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and extract the local equivalent stiffness values ​​at the target hair embedding location; S104, using stray hairs as graph nodes, constructing node features based on the geometric features, the degree of overlap, and the equivalent stiffness distribution, and generating initial impedance parameters through a graph neural network; S105, establish a task coordinate system based on the extension direction of the target stray hairs, calculate the position correction amount of the picking direction with the initial impedance parameters, and perform lateral position servoing; S106, acquire a microscopic image of the area where the target stray hairs overlap with the surrounding filaments, detect the initial event of target stray hair removal and the filament pulling state based on the microscopic image, and input the picking force error change rate, microscopic image optical flow characteristics, filament pulling probability and damping value in the initial impedance parameters into the removal adaptation network according to the detection results, obtain the damping increment and update the damping parameters, and input the updated damping parameters into the impedance relationship of the picking direction.

[0030] In this embodiment, a hair-picking actuator is mounted on the end effector of a six-axis industrial robot. The hair-picking actuator includes a miniature parallel gripper, a six-dimensional force sensor, a high-speed microscope camera, a micro-displacement loading head, a miniature structured light projection device, and a snapshot camera. The miniature parallel gripper is used to grip the grippable segment of the target hair; the six-dimensional force sensor is mounted on the gripper base to output three-dimensional force and three-dimensional torque; the high-speed microscope camera is coaxially arranged with the gripper to acquire microscopic images of the area where the target hair overlaps with the surrounding hair fibers; the micro-displacement loading head is located adjacent to the gripper to apply a preset micro-displacement to the neighborhood of the target hair before picking; the miniature structured light projection device and the snapshot camera are used to record the structured light patterns before and after the micro-displacement loading. The controller receives data from each sensor and performs network inference, impedance parameter calculation, and robot position control.

[0031] To provide a clearer explanation, the steps in the embodiments of the present invention are described in detail below: S101, acquire color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, and calculate the deformation characteristics of the bird's nest. Furthermore, color images, depth point clouds, and structured light patterns and force values ​​before and after micro-displacement loading of the area to be cleaned from the bird's nest are acquired to calculate the deformation characteristics of the bird's nest, including: Acquire color images and depth point clouds of the area to be cleaned from the bird's nest, as well as structured light patterns and force values ​​before and after applying a preset micro-displacement to the surface of the bird's nest. The micro-displacement loading head is placed against the surface of the bird's nest in the target area of ​​hairy feathers and a preset micro-displacement is applied. A speckle pattern is projected onto the surface of the bird's nest, and speckle images before and after the micro-displacement is applied, as well as the force value when the preset micro-displacement is applied, are recorded. A pixel-by-pixel matching calculation is performed on the speckle image to obtain the local displacement field on the surface of the bird's nest. The local deformation field of the bird's nest is then calculated based on the local displacement field, and the local deformation field of the bird's nest is used as the deformation feature.

[0032] In this embodiment, after the bird's nest sample to be cleaned is transported to the cleaning station and spatially positioned, the area of ​​the bird's nest to be cleaned is photographed, and a color image and a dense depth point cloud aligned with the color image are output. The color image records information about stray feathers, bird's nest fibers, and local texture, while the depth point cloud records the three-dimensional coordinate information of each pixel position. For example, the spatial sampling resolution is 0.05 mm, so that the visible segment of the target stray feathers and the surrounding bird's nest fiber structure have separable pixel and point cloud data.

[0033] The feather-picking actuator moves to the vicinity of the target feather area, and the micro-displacement loading head makes light contact with the surface of the bird's nest, applying a preset micro-displacement along the local surface normal. A micro-structured light projection device projects speckle patterns onto the surface of the bird's nest before loading and after loading stabilization, respectively. A fast-capture camera records the first and second speckle images; a six-dimensional force sensor or the force sensing unit built into the loading head synchronously records the corresponding force values. The preset micro-displacement is constrained to not cause permanent deformation of the bird's nest fibers and is pre-calibrated for different water content or density states of the bird's nest.

[0034] In this embodiment, using the local speckle windows in the first speckle image as a reference, a displacement matching window is searched in the second speckle image, and the lateral and longitudinal displacement components of each local speckle window are calculated to form a two-dimensional local displacement field. The local deformation field of the bird's nest is calculated based on the spatial rate of change of the two-dimensional local displacement field along the image coordinate direction. The local deformation field of the bird's nest includes the normal strain along the first image coordinate direction, the normal strain along the second image coordinate direction, and the shear strain.

[0035] When a local speckle area is affected by reflection, occlusion, or image blurring, preventing the formation of an effective displacement match, pixels with a displacement match confidence score lower than a preset confidence threshold are marked as invalid pixels, and the invalid pixels are filled using the interpolation results of surrounding valid pixels. For example, the preset confidence threshold is 0.85. The filled local deformation field of the bird's nest, along with the applied force value, is input into the stiffness inversion network. The applied force value represents the current loading magnitude, and the local deformation field of the bird's nest represents the local deformation distribution under the corresponding loading condition.

[0036] In this embodiment, depth point clouds can also be acquired using a binocular camera, a structured light camera, or a multi-view camera; multiple sets of structured light patterns and force values ​​can also be acquired by using multiple preset micro-displacement levels, and multiple sets of local deformation fields of swallow nests can be sorted by micro-displacement levels and input into the stiffness inversion network to form a local equivalent stiffness inversion input with multiple loading conditions.

[0037] S102, Instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow's hairs; Furthermore, instance segmentation is performed based on color images and depth point clouds to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow's filaments, including: A color image and a depth point cloud are input into a stray hair instance segmentation network. Based on the stray hair instance segmentation network, stray hair masks, filament masks, and embedding locations are output from the color image. Extract a 3D point set from the depth point cloud that coincides with the hair mask, perform principal direction analysis on the 3D point set to obtain the extension direction vector of the target hair in 3D space, and calculate the visible length based on the projection range of the 3D point set along the extension direction vector. The degree of overlap is calculated based on the ratio of the number of overlapping pixels between the stray hair mask and the swallow wire mask to the number of pixels in the stray hair mask. The geometric features include the embedding location, the extension direction vector, and the visible length.

[0038] In this embodiment, the color image and depth point cloud are aligned to pixel coordinates using camera calibration parameters. The stray feather instance segmentation network receives color texture information from the color image and spatial morphological information from the depth point cloud, and performs instance-level parsing on multiple stray feathers within the same area to be cleaned from the bird's nest. Each stray feather instance output by the network includes a stray feather identifier, a stray feather mask, a bird's nest thread mask, an embedding location, and a visible segment. The stray feather identifier is used to associate the same stray feather as the same processing object in subsequent stiffness extraction, graph node construction, impedance control, and process recording.

[0039] In this embodiment, the stray hair instance segmentation network adopts an instance segmentation structure including depthwise separable convolutions. The training annotations include stray hair regions and filament regions. During network inference, the network outputs stray hair masks and filament masks, and searches along the stray hair centerline from the exposed end to the dense filament region. The position where the stray hair mask begins to continuously overlap with the filament mask is determined as the embedding position. The stray hair mask is projected onto the depth point cloud to obtain the corresponding 3D point set. The principal direction analysis is performed on the 3D point set to calculate the direction with the largest variance of the 3D point set distribution, and the unit vector of the direction is recorded as the extension direction vector.

[0040] The sign of the extension direction vector points from the embedding position to the exposed end of the stray hair, ensuring that the subsequently established picking direction coordinate axis always points towards the stray hair removal direction. The 3D point set is projected onto the extension direction vector, and the difference between the maximum and minimum projection values ​​is taken as the visible length of the target stray hair. The overlap degree is obtained by dividing the number of overlapping pixels between the stray hair mask and the swallow-wire mask by the number of pixels in the stray hair mask, and is used to characterize the spatial overlap between the target stray hair and the swallow-wire network.

[0041] When multiple separated regions exist in the hair mask, the connected region that is continuous with the exposed segment and includes the embedding location is selected as the target hair mask. When there are holes in the depth point cloud within the neighborhood of the embedding location, the centerline is 3D fitted using adjacent effective depth points within the hair mask, and the missing points are filled in. The processed extension direction vector, visible length, overlap degree, and embedding location are then fed into the stiffness extraction and graph node construction process.

[0042] In this embodiment, a stray hair resolution method based on color difference, edge detection, and 3D geometric clustering can also be used to output the stray hair mask and embedding position, and the extension direction vector can be calculated using the 3D curve fitting results of the visible stray hair segments. Different resolution methods all provide the embedding position, extension direction vector, visible length, and overlap degree to the graph node construction process.

[0043] S103, input the deformation features and the force values ​​into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and extract the local equivalent stiffness values ​​at the target hair embedding location; Furthermore, the deformation characteristics and force values ​​are input into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and the local equivalent stiffness values ​​at the embedding locations of the target hairs are extracted, including: The local deformation field and force values ​​of bird's nest are input into the stiffness inversion network to obtain a spatial equivalent stiffness distribution map; The two-dimensional map of the local deformation field of the bird's nest is stitched together with the force value to form a multi-channel tensor, which is then input into the stiffness inversion network of the encoder and decoder structure to generate a spatial equivalent stiffness distribution map of the same size as the two-dimensional map of the local deformation field of the bird's nest. Read the effective pixel values ​​of the embedding position and the preset neighborhood of the embedding position in the spatial equivalent stiffness distribution map, and determine the local equivalent stiffness value based on the effective pixel values; The equivalent stiffness distribution includes a spatial equivalent stiffness distribution map, and the deformation characteristics include the local deformation field of the bird's nest.

[0044] In this embodiment, the local deformation field of the bird's nest reflects the degree of local deformation of the bird's nest fiber network under micro-displacement loading, and the force value reflects the magnitude of the loading. The stiffness inversion network infers the local equivalent stiffness of the bird's nest based on the deformation differences at different locations under the same or similar loading displacements. Regions with larger local equivalent stiffness exhibit smaller deformation under the same loading conditions, while regions with smaller local equivalent stiffness exhibit larger deformation. Each pixel of the spatial equivalent stiffness distribution map stores the corresponding local equivalent stiffness value and uses the same image coordinate system as the color image and depth point cloud.

[0045] In this embodiment, a two-dimensional image of the local deformation field of the bird's nest is concatenated with the force values ​​to form a multi-channel tensor. The local deformation field of the bird's nest includes a first normal strain channel, a second normal strain channel, and a shear strain channel; the force values ​​are normalized and expanded into force channels with the same width and height as the local deformation field of the bird's nest. The encoder of the stiffness inversion network downsamples the multi-channel tensor to extract local deformation texture and regional mechanical features; the decoder upsamples the encoded features and outputs a spatially equivalent stiffness distribution map aligned with the pixels of the input image.

[0046] The training data for the stiffness inversion network consists of multi-location micro-displacement loading data and simulation data from bird's nest samples. A known micro-displacement is applied to each training location, and the corresponding force is measured. The ratio of force to micro-displacement or the calibrated equivalent stiffness value forms a local equivalent stiffness label. After training, the stiffness inversion network receives the local deformation field and force values ​​of the bird's nest during online feather removal, outputs a spatial equivalent stiffness distribution map of the current feather-to-be-removed area, and writes the local equivalent stiffness value corresponding to the embedding location of the target feather into the graph node features of that feather.

[0047] When the embedding location is located on the pixel boundary of the spatial equivalent stiffness distribution map, the mean local equivalent stiffness is calculated from effective pixels with a radius of 2px around the embedding location, and this mean local equivalent stiffness is used as the local equivalent stiffness value of the target hair. When the proportion of effective pixels in the spatial equivalent stiffness distribution map is less than 0.8, the local equivalent stiffness value obtained in the previous control cycle is retained, and the local deformation field acquisition and stiffness inversion are re-executed in the next control cycle.

[0048] In this embodiment, the force value, the statistics of the local deformation field of the bird's nest, the local density of the bird's nest fibers, and the surface curvature of the bird's nest can also be input into the stiffness inversion network. The local density of the bird's nest fibers is calculated from the pixel ratio of the bird's nest fiber mask in a preset neighborhood, and the surface curvature of the bird's nest is calculated from the local surface fitted by the depth point cloud.

[0049] S104 uses stray hairs as graph nodes, constructs node features based on geometric characteristics, overlap degree, and equivalent stiffness distribution, and generates initial impedance parameters through a graph neural network.

[0050] Furthermore, using stray hairs as graph nodes, node features are constructed based on geometric characteristics, overlap degree, and equivalent stiffness distribution. Initial impedance parameters are generated through a graph neural network, including: Using stray hairs as graph nodes, and the extension direction vector, local equivalent stiffness value, visible length and overlap degree as node features, the initial impedance parameters are generated through message passing based on the graph neural network. Triangulation is performed based on the embedding position of each stray hair, and connecting edges with a length greater than a preset threshold are deleted. The remaining connecting edges form the connection relationship between graph nodes. The Euclidean distance between graph nodes and the change in the density of the swallow's thread on the connecting path are used as edge features. The message weight is calculated using node features and edge features, and the geometric and mechanical information of neighboring graph nodes is aggregated. The desired damping and desired stiffness in the picking direction are output, as well as the tangential compliance center stiffness and tangential damping as the compliance center adjustment parameters perpendicular to the picking direction. The initial impedance parameters include the desired damping and desired stiffness in the pick direction, as well as the tangential compliance center stiffness and tangential damping.

[0051] Furthermore, message weights are calculated based on node and edge features, and the geometric and mechanical information of neighbor graph nodes is aggregated, including: The attention weight of the connecting edge is calculated based on the difference in the extension direction vector of the graph node, the difference in the local equivalent stiffness value, the Euclidean distance between the graph nodes, and the change in the filament density on the connecting path. The node features of neighboring graph nodes are aggregated using attention weights to update the geometric and mechanical features of the graph nodes; The updated geometric and mechanical features are input to the parameter output layer to generate the desired damping in the pick direction, the desired stiffness in the pick direction, the tangential compliance center stiffness, and the tangential damping.

[0052] In this embodiment, each stray hair identified within the current field of view constitutes a graph node. Graph node features include the extension direction vector, local equivalent stiffness value, visible length, and degree of overlap. The connecting edges between graph nodes represent the spatial adjacency relationships between adjacent stray hairs within the bird's nest area and the structural associations of the bird's nest fibers along the connecting paths. Edge features include the Euclidean distance between graph nodes and the variation in bird's nest fiber density. By constructing a stray hair attribute graph through graph nodes and connecting edges, the graph neural network inputs the target stray hair's own geometric and mechanical information along with the local structural information of the bird's nest fiber network containing adjacent stray hairs into the message passing process.

[0053] In this embodiment, an initial set of connecting edges is formed by triangulation based on the embedding position of each stray hair, and connecting edges with a length greater than 3mm are deleted. For any connecting edge, the Euclidean distance between the embedding positions of two graph nodes is calculated, and the swallow hair mask is sampled along the connecting path. The percentage of swallow hair pixels within an equal-length path window is calculated, and the change in the percentage of swallow hair pixels in adjacent windows is used to characterize the change in swallow hair density. The Euclidean distance and the change in swallow hair density together serve as the edge features of the connecting edge.

[0054] The graph neural network employs a graph sampling aggregation network with an attention-based message passing structure and performs two layers of message passing. The first layer of message passing fuses the node features and edge features directly connected to the graph nodes, while the second layer fuses the graph node features updated by the first layer. After message passing, the parameter output layer outputs the desired damping and desired stiffness in the picking direction for each graph node, as well as the tangential compliance center stiffness and tangential damping, which serve as adjustment parameters for the compliance center perpendicular to the picking direction. The desired inertia of the robot end effector and the picking actuator is fixed based on the nominal inertia.

[0055] In this embodiment, for graph nodes Graph Nodes The attention weights of the connection edges are denoted as The calculation relationship is as follows: ; in, Represents graph nodes The set of connected neighbor graph nodes; Represents graph nodes With graph nodes Attention rating between; The difference in the extension direction vectors, the difference in local equivalent stiffness values, the Euclidean distance between graph nodes, and the change in filament density are obtained by adding them together after full connection mapping; the difference in the extension direction vectors is calculated by the angle between two extension direction vectors; the difference in local equivalent stiffness values ​​is calculated by the absolute difference between two local equivalent stiffness values. Representation graph nodes Geometric and mechanical information enters the graph nodes The weight during the message passing process.

[0056] Graph Node In the The update feature after layer message passing is denoted as The calculation relationship is as follows: ; in, Represents graph nodes In the Layer node characteristics; Represents the nodes in the neighbor graph In the Layer node characteristics; Represents graph nodes With graph nodes Edge features between them; Mapping parameters representing the characteristics of the node itself; The mapping parameters represent the features of neighboring nodes and the features of edges; This represents a non-linear activation function.

[0057] Based on the above calculations, graph nodes The updated features simultaneously include information on the direction, stiffness, visible length, and degree of overlap of the target stray hairs, as well as the geometric and mechanical information of the neighboring stray hairs connected by the swallowtail network.

[0058] The parameter output layer maps the graph node features after the second-layer message passing to four parameters. The desired damping and desired stiffness in the picking direction are entered into the picking direction impedance relationship; the tangential compliance center stiffness and tangential damping are entered into the admittance relationship perpendicular to the picking direction. The output of the parameter output layer is processed by a limiting function to ensure that all the above parameters are within the parameter range allowed by the robot's end effector.

[0059] In this embodiment, the connection relationship between graph nodes can also be formed through radius neighborhood search. For each graph node, neighboring graph nodes are connected with a search radius of 3mm, and edge features are constructed using Euclidean distance and changes in feather density. Both the feather attribute map formed by radius neighborhood search and the feather attribute map formed by triangulation are entered into the message passing process of the graph neural network.

[0060] S105, establish a task coordinate system based on the extension direction of the target stray hairs, calculate the position correction amount of the picking direction with the initial impedance parameters, and perform lateral position servoing.

[0061] Furthermore, a task coordinate system is established based on the extension direction of the target stray hairs. The position correction amount for the picking direction is calculated using the initial impedance parameters, and lateral position servoing is performed, including: Move to the target stray hairs and establish a task coordinate system based on the extending direction vector; Establish the pick-up direction coordinate axis with the extended direction vector, obtain the actual force in the pick-up direction, and calculate the pick-up force error between the actual force and the expected pick-up force; substitute the expected damping and expected stiffness in the pick-up direction from the fixed expected inertia and initial impedance parameters into the time-varying impedance relationship, calculate the position correction amount according to the backward Euler recursion, and superimpose the position correction amount onto the expected position of the robot position loop. The lateral force in the plane perpendicular to the picking direction is obtained. The admittance relationship is established with the tangential compliance center stiffness and tangential damping in the compliance center adjustment parameters. The lateral position correction is calculated based on the lateral force and input into the robot position loop. The lateral position correction is updated in each position control cycle.

[0062] In this embodiment, a target stray hair is selected from the graph nodes output by the graph neural network, and moved above the target stray hair based on its embedding position and grippable segment. The extension direction vector from the embedding position to the exposed end of the stray hair is used as the Z-axis of the task coordinate system, with the X and Y axes of the task coordinate system located in a plane perpendicular to the Z-axis. Force-position hybrid compliant control is performed in the Z-axis direction, while position servoing and lateral admittance adjustment are performed in the X and Y-axis directions.

[0063] In this embodiment, a pick-up direction coordinate axis is established using the extension direction vector, the measured force in the pick-up direction is obtained, and the pick-up force error between the measured force and the expected pick-up force is calculated. The position correction amount is denoted as... The position correction speed is denoted as The position correction acceleration is denoted as Expected picking force is denoted as The measured force in the picking direction is recorded as The picking force error is denoted as The picking force error satisfies The fixed expected inertia is denoted as... The time-varying damping parameter is denoted as The time-varying stiffness parameter is denoted as The time-varying impedance relationship for the chosen direction is as follows: ; in, This is the position correction amount; For position correction speed; To correct acceleration for position; To pick out force errors, satisfy , To seek the desired force, To determine the direction of the measured force; To fix the desired inertia; These are time-varying damping parameters; This is a time-varying stiffness parameter.

[0064] The desired damping direction in the initial impedance parameters is used as The desired stiffness in the pick direction in the initial impedance parameters is used as The output of the six-dimensional force sensor is read in each control cycle. Calculated based on the expected picking force curve The position correction for the current control cycle is obtained according to the backward Euler recursion. The corrected expected position is determined by the original expected position and... The results are added together and sent to the robot's position loop.

[0065] The desired picking force curve starts from 0 and slowly rises to a preset holding value, which is then maintained. The preset holding value is determined by establishing a parameter table based on the diameter of stray hairs, visible length, degree of overlap, and local equivalent stiffness value.

[0066] In this embodiment, the expected rise time of the pick-up force curve is 0.5s, and the control cycle is 5ms. When the pick-up force error exceeds the upper boundary of the safety envelope, the position correction amount for the next control cycle is reduced; when the pick-up force error is below the lower boundary of the safety envelope, the expected pick-up force curve is maintained and the position correction amount continues to be calculated.

[0067] Acquire the lateral force in the plane perpendicular to the picking direction. The lateral force includes the X-axis lateral force and the Y-axis lateral force. Calculate the lateral position correction based on the X-axis lateral force, Y-axis lateral force, tangential compliance center stiffness, and tangential damping. The lateral position correction is superimposed on the desired X-axis and Y-axis positions of the robot's position loop, ensuring the gripper center remains aligned with the grippable segment of the target hair and adjusts with the extraction trajectory. When the lateral force exceeds a preset lateral force threshold, reduce the maximum step size of the lateral position correction and recalculate the admittance relationship in the next control cycle.

[0068] In this embodiment, the lateral position correction can also be calculated using the center position of the gripper segment of the target stray hair, the center position of the gripper, and the lateral force. The center position of the gripper segment provides the lateral centering reference, the center position of the gripper provides the current position, and the lateral force provides the lateral force state during the gripping and extraction process. All three are incorporated into the lateral position servo process.

[0069] S106. Acquire a microscopic image of the area where the target stray hairs overlap with the surrounding filaments. Detect the initial event of target stray hair removal and the filament pulling state based on the microscopic image. Input the picking force error change rate, microscopic image optical flow characteristics, filament pulling probability, and damping value in the initial impedance parameters into the removal adaptation network according to the detection results. Obtain the damping increment and update the damping parameters. Input the updated damping parameters into the impedance relationship of the picking direction.

[0070] Furthermore, based on the detection of the target stray hair extraction initiation event and the filament pulling state using microscopic images, and according to the detection results, the rate of change of picking force error, the optical flow characteristics of the microscopic image, the filament pulling probability, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction, including: A microscopic image of the area where the target feathers overlap with the surrounding bird's nest fibers is acquired. The microscopic image is then input into the extraction state detection network to detect the extraction initiation event of the target feathers relative to the bird's nest matrix and the traction state of the bird's nest fibers. When the extraction initiation event or the traction state of the bird's nest fibers is detected, the picking force error change rate, the optical flow characteristics of the microscopic image, the traction probability of the bird's nest fibers, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction. Among them, the extraction state detection network outputs the extraction probability, the filament pulling probability and the direction of movement. When the extraction probability is greater than the set extraction judgment threshold and continues for a preset number of frames, the extraction start event is determined. When the filament pulling probability is greater than the set pulling judgment threshold, the filament pulling state is determined. After applying a limiting process to the damping increment, it is superimposed on the current damping parameter. When the filament pulling state is established or the picking force error exceeds the safety envelope, the stiffness parameter in the picking direction is reduced by a preset ratio.

[0071] Furthermore, the extraction state detection network outputs the extraction probability, the wire pulling probability, and the direction of movement, and determines the extraction initiation event and the wire pulling state, including: Inter-frame optical flow is extracted from the target hairy area and the adjacent swallow silk area from the continuous microscopic images, and the inter-frame optical flow is aggregated in time order to form extraction time sequence features and swallow silk traction time sequence features; The extraction probability and the direction of movement of the target stray hairs are calculated based on the extraction time sequence characteristics. When the extraction probability is continuously higher than the set extraction judgment threshold, the extraction start event is output. The filament pulling probability is calculated based on the filament pulling time sequence characteristics. When the filament pulling probability is higher than the set pulling judgment threshold, the filament pulling state is output. When the probability of extraction is lower than the extraction recovery threshold, the probability of filament pulling is lower than the set pulling judgment threshold, and the picking force error is within the safe envelope, the damping parameter is gradually restored to the damping value in the initial impedance parameter.

[0072] Furthermore, it also includes: Record the characteristics of stray hairs, initial impedance parameters, force and torque curves, microscopic image frames, extraction trigger time, filament tension state, damping update history, and removal results for each hair removal process; Positive examples are selected from the records where stray hairs are completely removed, the probability of tufting is lower than the set tufting threshold, and the force curve is smooth. Based on the positive examples and the remaining records, the stiffness inversion network and graph neural network are periodically fine-tuned.

[0073] In this embodiment, continuous microscopic images of the overlapping area between the target stray feathers and the surrounding swallow silk are acquired at a frame rate of no less than 200 fps. The extraction state detection network extracts the inter-frame optical flow of the target stray feather and swallow silk regions from adjacent image frames, and analyzes the motion differences between the two in the time dimension. When the target stray feathers undergo continuous displacement along the extraction direction while the adjacent swallow silk remains relatively stable, the extraction probability increases; when the target stray feathers and adjacent swallow silk exhibit synchronous displacement in the same direction, the swallow silk traction probability increases.

[0074] In this embodiment, the detachment state detection network outputs the detachment probability. The extraction state detection network includes an optical flow feature extraction network and a temporal feature aggregation network. The optical flow feature extraction network calculates the optical flow components of the overlapping area between the target stray hairs and the surrounding swallow hairs from adjacent microscopic image frames. The temporal feature aggregation network aggregates the optical flow components of multiple control cycles in time sequence and outputs the extraction probability. The probability and direction of the swallow's silk being pulled are determined. The extraction threshold is set to 0.75, and the preset frame rate is 3 frames. Extraction probability... If the threshold for withdrawal is exceeded for three consecutive frames, a withdrawal start event is output.

[0075] The extraction adaptation network employs a multilayer perceptron, and the input vector includes the desired damping of the picking direction from the initial impedance parameters and the rate of change of the picking force error at the current moment. The first directional component of the average optical flow in the target hairy region Second direction component And the probability of the swallow's silk being pulled. The rate of change of the picking force error. The output damping increment of the extraction adaptive network is obtained by dividing the difference in picking force error between the current control cycle and the previous control cycle by the control cycle duration. The damping parameter update relationship is as follows: ; in, The damping parameter represents the current control cycle; This represents the damping parameter of the previous control cycle; This represents the damping increment of the output of the extraction adaptive network; This indicates the upper limit of the damping increment. In this embodiment, the upper limit of the damping increment is 20% of the desired damping in the selected direction from the initial impedance parameters. The damping parameters after limiting are input to the time-varying impedance relationship.

[0076] When the picking force error exceeds the safety envelope or the probability of the swallow wire being pulled is higher than the set pulling judgment threshold, the stiffness parameter in the picking direction is reduced by a preset ratio according to the excess amount. The preset ratio is between 0.05 and 0.15; in this embodiment, 0.1 is selected. The adjusted stiffness parameter and the updated damping parameter are used together to input the time-varying impedance relationship for the next control cycle.

[0077] Using the current and previous microscopic image frames as input, dense optical flow is extracted from the stray hair mask region and the surrounding swallow silk mask region, respectively. The mean, variance, and orientation distribution of the optical flow components of each pixel in the two regions are calculated to form the current inter-frame optical flow feature. The inter-frame optical flow features of five consecutive frames are input into a temporal feature aggregation network in chronological order to form extracted temporal features and swallow silk traction temporal features.

[0078] Calculate the withdrawal probability based on the withdrawal time series characteristics. And the direction of motion, in the probability of withdrawal When the probability continuously exceeds the set withdrawal threshold, a withdrawal start event is output; When the error is below the extraction recovery threshold, the probability of the swallow silk being pulled is below the set pulling judgment threshold, and the picking force error is within the safe envelope, the damping parameter is gradually restored to the damping value in the initial impedance parameter.

[0079] Temporal feature aggregation network outputs extraction probability And direction of motion. The detachment recovery threshold is set to 0.30. Detachment probability. When the error is below the extraction recovery threshold, the probability of the swallow silk being pulled is below the set pulling judgment threshold, and the picking force error is within the safe envelope, in each control cycle, the difference between the current damping parameter and the damping value in the initial impedance parameter is reduced by 5% of the expected damping in the picking direction in the initial impedance parameter, until the current damping parameter recovers to the damping value in the initial impedance parameter. The damping parameter during the recovery process continuously inputs the time-varying impedance relationship.

[0080] Each feather picking record is indexed by a stray feather identifier. Stray feather characteristics include the extension direction vector, local equivalent stiffness value, visible length, and overlap degree. Initial impedance parameters include desired damping in the picking direction, desired stiffness in the picking direction, tangential compliance center stiffness, and tangential damping. Force and torque curves are formed by arranging data output from a six-dimensional force sensor in chronological order across each control cycle. The damping update history includes the extraction start event time and extraction probability. , probability and state of filament pulling, damping increment Damping parameters and stiffness adjustment amount after amplitude limiting.

[0081] After feather removal, images of the original embedded regions of the target feathers are acquired, and the removal results are recorded in conjunction with the gripper release state. Removal results include complete removal of feathers, broken feather residue, simultaneous pulling of swallow feather strands, and broken swallow feather strands. Records with complete feather removal, swallow feather strand pulling probability below a set pulling threshold, and smooth force curves are designated as positive examples, and these positive examples, along with the remaining records, form an experience pool. During offline periods, the experience pool is input into a stiffness inversion network and a graph neural network, undergoing incremental training and updating network weights periodically. The updated network is then deployed to the online system after boundary verification of control parameters.

[0082] In this embodiment, the experience pool can be grouped according to the water content of the bird's nest, the density of bird's nest fibers in the target area, the visible length of impurities, and the degree of overlap. Incremental training of the stiffness inversion network and the graph neural network is performed within each group. The network weights after grouping are jointly selected by the image features of the current bird's nest area, the density of bird's nest fibers, the visible length of impurities, and the degree of overlap during online inference.

[0083] The following example illustrates this embodiment: In a specific example, 12 stray feathers were identified within the area of ​​the bird's nest to be cleaned. Color images and depth point clouds were acquired, and a micro-displacement loading head applied a preset micro-displacement to the target area, recording the structured light speckle images and corresponding force values ​​before and after loading. Pixel-by-pixel matching was used to obtain the local deformation field of the bird's nest, and a stiffness inversion network generated a spatial equivalent stiffness distribution map. The local equivalent stiffness value, extension direction vector, visible length, and overlap degree at the embedding location of the target stray feathers collectively constitute the node features of the target map nodes.

[0084] Triangulation is performed based on the embedding positions of 12 stray hairs, and connecting edges longer than 3mm are removed to obtain a stray hair attribute map. A graph neural network calculates message weights based on the node features of the target graph node, the node features of neighboring graph nodes, Euclidean distance, and changes in filament density. It outputs the expected damping and expected stiffness of the target stray hair picking direction, as well as the tangential compliance center stiffness and tangential damping. After moving to the target stray hair, the task coordinate system Z-axis is formed by the extension direction vector from the embedding position to the exposed end. The robot position loop calculates the position correction amount of the picking direction based on the time-varying impedance relationship.

[0085] During the feather removal process, microscopic images of the overlapping areas of the target feathers and surrounding swallow feathers are continuously acquired. The extraction state detection network extracts inter-frame optical flow from the continuous microscopic images, forming extraction temporal features and swallow feather traction temporal features. Extraction probability. When three consecutive frames exceed the set extraction threshold, an extraction start event is output. Extraction adapts to the network's receiving pick-up force error variation rate. Mean optical flow component and The probability of drawing the swallow wire and the damping value in the initial impedance parameters, and the output damping increment. Damping increment After being limited, the damping parameters are superimposed onto the current damping parameters, and the updated damping parameters are input to the time-varying impedance relationship. Extraction probability. When the pull-out recovery threshold is lower than the set pull-out judgment threshold, and the picking force error enters the safety envelope, the damping parameter gradually recovers to the damping value in the initial impedance parameter according to the control cycle. When the pull-out probability of the swallow wire is higher than the set pull-out judgment threshold or the picking force error exceeds the safety envelope, the stiffness parameter in the picking direction is reduced by a preset ratio.

[0086] For the purpose of simplicity, the method steps disclosed in the above embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps may be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are exemplary embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0087] like Figure 2 As shown, the present invention also provides a bird's nest picking force-position mixing and smoothing adjustment system based on an impedance model, comprising: The data acquisition module 201 is configured to acquire color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, as well as before and after micro-displacement loading, and to calculate the deformation characteristics of the bird's nest. The stray hair analysis module 202 is configured to perform instance segmentation based on color images and depth point clouds to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow hairs; The stiffness inversion module 203 is configured to input deformation features and force values ​​into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest and extract the local equivalent stiffness value at the embedding location of the target hair. The parameter generation module 204 is configured to use stray hairs as graph nodes, construct node features based on geometric features, overlap degree and equivalent stiffness distribution, and generate initial impedance parameters through a graph neural network. The compliance control module 205 is configured to establish a task coordinate system based on the extension direction of the target stray hairs, calculate the position correction amount of the picking direction with the initial impedance parameters, and perform lateral position servoing. The damping reconstruction module 206 is configured to acquire a microscopic image of the area where the target stray hair overlaps with the surrounding filaments, detect the initial event of the target stray hair removal and the filament traction state based on the microscopic image, and input the rate of change of picking force error, the optical flow characteristics of the microscopic image, the filament traction probability and the damping value in the initial impedance parameters into the removal adaptation network according to the detection results, obtain the damping increment and update the damping parameters, and input the updated damping parameters into the impedance relationship of the picking direction.

[0088] It is worth noting that although only some basic functional modules are disclosed in the embodiments of this invention, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules should not be considered as the scope of protection of this invention being limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described separately according to their functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adjusting the force and smoothness of bird's nest picking based on an impedance model, characterized in that, include: Acquire color images, depth point clouds, and structured light patterns and force values ​​of the area to be cleaned in the bird's nest before and after micro-displacement loading, and calculate the deformation characteristics of the bird's nest. Instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow hairs; The deformation features and the force values ​​are input into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and the local equivalent stiffness values ​​of the target hair embedding positions are extracted. Using stray hairs as graph nodes, node features are constructed based on the geometric features, the degree of overlap, and the equivalent stiffness distribution. Initial impedance parameters are generated through a graph neural network. A task coordinate system is established based on the extension direction of the target stray hairs. The position correction amount of the picking direction is calculated using the initial impedance parameters, and the lateral position servo is executed. A microscopic image of the area where the target stray hair overlaps with the surrounding filaments is acquired. Based on the microscopic image, the initiation event of the target stray hair extraction and the filament traction state are detected. According to the detection results, the rate of change of the picking force error, the optical flow characteristics of the microscopic image, the filament traction probability, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction.

2. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 1, characterized in that, Acquire color images, depth point clouds, and structured light patterns and force values ​​of the area to be cleaned from the bird's nest before and after micro-displacement loading; calculate the deformation characteristics of the bird's nest; further including: Acquire color images and depth point clouds of the area to be cleaned from the bird's nest, as well as structured light patterns and force values ​​before and after applying a preset micro-displacement to the surface of the bird's nest. The micro-displacement loading head is placed against the surface of the bird's nest in the vicinity of the target hairy area and the preset micro-displacement is applied. A speckle pattern is projected onto the surface of the bird's nest, and the speckle images before and after the micro-displacement is applied, as well as the force value when the preset micro-displacement is applied, are recorded. A pixel-by-pixel matching calculation is performed on the speckle image to obtain the local displacement field of the bird's nest surface, and the local deformation field of the bird's nest is calculated based on the local displacement field, and the local deformation field of the bird's nest is used as the deformation feature.

3. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 1, characterized in that, Instance segmentation is performed based on the color image and the depth point cloud to obtain the geometric features of the target stray hairs and the degree of overlap between the target stray hairs and the swallow's filaments, further including: The color image and the depth point cloud are input into the stray hair instance segmentation network, and the stray hair instance segmentation network outputs stray hair mask, swallow hair mask and embedding position in the color image based on the stray hair instance segmentation network. A three-dimensional point set is extracted from the depth point cloud that coincides with the hair mask. A principal direction analysis is performed on the three-dimensional point set to obtain the extension direction vector of the target hair in three-dimensional space. The visible length is calculated based on the projection range of the three-dimensional point set along the extension direction vector. The degree of overlap is calculated based on the ratio of the number of overlapping pixels between the stray hair mask and the swallow silk mask to the number of pixels in the stray hair mask; The geometric features include the embedding location, the extension direction vector, and the visible length.

4. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 2, characterized in that, The deformation characteristics and the applied force values ​​are input into a stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest, and the local equivalent stiffness values ​​at the target hair embedding locations are extracted, further including: The local deformation field of the bird's nest and the force value are input into the stiffness inversion network to obtain a spatial equivalent stiffness distribution map; The two-dimensional image of the local deformation field of the bird's nest is spliced ​​with the force value to form a multi-channel tensor, which is then input into the stiffness inversion network of the encoder and decoder structure to generate a spatial equivalent stiffness distribution map of the same size as the two-dimensional image of the local deformation field of the bird's nest. Read the effective pixel values ​​of the embedding position and the preset neighborhood of the embedding position in the spatial equivalent stiffness distribution map, and determine the local equivalent stiffness value based on the effective pixel values; The equivalent stiffness distribution includes the spatial equivalent stiffness distribution map, and the deformation characteristics include the local deformation field of the bird's nest.

5. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 3, characterized in that, Using stray hairs as graph nodes, node features are constructed based on the aforementioned geometric features, the degree of overlap, and the equivalent stiffness distribution. Initial impedance parameters are generated through a graph neural network, further including: Using stray hairs as graph nodes, and the extension direction vector, the local equivalent stiffness value, the visible length, and the degree of overlap as node features, initial impedance parameters are generated through message passing based on a graph neural network. Triangulation is performed based on the embedding position of each stray hair, and connecting edges with a length greater than a preset threshold are deleted. The remaining connecting edges form the connection relationship between graph nodes. The Euclidean distance between graph nodes and the change in the density of the swallow's thread on the connecting path are used as edge features. The message weight is calculated using the node features and the edge features, and the geometric and mechanical information of neighboring graph nodes is aggregated. The desired damping and desired stiffness in the picking direction are output, as well as the tangential compliance center stiffness and tangential damping as the compliance center adjustment parameters perpendicular to the picking direction. The initial impedance parameters include the desired damping and desired stiffness in the picking direction, as well as the tangential compliance center stiffness and tangential damping.

6. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 5, characterized in that, A task coordinate system is established based on the extension direction of the target stray hairs. The position correction amount of the picking direction is calculated using the initial impedance parameters, and lateral position servoing is performed. This further includes: Move to the target stray hairs and establish a task coordinate system based on the extended direction vector; Establish the picking direction coordinate axis with the extended direction vector, obtain the actual force in the picking direction, and calculate the picking force error between the actual force and the expected picking force; substitute the fixed expected inertia, the expected damping and expected stiffness in the picking direction in the initial impedance parameters into the time-varying impedance relationship, calculate the position correction amount according to the backward Euler recursion, and superimpose the position correction amount onto the expected position of the robot position loop; The lateral force in the plane perpendicular to the picking direction is obtained. An admittance relationship is established with the tangential compliance center stiffness and tangential damping in the compliance center adjustment parameters. The lateral position correction is calculated based on the lateral force and input into the robot position loop. The lateral position correction is updated in each position control cycle.

7. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 1, characterized in that, Based on the detection of the target stray hair extraction initiation event and the filament pulling state using the microscopic image, and according to the detection results, the rate of change of picking force error, the optical flow characteristics of the microscopic image, the filament pulling probability, and the damping value in the initial impedance parameters are input into the extraction adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction, further including: A microscopic image of the area where the target feathers overlap with the surrounding bird's nest fibers is acquired. The microscopic image is then input into a separation state detection network to detect the separation initiation event of the target feathers relative to the bird's nest matrix and the traction state of the bird's nest fibers. When the separation initiation event or the traction state of the bird's nest fibers is detected, the picking force error change rate, the optical flow characteristics of the microscopic image, the traction probability of the bird's nest fibers, and the damping value in the initial impedance parameters are input into a separation adaptation network to obtain the damping increment and update the damping parameters. The updated damping parameters are then input into the impedance relationship of the picking direction. The extraction state detection network outputs the extraction probability, the filament pulling probability, and the direction of movement. When the extraction probability is greater than a set extraction judgment threshold and continues for a preset number of frames, the extraction start event is determined. When the filament pulling probability is greater than a set pulling judgment threshold, the filament pulling state is determined. After the damping increment is subjected to amplitude limiting processing, it is superimposed on the current damping parameter. When the swallow wire pulling state is established or the picking force error exceeds the safety envelope, the stiffness parameter in the picking direction is reduced by a preset ratio.

8. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 7, characterized in that, The extraction state detection network outputs the extraction probability, the strand pulling probability, and the direction of movement, and determines the extraction initiation event and the strand pulling state, further including: Inter-frame optical flow is extracted from the target hairy area and the adjacent swallow silk area from the continuous microscopic images, and the inter-frame optical flow is aggregated in time order to form extraction time sequence features and swallow silk traction time sequence features; The extraction probability and the target hair movement direction are calculated based on the extraction timing characteristics. When the extraction probability is continuously higher than the set extraction judgment threshold, the extraction start event is output. The swallow silk pulling probability is calculated based on the swallow silk pulling timing characteristics. When the swallow silk pulling probability is higher than the set pulling judgment threshold, the swallow silk pulling state is output. When the extraction probability is lower than the extraction recovery threshold, the filament pulling probability is lower than the set pulling judgment threshold, and the picking force error is within the safety envelope, the damping parameter is gradually restored to the damping value in the initial impedance parameter.

9. The method for adjusting the force and smoothness of bird's nest picking based on an impedance model according to claim 1, characterized in that, Also includes: Record the characteristics of stray hairs, initial impedance parameters, force and torque curves, microscopic image frames, extraction trigger time, filament tension state, damping update history, and removal results for each hair removal process; Positive examples are selected from the records where stray hairs are completely removed, the probability of tufting of swallow silk is lower than the set tufting judgment threshold, and the force curve is smooth. Based on the positive examples and the remaining records, the stiffness inversion network and the graph neural network are periodically fine-tuned.

10. A bird's nest picking force-position mixing and smoothing adjustment system based on an impedance model, characterized in that, include: The data acquisition module is configured to acquire color images, depth point clouds, and structured light patterns and force values ​​of the area of ​​the bird's nest to be cleaned, as well as before and after micro-displacement loading, and to calculate the deformation characteristics of the bird's nest. The stray hair analysis module is configured to perform instance segmentation based on the color image and the depth point cloud to obtain the geometric features of the target stray hair and the degree of overlap between the target stray hair and the swallow hair; The stiffness inversion module is configured to input the deformation features and the force values ​​into the stiffness inversion network to obtain the equivalent stiffness distribution of the bird's nest and extract the local equivalent stiffness values ​​at the target hair embedding location. The parameter generation module is configured to use stray hairs as graph nodes, construct node features based on the geometric features, the degree of overlap and the equivalent stiffness distribution, and generate initial impedance parameters through a graph neural network. The compliance control module is configured to establish a task coordinate system based on the extension direction of the target stray hairs, calculate the position correction amount of the picking direction using the initial impedance parameters, and perform lateral position servoing. The damping reconstruction module is configured to acquire a microscopic image of the area where the target stray hair overlaps with the surrounding filaments, detect the initial event of target stray hair removal and the filament traction state based on the microscopic image, and input the picking force error change rate, microscopic image optical flow characteristics, filament traction probability and damping value in the initial impedance parameters into the removal adaptation network according to the detection results, obtain the damping increment and update the damping parameters, and input the updated damping parameters into the impedance relationship of the picking direction.