Six-dimensional pose recognition and control method for robot arm based on reinforcement learning

Abstract

This invention relates to the field of robotic arm control technology, specifically a six-dimensional posture recognition and control method for robotic arms based on reinforcement learning. The method includes: acquiring depth visual data of a wire harness to be gripped; segmenting the wire harness and determining the six-dimensional pose of each segment; establishing a six-dimensional pose graph based on the relationship between adjacent segments; determining candidate gripping poses of the robotic arm gripper from the six-dimensional pose graph; determining the deformation state of each wire harness segment based on changes in gripping position; forming action state clusters and determining confidence levels based on the candidate gripping poses, deformation states, and task completion status corresponding to different six-dimensional pose graphs; constructing reinforcement learning states and obtaining decision regret levels by combining the action state clusters, confidence levels, and the performance of candidate gripping actions; selecting direct gripping or executing the observed action based on the decision regret levels; and updating the six-dimensional pose graph and action state clusters with newly added visual and force information to achieve six-dimensional posture recognition and adaptive gripping control of the robotic arm on the flexible wire harness.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control technology, specifically a six-dimensional posture recognition and control method for robotic arms based on reinforcement learning. Background Technology

[0002] Flexible wire harnesses are easily bent, twisted, stacked, or obstructed by their own weight, external contact, and constraints from surrounding objects during automated grasping, handling, and assembly by robotic arms. Their spatial shape is highly uncertain, and the shape of other areas will change after local operations. Therefore, it is difficult for robotic arms to accurately determine the actual grasping conditions in complex wire harness scenarios based on limited current state information.

[0003] In current robotic arm operations involving wire harnesses, there is a lack of sufficient assessment of obscured areas, partially overlapping areas, and potential post-operation morphological changes. This can easily lead to discrepancies between the current identification result and the actual overall state of the wire harness. Furthermore, when the current state information is uncertain, there is a lack of a mechanism to effectively measure different operational outcomes and the reliability of decisions. This makes it difficult to reasonably determine whether current conditions are suitable for direct operation, easily resulting in problems such as unreasonable clamping positions, excessive bending or pulling of the wire harness, target position deviations, and repeated trials. These issues affect the stability and operational efficiency of the robotic arm when handling flexible wire harnesses. Therefore, it is necessary to address the problems of incomplete local observation of flexible wire harnesses and the uncertainty of deformation after clamping, which leads to unreliable clamping decisions by the robotic arm.

[0004] To address this, a six-dimensional posture recognition and control method for robotic arms based on reinforcement learning is proposed. Summary of the Invention

[0005] This invention aims to provide a six-dimensional posture recognition and control method for robotic arms based on reinforcement learning. By recognizing and controlling the six-dimensional posture of the robotic arm, the method improves the robotic arm's ability to judge the spatial state of the wire harness and the gripping result under complex flexible wire harness conditions, and implements corresponding gripping control according to the reliability of the current state information.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A reinforcement learning-based method for six-dimensional posture recognition and control of robotic arms includes: Acquire depth visual data of the wire bundle to be gripped, divide the wire bundle into multiple wire bundle segments along the extension direction of the wire bundle, determine the six-dimensional pose of each wire bundle segment, and establish a six-dimensional pose map based on the connection relationship between adjacent wire bundle segments. Based on the six-dimensional pose diagram, the candidate gripping pose corresponding to the robotic arm gripper is determined, and the deformation state of each segment of the wire harness is determined based on the change in gripping position. Based on the candidate gripping pose and deformation state corresponding to different six-dimensional pose diagrams, the six-dimensional pose diagrams that can complete the target task by using the same gripping action are grouped into the same action state cluster, and the confidence level of each action state cluster is determined. Based on the action state cluster, confidence level, and task performance of candidate grabbing actions, a reinforcement learning state is constructed, and the decision regret of the current grabbing action is determined. When the decision regret degree meets the direct gripping condition, the six-dimensional control action of the gripper is output to perform gripping; when the decision regret degree does not meet the direct gripping condition, the robotic arm is controlled to perform observation actions to obtain new visual and force information, and the six-dimensional pose graph and action state cluster are updated according to the new information until the direct gripping condition is met.

[0007] Preferably, establishing the six-dimensional pose map includes: acquiring depth visual data, which includes a depth image of the wire bundle and a corresponding three-dimensional point cloud; extracting the spatial centerline of the wire bundle based on the three-dimensional point cloud, and dividing the wire bundle into multiple continuous wire bundle segments along the spatial centerline according to the wire bundle extension direction; determining the center position and extension direction of each wire bundle segment, acquiring a point cloud located on the periphery of the wire bundle segment, and determining the radial reference direction of the wire bundle segment based on the distribution of the peripheral point cloud relative to the center position of the wire bundle segment and in conjunction with a preset spatial reference direction; determining the six-dimensional pose of each wire bundle segment based on the center position, extension direction, and radial direction; calculating the position and attitude changes between adjacent wire bundle segments, using the position and attitude changes as the connection relationship between adjacent wire bundle segments, and representing the bending and twisting state of the wire bundle with the six-dimensional pose of each wire bundle segment and the change relationship between adjacent segments.

[0008] Preferably, determining the candidate gripping pose corresponding to the robotic arm gripper based on the six-dimensional pose graph includes: Based on the spatial position and extension direction of each wire harness segment in the six-dimensional pose diagram, the gripping position and gripping direction of the robotic arm gripper relative to the corresponding wire harness segment are determined. Based on the gripping direction, the spatial posture of the gripper when the two gripping parts of the gripper approach the wire harness segment from both sides is determined, and the six-dimensional pose of the gripper that meets the gripping requirements is determined in combination with the opening and closing range of the gripper. The accessibility of the six-dimensional pose of the gripper is screened according to the range of motion of the robotic arm, and collision judgment is performed according to the working environment and the spatial position of adjacent wire harness segments in the six-dimensional pose diagram. The six-dimensional poses of the gripper that are unreachable or have collisions are eliminated to obtain the candidate gripping poses.

[0009] Preferably, determining the deformation state of each segment of the wire harness based on changes in the clamping position includes: For the candidate gripping pose, the corresponding gripped wire harness segment is determined. The position and orientation changes of the gripped wire harness segment are determined according to the gripping position changes of the robotic arm gripper. Based on the connection relationship and relative orientation change constraints between adjacent wire harness segments in the six-dimensional pose diagram, the position and orientation changes of each wire harness segment are determined sequentially from the gripped wire harness segment to the adjacent wire harness segments to obtain the spatial position and orientation of each wire harness segment after the change. Based on the spatial position and orientation of each wire harness segment after the change, it is determined whether the wire harness has undergone bending exceeding a preset bending degree or whether there is compression between adjacent wire harness segments. The determination results and the six-dimensional pose of each wire harness segment after the change are determined as the deformation state of each segment of the wire harness.

[0010] Preferably, determining the action state cluster specifically includes: for occluded wire bundle regions in the visual data, generating possible six-dimensional pose maps based on existing visual information; applying the same candidate gripping action to each of the six-dimensional pose maps respectively, determining the deformation state of each wire bundle segment based on the corresponding candidate gripping pose and gripping position change; comparing the spatial position and posture changes of each wire bundle segment, the bending and compression of the wire bundle as a whole after applying the same gripping action to each six-dimensional pose map, and comparing the gripping stability and target position arrival of the corresponding gripping action; when different six-dimensional pose maps correspond to the same gripping action and can all meet the target task requirements, they are classified into the same action state cluster, and the confidence of the action state cluster is determined based on the recognition confidence.

[0011] Preferably, determining the decision regret includes: determining the task effect based on the clamping stability, harness deformation, and target position arrival status of the candidate clamping actions corresponding to each action state cluster; determining the expected task effect of the current candidate clamping action based on the confidence level of each action state cluster; further determining the optimal task effect when each action state is clear, and obtaining the expected optimal task effect based on the corresponding confidence level; calculating the difference between the expected optimal task effect and the expected task effect of the current candidate clamping action, and determining the difference as the decision regret.

[0012] Preferably, when the decision regret degree meets the direct gripping condition, the candidate gripping pose with the highest expected task effect is selected as the target gripping pose, and the robotic arm is controlled to move to the target gripping pose and close the gripper; during the gripping process, the force information of the end of the robotic arm is acquired, and when the wire harness is detected to be subjected to a continuously increasing tension, the robotic arm is controlled to make a small-range displacement adjustment along the force direction of the wire harness to reduce the tension on the wire harness.

[0013] Preferably, when the decision regret does not meet the direct gripping condition, the target observation position is selected based on the influence of different observation positions on the action state cluster differentiation results; the robotic arm moves the vision sensor to collect new local visual data of the wire harness, the gripper is controlled to move the obstructed wire harness segment with low contact force, and the position change and contact feedback of the wire harness before and after the movement are obtained; the vertical relationship and connection relationship between the obstructed wire harness segments are determined based on the position change and contact feedback, the six-dimensional pose of the corresponding wire harness segment and the six-dimensional pose diagram are updated, and the action state cluster and its confidence are re-determined.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By continuously characterizing the spatial attitude, local morphology, and changes in adjacent areas of the flexible wire harness, the robotic arm can obtain more continuous spatial state information when facing complex states such as bending, twisting, occlusion, and local stacking. This reduces attitude judgment bias caused by relying on a single observation result, provides a more stable state basis for gripping position selection, attitude planning, and subsequent motion control, and improves the adaptability of gripping control in complex wire harness scenarios.

[0015] 2. By comprehensively considering the task effect and reliability under different possible states during the gripping decision process, and choosing to directly execute or supplement the observation based on the decision result, the robotic arm can adjust the control mode according to the sufficiency of the current information, reducing the risk of misgrabbing, pulling or positioning deviation caused by directly implementing gripping when there is occlusion or uncertainty in state recognition, while avoiding unnecessary repeated observation when the state is relatively clear.

[0016] 3. Through the coordinated efforts of six-dimensional pose graph construction, candidate gripping pose screening, harness deformation state determination, action state cluster and decision regret evaluation and supplementary observation, the harness state recognition results can be applied to the gripping decision and execution process, and the harness state and decision basis can be updated in reverse using the information obtained later, thereby reducing the risk of harness deformation in robotic arm gripping in complex harness scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the steps of the six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning, as described in this invention. Figure 2 This is a flowchart illustrating the sub-process for calculating the regret degree of intelligent agent decision-making in this invention. Figure 3 This is a schematic diagram of the adaptive execution control state transition process of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figures 1 to 3 This invention provides a six-dimensional posture recognition and control method for robotic arms based on reinforcement learning, referring to... Figure 1 Step-by-step flowchart Figure 2 A flowchart illustrating the agent's decision regret calculation sub-process, and Figure 3 The flowchart of adaptive execution control state transition is shown below, and the technical solution is as follows: First, establishing the six-dimensional pose map includes: acquiring depth visual data, which includes the depth image of the wire bundle and the corresponding three-dimensional point cloud; extracting the spatial centerline of the wire bundle based on the three-dimensional point cloud, and dividing the wire bundle into multiple continuous wire bundle segments along the spatial centerline according to the wire bundle extension direction; determining the center position and extension direction of each wire bundle segment, acquiring the point cloud located on the periphery of the wire bundle segment, and determining the radial reference direction of the wire bundle segment based on the distribution of the peripheral point cloud relative to the center position of the wire bundle segment and in combination with a preset spatial reference direction; establishing a local coordinate system of the wire bundle segment based on the center position, extension direction, and radial reference direction, and determining the six-dimensional pose of each wire bundle segment; calculating the position change and attitude change between adjacent wire bundle segments, using the position change and attitude change as the connection relationship between adjacent wire bundle segments, and representing the bending and twisting state of the wire bundle with the six-dimensional pose of each wire bundle segment and the change relationship between adjacent segments.

[0020] In this embodiment, a depth camera is mounted on the end of the robotic arm. Before the gripping action is performed, it captures an image of the area where the wire harness to be gripped is located, and outputs a depth image of the area and the corresponding 3D point cloud data. The depth image reflects the distance information of each point on the wire harness surface relative to the camera, and the 3D point cloud data reflects the 3D coordinates of each point on the wire harness surface in the spatial coordinate system. For example, the depth image output by the depth camera can have a resolution of 640×480. The acquired depth image and 3D point cloud data are sent to the controller via the data transmission interface and stored in the controller's storage module for subsequent processing modules to call. The processor performs background and noise point filtering on the stored 3D point cloud data, retaining the valid point cloud data located on the wire harness surface as input data for subsequent centerline extraction and segmentation.

[0021] Specifically, the processor performs skeletonization processing on the filtered 3D point cloud data to obtain a spatial centerline that reflects the overall extension direction of the wire bundle. Along this spatial centerline, the wire bundle is divided into multiple consecutive wire bundle segments according to its extension direction. For example, a wire bundle with a total length of about 1.2 meters is divided into 40 consecutive segments along the centerline. The length of each segment along the centerline is approximately equal. Each segment is stored in the storage module in the form of a segment number and the corresponding centerline coordinate interval, which serves as the basic data for subsequent segment pose calculations.

[0022] Specifically, for each wire bundle segment, the processor calculates the geometric center of the point cloud data falling within the corresponding centerline interval of the segment, determines this center position of the wire bundle segment, and determines the extension direction of the wire bundle segment based on the direction of the line connecting the centerline coordinates of the beginning and end ends of the segment. Simultaneously, it extracts the point cloud data located on the periphery of the wire bundle segment, and determines the radial reference direction of the wire bundle segment based on the distribution of the peripheral point cloud relative to the center position in the radial cross-section and in conjunction with a preset spatial reference direction. For example, the direction of the line connecting the relatively densely distributed peripheral point cloud relative to the center position can be determined as the radial reference direction, and this radial reference direction can be used as the radial attitude reference for establishing the local coordinate system of the wire bundle segment.

[0023] Specifically, the center position is taken as the origin of the coordinate system, the extension direction is taken as the first coordinate axis direction, the radial reference direction is taken as the second coordinate axis direction, and the third coordinate axis direction is determined according to the preset coordinate system determination rules. The local coordinate system of the wire harness segment is established. The three-dimensional position and three-dimensional attitude angle of the local coordinate system relative to the robot arm base coordinate system are combined to determine the six-dimensional pose of the wire harness segment. The six-dimensional poses of each wire harness segment are stored in the storage module with the segment number as the index to form an initial pose data set.

[0024] Specifically, the processor sequentially reads the six-dimensional poses of two adjacent wire harness segments, calculates the positional change between their center positions and the attitude change between their local coordinate systems, and determines the positional and attitude changes as the connection relationships between the two adjacent wire harness segments. Then, according to the wire harness extension direction, the six-dimensional poses of each wire harness segment and the connection relationships between adjacent segments are sequentially associated to form the six-dimensional pose diagram. The positional and attitude changes of adjacent segments together serve as characterization data for the local bending and twisting states of the wire harness. When the attitude change angle between two adjacent segments exceeds a preset angle threshold, the region is marked as having a significant attitude change.

[0025] By segmenting the wire harness and establishing the six-dimensional pose of each segment and the connection relationship between adjacent segments, the bending and twisting states of the wire harness can be characterized with continuous spatial data, which helps to improve the precision and continuity of the pose recognition results of complex wire harnesses.

[0026] Through the processing flow of depth vision data acquisition, centerline extraction, segment division, local coordinate system establishment, and calculation of the connection relationship between adjacent segments, the overall spatial morphology of the line bundle is transformed into storable and callable structured six-dimensional pose graph data, providing a relatively complete data foundation for subsequent candidate gripping pose determination and deformation state judgment.

[0027] Further, determining the candidate gripping pose corresponding to the robotic arm gripper based on the six-dimensional pose diagram includes: determining the gripping position and gripping direction of the robotic arm gripper relative to the corresponding wire harness segment based on the spatial position and extension direction of each wire harness segment in the six-dimensional pose diagram; determining the spatial posture of the gripper when the two gripping parts of the gripper approach the wire harness segment from both sides based on the gripping direction, and determining the six-dimensional pose of the gripper that meets the gripping requirements based on the opening and closing range of the gripper; performing accessibility screening on the six-dimensional pose of the gripper based on the movement range of the robotic arm, and performing collision judgment based on the working environment and the spatial position of adjacent wire harness segments in the six-dimensional pose diagram, eliminating the six-dimensional poses of the gripper that are unreachable or have collisions, and obtaining the candidate gripping pose.

[0028] Specifically, the processor retrieves the six-dimensional pose map from the storage module and sequentially reads the center position coordinates and extension direction data of each wire bundle segment. For each wire bundle segment, the center position of the segment is used as a reference point, and the gripping position of the robotic arm gripper relative to the segment is determined in combination with the extension direction of the segment. The gripping position is generally selected in the area near the center position of the segment. At the same time, the direction perpendicular to the extension direction of the segment is used as the gripping direction of the gripper. For example, for a wire bundle segment whose extension direction is along a certain horizontal axis, the corresponding gripping direction is determined to be a direction approximately perpendicular to that axis. After the gripping position and gripping direction data are determined, they are stored in the storage module for subsequent gripper posture calculation.

[0029] Specifically, based on the determined clamping direction, the processor calculates the corresponding spatial pose of the clamp when the two clamping parts of the clamp approach the wire harness segment from opposite sides along opposite paths. That is, the approach directions of the two clamping parts are collinear with the clamping direction and opposite in direction. At the same time, it reads the preset clamp opening and closing range parameters, such as the maximum opening and closing width of the clamp being 45 mm, and determines whether the outer diameter of the wire harness segment at the clamping position is within the opening and closing range. If the outer diameter is within the opening and closing range, the corresponding clamp spatial pose is combined with the clamping position to determine the clamp six-dimensional pose that meets the clamping requirements. If the outer diameter exceeds the opening and closing range, the clamp six-dimensional pose corresponding to the wire harness segment is discarded.

[0030] Specifically, the processor reads the preset motion range parameters of each joint of the robotic arm, performs kinematic calculations on each gripper six-dimensional pose that meets the gripping requirements, and determines whether the joint angles required by each joint of the robotic arm to reach the corresponding gripper six-dimensional pose are all within the preset motion range. Gripper six-dimensional poses with joint angles exceeding the motion range are marked as unreachable poses and discarded. For example, if the required rotation angle of a certain joint corresponding to a certain gripper six-dimensional pose exceeds the preset maximum rotation angle of the joint, then the gripper six-dimensional pose is determined to be unreachable.

[0031] Specifically, the processor, for each gripper six-dimensional pose retained after accessibility filtering, combines the pre-set three-dimensional model data of the working environment and the spatial position data of adjacent wire bundle segments in the six-dimensional pose diagram to determine whether the robotic arm gripper and its connecting parts overlap with fixed objects in the working environment or with the spatial position of adjacent wire bundle segments in the motion path to the corresponding gripper six-dimensional pose. The gripper six-dimensional poses that are determined to overlap are marked as collision poses and eliminated. The six-dimensional poses of each gripper retained after accessibility filtering and collision judgment are determined as the candidate gripping poses and stored in the storage module for subsequent deformation state determination and action state cluster division and calling.

[0032] In this embodiment, the subsequent candidate gripping action refers to the gripping control process corresponding to the target gripping pose of the gripper. The processor uses the candidate gripping pose as an index to associate, store, and retrieve the harness deformation state, task effect, and decision regret under the corresponding candidate gripping action.

[0033] By combining the gripper's opening and closing range, the robotic arm's range of motion, and the spatial relationship of the working environment to screen the six-dimensional pose of the gripper layer by layer, the possibility of unreachable or collision situations during subsequent gripping can be reduced, thereby improving the usability of candidate gripping pose data.

[0034] By employing a layered processing flow that includes determining the gripping position and orientation, calculating the spatial posture of the gripper, filtering accessibility, and determining collisions, the spatial geometric relationship between the robotic arm gripper and the wire harness segment, as well as the robotic arm's own motion constraints, are incorporated into the process of determining the candidate gripping pose. This provides a pose data foundation that has undergone multiple condition checks for the subsequent gripping action.

[0035] Further, determining the deformation state of each segment of the wire harness based on the change in gripping position includes: determining the corresponding clamped wire harness segment for the candidate gripping pose; determining the position and orientation changes of the clamped wire harness segment based on the change in gripping position of the robotic arm gripper; determining the position and orientation changes of each wire harness segment sequentially from the clamped wire harness segment to the adjacent wire harness segments based on the connection relationship and relative orientation change constraints between adjacent wire harness segments in the six-dimensional pose diagram, thereby obtaining the spatial position and orientation of each wire harness segment after the change; determining whether the wire harness has undergone bending exceeding a preset bending degree or whether there has been compression between adjacent wire harness segments based on the spatial position and orientation of each wire harness segment after the change, and determining the determination result and the six-dimensional pose of each wire harness segment after the change as the deformation state of each segment of the wire harness.

[0036] Specifically, for each candidate gripping pose, the processor determines the corresponding gripped wire harness segment in the six-dimensional pose graph based on the gripping position corresponding to that candidate gripping pose. When evaluating the deformation state of the corresponding candidate gripping action, the processor determines the position and orientation changes of the gripped wire harness segment under the action of that candidate gripping action, based on the position and orientation changes corresponding to that action, and uses these position and orientation changes as the initial input for determining the deformation state of adjacent wire harness segments. For example, when the candidate gripping action moves the gripped wire harness segment 20 mm in a certain direction accompanied by an orientation deflection, the corresponding position and orientation changes are stored in the storage module.

[0037] Specifically, the processor retrieves connection relationship data between the clamped wire harness segment and its adjacent wire harness segments from the storage module. This connection relationship data includes the relative pose change constraint range between adjacent segments. Using the position and pose changes of the clamped wire harness segment as initial inputs, within the relative pose change constraint range, the processor estimates the position and pose changes segment by segment along the wire harness extension direction from the clamped wire harness segment to the adjacent wire harness segments on both sides. This estimation process is then passed along the wire harness extension direction to the remaining wire harness segments, thereby obtaining the possible spatial position and pose data of each wire harness segment after changes corresponding to the current candidate clamping action.

[0038] Specifically, the processor compares the spatial position and attitude data of each wire harness segment after the change with the six-dimensional pose map before the change. For two adjacent wire harness segments, it calculates the attitude change angle between them after the change. When the angle exceeds a preset bending degree threshold, it determines that the wire harness in that area has bent beyond the preset bending degree. At the same time, it calculates the spatial distance between adjacent wire harness segments after the change. When the distance is less than a preset minimum spacing threshold, it determines that the adjacent wire harness segments have been squeezed. For example, the preset bending degree threshold can be 60 degrees and the preset minimum spacing threshold can be 3 millimeters.

[0039] Specifically, the processor combines the bending and compression judgment results with the six-dimensional pose data of each wire harness segment after its transformation to determine the deformation state of each wire harness segment corresponding to the candidate gripping pose. This deformation state data is then stored in the storage module for subsequent action state cluster division and decision regret calculation. The spatial position and orientation of each wire harness segment after its transformation are used to form the deformation state evaluation result corresponding to the candidate gripping action. This evaluation result is used to compare the possible bending and compression situations of the wire harness under different candidate gripping actions and serves as data input for subsequent action state cluster division and task performance evaluation.

[0040] By transmitting and calculating position and attitude changes segment by segment along the extension direction of the harness, the potential deformation of the entire harness can be known in advance before the gripping position changes, providing a reference for the comparison and selection of subsequent gripping poses.

[0041] By employing a processing flow that determines the changes in the clamped segment, calculates the changes in adjacent segments step by step, and judges bending and squeezing, the impact of local clamping actions on the overall shape of the harness is transformed into quantifiable deformation state data. This provides a relatively complete basis for deformation judgment for subsequent action state cluster division and decision regret evaluation.

[0042] Further, determining the action state cluster specifically includes: for occluded wire harness regions in the visual data, generating possible six-dimensional pose maps based on existing visual information; applying the same candidate gripping action to each of the six-dimensional pose maps respectively, determining the deformation state of each wire harness segment based on the corresponding candidate gripping pose and gripping position change; comparing the spatial position and posture changes of each wire harness segment, the bending and compression of the entire wire harness, and the gripping stability and target position arrival status of the corresponding gripping action after applying the same gripping action in each six-dimensional pose map; when different six-dimensional pose maps correspond to the same gripping action and can all meet the target task requirements, they are classified into the same action state cluster, and the confidence of the action state cluster is determined based on the recognition confidence.

[0043] Specifically, for areas in depth vision data where point clouds are missing due to overlapping of line bundles or obstruction by external objects, the processor reads the valid visual information already acquired around the occluded area, including the position, extension direction, and connection relationship data of line bundle segments on both sides of the occluded area. Following the continuity assumption rule, and using the center position, extension direction, and connection relationship of adjacent visible line bundle segments on both sides of the occluded area as constraints, the processor interpolates and completes the position and orientation of line bundle segments within the occluded area, while satisfying the pose change constraint range for adjacent segments. Furthermore, by discretely selecting multiple sets of different values ​​for the curvature and twist of the completed path within the pose change constraint range, multiple six-dimensional pose diagrams with different orientation assumptions are obtained, all satisfying the known connection relationship on both sides. For example, for a certain occluded area, three different sets of curvature and twist values ​​can be selected in the above manner to generate three sets of six-dimensional pose diagrams with different orientation assumptions. Each pose diagram contains complete line bundle segment position and orientation data and is stored in the storage module.

[0044] Specifically, the processor retrieves the gripping position and gripping direction parameters corresponding to the same candidate gripping action from the storage module, substitutes the candidate gripping action into each of the previously generated six-dimensional pose diagrams, and calculates the spatial position and orientation of each wire segment after the change under the action of the candidate gripping action in each six-dimensional pose diagram according to the aforementioned deformation state determination method, thereby obtaining the deformation state data corresponding to each six-dimensional pose diagram. The deformation state data of each group is stored separately with the pose diagram number as the index.

[0045] Specifically, the processor compares the deformation state data corresponding to each group of six-dimensional pose diagrams one by one, comparing the spatial position and attitude change range of each segment of the harness in each group of data, whether the harness as a whole has bent beyond the preset bending degree, and whether there is compression between adjacent segments. At the same time, it determines the gripping stability by combining the force balance of the robotic arm gripper under the corresponding gripping action, and determines the target position arrival status based on the deviation between the position of the gripped segment and the preset target position after gripping. The six-dimensional pose diagrams that meet the preset task requirements in terms of both gripping stability and target position arrival status are selected.

[0046] Specifically, the processor determines whether multiple sets of six-dimensional pose graphs that meet the task requirements after screening correspond to the same candidate gripping action. If so, the multiple sets of six-dimensional pose graphs are grouped into the same action state cluster. An action state cluster can contain more than two sets of six-dimensional pose graphs with different orientation assumptions but which can all achieve the task requirements through the same gripping action. The processor further reads the completeness of the visual information when generating each set of six-dimensional pose graphs as the basis for recognition confidence. Based on the number of pose graphs grouped into the same action state cluster and the comprehensive distribution of their respective recognition confidence, the processor determines the confidence value of the action state cluster. For example, the confidence value corresponding to a certain action state cluster is 0.72. After the confidence value is determined, it is stored in the storage module along with the corresponding action state cluster for subsequent decision regret calculation.

[0047] By categorizing pose maps of multiple possible directions under occlusion based on whether the same gripping action can meet the task requirements, and quantifying them in conjunction with recognition confidence, a state classification basis with confidence level annotation can be provided for subsequent decision-making when local visual information is incomplete.

[0048] By generating multiple hypothetical pose diagrams of occluded regions, comparing deformation states group by group under the same gripping action, and determining the confidence of action state clusters, the uncertain local states of the harness are transformed into structured cluster partitioning data with confidence labels, providing a quantitative basis for subsequent decision regret calculation and selection of gripping or observation actions.

[0049] Further, determining the decision regret includes: determining the task effect based on the gripping stability, harness deformation, and target position arrival status of the candidate gripping actions corresponding to each action state cluster; determining the expected task effect of the current candidate gripping action based on the confidence level of each action state cluster; determining the optimal task effect when each action state is clear, and obtaining the expected optimal task effect based on the corresponding confidence level; calculating the difference between the expected optimal task effect and the expected task effect of the current candidate gripping action, and determining the difference as the decision regret.

[0050] The expected performance evaluation network is used to correct the expected task performance obtained by weighting by confidence level based on the correspondence between the actual task performance and confidence level and task performance score reflected in historical gripping and observation records, so as to make up for the nonlinear deviation that may exist between the confidence score and the actual task completion performance. When the processor calculates the expected task performance and the expected optimal task performance of the current candidate gripping action, it uses the same trained network and processes the two types of input data in the same way to keep the corrections introduced by the two consistent, thereby ensuring that the decision regret determined by the difference between the two can reflect the relative gap between the task performance of the current candidate gripping action and the optimal action under ideal conditions.

[0051] For each candidate grasping action corresponding to an action state cluster, assuming the actual bundle state corresponding to that action state cluster is known, the processor compares the task performance scores of all candidate grasping actions corresponding to their respective action state clusters under the actual state. It then selects the candidate grasping action with the highest task performance score and determines this highest score as the optimal task performance under the actual state. Next, the optimal task performance corresponding to each action state cluster is weighted and combined with its own confidence level, and input into the aforementioned expected performance evaluation network for processing. This yields the expected optimal task performance value after considering the confidence distribution of each action state cluster, as well as the expected optimal task performance value between the grasped fragment position and the preset target position after grasping. The deviation data, including the force balance data, bending and compression judgment results, and position deviation data, are normalized to a clamping stability score, a harness deformation score, and a target position arrival score, respectively, within their preset value ranges from 0 to 1. The closer the value is to 1, the more ideal the performance in the corresponding aspect. Then, the three scores are weighted and summed according to preset weights, and the sum is determined as the task effect of the candidate clamping action corresponding to the action state cluster. For example, the preset weights for the clamping stability score, harness deformation score, and target position arrival score can be 0.4, 0.3, and 0.3, respectively. The specific weights can be adjusted according to the actual situation. The comprehensive task effect score for the candidate clamping action corresponding to a certain action state cluster is 0.81.

[0052] Specifically, the processor reads the task performance scores corresponding to the same candidate grasping action under different action state clusters, as well as the pre-determined confidence values ​​of each action state cluster. The task performance scores and their corresponding confidence values ​​of each action state cluster are used as a set of input data and input to a pre-trained expected performance evaluation network. The expected performance evaluation network includes an input layer, several fully connected hidden layers, and an output layer. The input layer receives a feature vector composed of the task performance score and the confidence value. The hidden layer performs weighted fusion of the feature vector layer by layer. The output layer outputs the expected task performance value of the current candidate grasping action after comprehensively considering the confidence distribution of each action state cluster. After being determined, this value is stored in the storage module as one of the input data for subsequent difference calculation.

[0053] Specifically, for each action state cluster, assuming that the actual bundle state corresponding to the action state cluster is known, the processor selects the candidate gripping action with the highest task effect score from the candidate gripping actions contained in the action state cluster, determines the highest task effect score as the optimal task effect corresponding to the action state cluster, and then weights and combines the optimal task effect of each action state cluster with its corresponding confidence score, inputs it into the aforementioned expected effect evaluation network for processing, obtains the expected optimal task effect value after integrating the confidence distribution of each action state cluster, and stores the value in the storage module.

[0054] Specifically, the processor retrieves the expected task effect value and the expected optimal task effect value of the current candidate grasping action from the storage module, calculates the difference between the two, and determines the difference as the decision regret value corresponding to the current candidate grasping action. The smaller the decision regret value, the smaller the task effect gap between the current candidate grasping action and the optimal action under ideal conditions. After the decision regret value is determined, it is stored in the storage module for subsequent judgment on whether the direct grasping condition is met.

[0055] By weighting the performance of candidate grabbing actions under different action state clusters according to confidence level to obtain the expected task performance, and comparing it with the expected optimal task performance when the assumed state is clear, a reference with a certain degree of quantification can be provided for whether it is appropriate to directly implement grabbing at the present time.

[0056] The aforementioned expected performance evaluation network uses the task performance scores marked in historical gripping and observation records as training labels during the training phase, and the corresponding candidate gripping action task performance scores and confidence scores as training inputs. Mean squared error is used as the loss function to measure the deviation between the network output and the labeled labels, and the Adam optimizer is used to iteratively update the network parameters. The network parameters and intermediate feature data during the training process are stored in the controller's storage module. The network inference calculation is completed by the processor inside the controller. The processor calls the trained network parameters in real time based on the currently collected and stored action state cluster data to calculate the expected task performance and the expected optimal task performance. The calculation results are then written back to the storage module for subsequent decision regret judgment and gripping or observation action selection process. The entire data flow process is completed in a closed loop within the same controller without relying on external computing devices.

[0057] By employing a process that involves determining the task effect, calculating the expected task effect, calculating the expected optimal task effect, and determining the decision regret value based on the difference between the two, the uncertainty of each action state cluster is incorporated into the task effect evaluation in a confidence-weighted manner and transformed into a quantifiable and comparable decision regret value. This provides a quantitative basis with a certain degree of objectivity for subsequent judgments on whether the direct grab condition is met, and thus for selecting direct grab or supplementary observation actions, which helps to reduce the risk of grab misjudgment caused by state uncertainty.

[0058] Furthermore, when the decision regret degree meets the direct gripping condition, the candidate gripping pose with the highest expected task effect is selected as the target gripping pose, and the robotic arm is controlled to move to the target gripping pose and close the gripper; during the gripping process, the force information of the end of the robotic arm is acquired, and when the wire harness is detected to be subjected to a continuously increasing tension, the robotic arm is controlled to make a small-range displacement adjustment along the force direction of the wire harness to reduce the tension on the wire harness.

[0059] Specifically, the processor determines whether the decision regret value corresponding to the current candidate gripping action is less than a preset regret value threshold. When the decision regret value is less than the preset threshold, it is determined that the direct gripping condition is met. Then, it retrieves the expected task effect values ​​corresponding to all candidate gripping poses from the storage module, and filters out the candidate gripping pose with the highest expected task effect value by comparing the values. The candidate gripping pose is then determined as the target gripping pose. For example, the preset regret value threshold can be 0.15. After the target gripping pose data is determined, it is stored in the storage module for subsequent motion control to call.

[0060] Specifically, the processor calculates the joint angle increment required for each joint of the robotic arm to move from its current state to the target gripping pose based on the position coordinates and attitude angle data contained in the target gripping pose, combined with the actual angles of each joint of the robotic arm at present. It then generates the motion path of each joint according to the preset motion trajectory planning rules. The controller sends control commands to the drive devices of each joint of the robotic arm according to the motion path, driving the robotic arm to move along the motion path to the target gripping pose. After the robotic arm reaches the target gripping pose, the controller sends a closing command to the drive component of the gripper, controlling the two gripping parts of the gripper to move towards each other to complete the closed gripping.

[0061] Specifically, the force sensor installed between the end effector of the robotic arm and the gripper continuously collects force data at the contact point between the gripper and the wire harness during the gripping process, and sends the force data to the controller in real time via the data transmission interface. The processor compares the continuously received force data according to the acquisition sequence to determine whether the force value shows a continuously increasing trend. When it is determined that the force value continuously increases and exceeds the preset tension threshold in multiple consecutive acquisition moments, it is determined that the wire harness is subjected to a continuously increasing tension, for example, the preset tension threshold can be 8 Newtons.

[0062] Specifically, the processor determines the direction of the tension on the wire harness relative to the gripper of the robotic arm based on the force direction data in the force information, and generates a small-range displacement adjustment command according to the direction of the force. The processor controls the robotic arm to make a small-amplitude adjustment of the gripper position along the direction of the force. For example, the displacement of a single adjustment can be 2 mm. After the adjustment, the processor continues to read the force data collected by the force sensor and determines whether the tension on the wire harness has decreased to below the preset tension threshold. If it is still in a state of continuous increase, the small-range displacement adjustment is repeated in the same way until the tension on the wire harness no longer continues to increase.

[0063] By selecting the target gripping pose based on the desired task effect when the direct gripping conditions are met, and by making small-range displacement adjustments in conjunction with force information during the gripping process, the force state of the wire harness can be tracked and adjusted to a certain extent while the gripping action is completed, reducing the possibility of the wire harness being excessively stretched during the gripping process.

[0064] The determination of the aforementioned continuous upward trend of the force value can be achieved through a pre-trained force trend recognition network. The input of this network is a sequence of force values ​​from multiple acquisition times arranged in chronological order. After receiving the sequence data, the input layer processes it through several layers of temporal feature extraction units. The output layer outputs the judgment result of whether the force shows a continuous increasing trend. During the network training phase, force sequence samples labeled as "continuously increasing" or "not continuously increasing" during historical clamping processes are used as training data. Cross-entropy is used as the loss function to measure the deviation between the network output and the labeled results, and the Adam optimizer is used to iteratively update the network parameters. The inference calculation of this network is completed by the processor inside the controller. The raw data, network parameters, and judgment results collected by the force sensor are all stored in the controller's storage module. The processor reads the latest force data from the storage module according to the preset acquisition cycle and calls the trained network for real-time judgment. The judgment result is then used to generate displacement adjustment commands. The entire data flow is completed in a closed loop inside the controller.

[0065] By combining target gripping pose screening, robotic arm motion control, force monitoring during gripping, and small-range displacement adjustment based on the force direction, the force state of the wire harness during the gripping action is incorporated into the real-time feedback adjustment range. This provides a way to control the robotic arm under direct gripping conditions that takes into account both task effectiveness and wire harness force, and helps to reduce the degree of tension on the wire harness while completing the gripping.

[0066] Furthermore, when the decision regret does not meet the direct gripping condition, the target observation position is selected based on the influence of different observation positions on the action state cluster differentiation results; the robotic arm moves the vision sensor to collect new local visual data of the wire harness, and the gripper is controlled to move the occluded wire harness segment with low contact force, and the position change and contact feedback of the wire harness before and after the movement are obtained; the vertical relationship and connection relationship between the occluded wire harness segments are determined based on the position change and contact feedback, the six-dimensional pose of the corresponding wire harness segment and the six-dimensional pose diagram are updated, and the action state cluster and its confidence are re-determined.

[0067] Specifically, the processor determines whether the decision regret degree corresponding to the current candidate gripping action is less than a preset regret degree threshold. When the decision regret degree is not less than the preset threshold, it is determined that the direct gripping condition is not met. Then, it retrieves multiple candidate observation positions around the occluded area from the storage module. For each candidate observation position, it simulates the degree of differentiation of the existing action state clusters after the visual sensor collects visual data from that position. That is, it compares the number of action state clusters that the candidate observation position can eliminate the ambiguity of differentiation. The candidate observation position with the highest degree of differentiation is determined as the target observation position. For example, if a candidate observation position can distinguish two groups of six-dimensional pose maps that originally belong to the same action state cluster, the candidate observation position is selected as the target observation position.

[0068] Specifically, the processor performs kinematics calculations based on the spatial coordinates corresponding to the target observation position and the current posture of the robotic arm, generating a motion path that drives the robotic arm to move the vision sensor installed at its end to the target observation position. The controller sends control commands to the drive devices of each joint of the robotic arm according to the motion path, driving the robotic arm to move the vision sensor to the target observation position. After reaching the target observation position, the controller controls the vision sensor to take pictures of the area where the occluded wire bundle segment is located, obtaining the newly added local depth image and three-dimensional point cloud data of the area, and storing the newly added visual data in the storage module.

[0069] Specifically, the processor generates a control command for the control fixture to move the wire harness segment with a preset low contact force based on the spatial position of the wire harness segment within the obstructed area. The preset low contact force is used to avoid causing significant positional or orientation disturbance to the wire harness during the moving process. For example, the preset low contact force can be 2 Newtons. The controller drives one of the clamping parts of the fixture to contact and move the wire harness segment with the low contact force according to the control command. During the moving process, the force sensor installed at the end of the robotic arm continuously collects the contact feedback data between the fixture and the wire harness segment. At the same time, the vision sensor continuously collects the position change data of the wire harness segment before and after the moving. The above-mentioned contact feedback data and position change data are stored in the storage module after being collected.

[0070] Specifically, the processor retrieves the position change data and contact feedback data from the storage module. Based on the linkage change of the obscured wire harness segment with the adjacent visible wire harness segment during the tossing process, it determines the vertical stacking relationship between the obscured wire harness segment and the adjacent wire harness segment. Combining the continuity of the end positions of the wire harness segments before and after the tossing, it determines the connection relationship between the obscured wire harness segment and the adjacent wire harness segment. The determined vertical and connection relationship data are combined with the newly added visual data to update the six-dimensional pose corresponding to the obscured wire harness segment. The six-dimensional pose diagram is then corrected accordingly based on the updated six-dimensional pose and connection relationship.

[0071] Specifically, based on the updated six-dimensional pose map, the processor re-compares the deformation states of each candidate grasping action under different six-dimensional pose maps according to the aforementioned action state cluster determination method, re-divides the action state clusters, and re-determines the confidence of each action state cluster based on the completeness of the visual information of each bundle segment after the update. The re-determined action state clusters and confidence data are stored in the storage module for subsequent recalculation of decision regret. The processor then determines whether the direct grasping condition is met based on the updated decision regret. If it is still not met, the processor continues to select a new target observation position and repeat the observation according to the above process.

[0072] By selecting the target observation position based on the degree of differentiation of the action state cluster, and updating the harness state by combining the contact feedback obtained from low contact force tossing, the uncertainty of the harness state can be gradually reduced with a small contact disturbance when the decision regret does not meet the direct gripping condition.

[0073] The evaluation of the discriminative power of the aforementioned observation positions on action state clusters can be achieved through a pre-trained observation effect prediction network. The input of this network is the difference features between the spatial coordinates of the candidate observation positions and the pose graphs of the current action state clusters. After receiving the above feature combinations, the input layer performs fusion processing through several hidden layers, and the output layer outputs the predicted discriminative power value corresponding to the candidate observation position. During the network training phase, the actual discriminative power achieved by different observation positions in historical observation records is used as the training label. Mean squared error is used as the loss function, and the Adam optimizer is used to iteratively update the network parameters. The network parameters, candidate observation position features, and prediction results are all stored in the controller's storage module. The prediction calculation is completed by the processor inside the controller. The processor calls the predicted values ​​of each candidate observation position output by the trained network based on the real-time data of the current occluded area in the storage module, and completes the selection of the target observation position accordingly. The entire data flow is completed in a closed loop inside the controller.

[0074] By combining target observation location selection, visual data supplementation and acquisition, low-contact force to obtain contact feedback, and feedback-based updates of the six-dimensional pose map and motion state cluster, the robotic arm can reduce the uncertainty of the harness state by actively supplementing observations when the state information is insufficient. This provides updated state basis for recalculating decision regret and judging subsequent gripping or continued observation, which helps to reduce repeated trials caused by insufficient information.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning, characterized in that, include: Acquire depth visual data of the wire bundle to be gripped, divide the wire bundle into multiple wire bundle segments along the extension direction of the wire bundle, determine the six-dimensional pose of each wire bundle segment, and establish a six-dimensional pose map based on the connection relationship between adjacent wire bundle segments. Based on the six-dimensional pose diagram, the candidate gripping pose corresponding to the robotic arm gripper is determined, and the deformation state of each segment of the wire harness is determined based on the change in gripping position. Based on the candidate gripping pose and deformation state corresponding to different six-dimensional pose diagrams, the six-dimensional pose diagrams that can complete the target task by using the same gripping action are grouped into the same action state cluster, and the confidence level of each action state cluster is determined. Based on the action state cluster, confidence level, and task performance of candidate grabbing actions, a reinforcement learning state is constructed, and the decision regret of the current grabbing action is determined. When the decision regret degree meets the direct gripping condition, the six-dimensional control action of the gripper is output to perform gripping; when the decision regret degree does not meet the direct gripping condition, the robotic arm is controlled to perform observation actions to obtain new visual and force information, and the six-dimensional pose graph and action state cluster are updated according to the new information until the direct gripping condition is met.

2. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, Establishing the six-dimensional pose map includes: acquiring depth visual data, which includes a depth image of the wire bundle and a corresponding three-dimensional point cloud; extracting the spatial centerline of the wire bundle based on the three-dimensional point cloud, and dividing the wire bundle into multiple continuous wire bundle segments along the spatial centerline according to the wire bundle extension direction; determining the center position and extension direction of each wire bundle segment, acquiring a point cloud located on the periphery of the wire bundle segment, and determining the radial reference direction of the wire bundle segment based on the distribution of the peripheral point cloud relative to the center position of the wire bundle segment and in conjunction with a preset spatial reference direction; determining the six-dimensional pose of each wire bundle segment based on the center position, extension direction, and radial direction; calculating the position and attitude changes between adjacent wire bundle segments, using the position and attitude changes as the connection relationship between adjacent wire bundle segments, and representing the bending and twisting state of the wire bundle with the six-dimensional pose of each wire bundle segment and the change relationship between adjacent segments.

3. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, Based on the six-dimensional pose graph, the candidate gripping pose corresponding to the robotic arm gripper is determined, including: Based on the spatial position and extension direction of each wire harness segment in the six-dimensional pose diagram, the gripping position and gripping direction of the robotic arm gripper relative to the corresponding wire harness segment are determined. Based on the gripping direction, the spatial posture of the gripper when the two gripping parts of the gripper approach the wire harness segment from both sides is determined, and the six-dimensional pose of the gripper that meets the gripping requirements is determined in combination with the opening and closing range of the gripper. The accessibility of the six-dimensional pose of the gripper is screened according to the range of motion of the robotic arm, and collision judgment is performed according to the working environment and the spatial position of adjacent wire harness segments in the six-dimensional pose diagram. The six-dimensional poses of the gripper that are unreachable or have collisions are eliminated to obtain the candidate gripping poses.

4. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 3, characterized in that, The determination of the deformation state of each segment of the wire harness based on changes in the clamping position includes: For the candidate gripping pose, the corresponding gripped wire harness segment is determined. The position and orientation changes of the gripped wire harness segment are determined according to the gripping position changes of the robotic arm gripper. Based on the connection relationship and relative orientation change constraints between adjacent wire harness segments in the six-dimensional pose diagram, the position and orientation changes of each wire harness segment are determined sequentially from the gripped wire harness segment to the adjacent wire harness segments to obtain the spatial position and orientation of each wire harness segment after the change. Based on the spatial position and orientation of each wire harness segment after the change, it is determined whether the wire harness has undergone bending exceeding a preset bending degree or whether there is compression between adjacent wire harness segments. The determination results and the six-dimensional pose of each wire harness segment after the change are determined as the deformation state of each segment of the wire harness.

5. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, Determining the action state cluster specifically includes: for occluded wire harness regions in the visual data, generating possible six-dimensional pose maps based on existing visual information; applying the same candidate gripping action to each of the six-dimensional pose maps, determining the deformation state of each wire harness segment based on the corresponding candidate gripping pose and gripping position changes; comparing the spatial position and posture changes of each wire harness segment, the overall bending and compression of the wire harness, and the gripping stability and target position attainment of the corresponding gripping action after applying the same gripping action in each six-dimensional pose map; when different six-dimensional pose maps correspond to the same gripping action and all can meet the target task requirements, they are classified into the same action state cluster, and the confidence of the action state cluster is determined based on the recognition confidence.

6. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, Determining the decision regret includes: determining the task effect based on the gripping stability, harness deformation, and target position arrival status of the candidate gripping actions corresponding to each action state cluster; determining the expected task effect of the current candidate gripping action based on the confidence level of each action state cluster; determining the optimal task effect when each action state is clear, and obtaining the expected optimal task effect based on the corresponding confidence level; calculating the difference between the expected optimal task effect and the expected task effect of the current candidate gripping action, and determining the difference as the decision regret.

7. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, When the decision regret degree meets the direct gripping condition, the candidate gripping pose with the highest expected task effect is selected as the target gripping pose, and the robotic arm is controlled to move to the target gripping pose and close the gripper. During the clamping process, force information is obtained from the end of the robotic arm. When the increasing tension on the wire harness is detected, the robotic arm is controlled to make a small-range displacement adjustment along the direction of the force on the wire harness to reduce the tension on the wire harness.

8. The six-dimensional posture recognition and control method for a robotic arm based on reinforcement learning according to claim 1, characterized in that, When the decision regret does not meet the direct gripping condition, the target observation position is selected based on the influence of different observation positions on the action state cluster differentiation results; the robotic arm moves the vision sensor to collect new local visual data of the wire harness, and the gripper is controlled to move the occluded wire harness segment with low contact force, and the position change and contact feedback of the wire harness before and after the movement are obtained; the vertical relationship and connection relationship between the occluded wire harness segments are determined based on the position change and contact feedback, the six-dimensional pose of the corresponding wire harness segment and the six-dimensional pose diagram are updated, and the action state cluster and its confidence are re-determined.

Images