Method for modeling a vehicle door handle and robot

Abstract

The application provides a vehicle door handle modeling method and a robot, and relates to the field of robot perception and control. The method comprises the following steps: controlling an execution component of the robot to move to a vehicle door handle at a first pose relative to the vehicle door handle; in the case that force data indicating that the execution component contacts the vehicle door handle, determining a first contact point position according to the force data and pose data of the execution component, and controlling the execution component to move along the vehicle door handle to obtain a plurality of moving contact point positions; switching the first pose and / or the path of the execution component moving along the vehicle door handle at least once, and obtaining a corresponding first contact point position and a corresponding plurality of moving contact point positions; and generating a local point cloud of the vehicle door handle based on the first contact point position and the plurality of moving contact point positions. The method provided by the application can accurately model the vehicle door handle.

Description

Technical Field

[0001] This application relates to the field of robot perception and control, and in particular to a method for modeling vehicle door handles and a robot thereof. Background Technology

[0002] In vehicle cleaning robots and auxiliary operation scenarios, vehicle door handle operation is usually performed using a preset path execution method that combines visual recognition modeling or force sensing.

[0003] However, the above solutions are highly dependent on the complete visual information or preset path of the vehicle door handle. Under conditions of occlusion, reflection, pollution, and changes in the type of vehicle door handle, the geometric modeling accuracy of the vehicle door handle is insufficient.

[0004] Therefore, how to accurately model vehicle door handles has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a method and robot for modeling vehicle door handles, which can be used to accurately model vehicle door handles.

[0006] In a first aspect, this application provides a method for modeling a vehicle door handle. The method includes: controlling an actuator of a robot to move toward the vehicle door handle at a first pose relative to the vehicle door handle; when force data indicating that the actuator is in contact with the vehicle door handle indicates that the actuator is in contact with the vehicle door handle, determining the initial contact point position based on the force data and the pose data of the actuator, and controlling the actuator to move along the vehicle door handle to obtain multiple moving contact point positions; switching the first pose and / or the path of the actuator moving along the vehicle door handle at least once and obtaining the corresponding initial contact point position and the corresponding multiple moving contact point positions; and generating a local point cloud of the vehicle door handle based on the initial contact point position and the multiple moving contact point positions.

[0007] In one possible embodiment, after generating a local point cloud of a vehicle door handle, the method further includes: performing at least one of curvature calculation, normal vector analysis, and edge detection on the local point cloud to obtain features of the vehicle door handle, wherein the features of the vehicle door handle include at least one of a region with curvature greater than a threshold, a planar region, a curved surface region, and a door handle boundary; registering a local point cloud template of a preset type of vehicle door handle with the local point cloud based on the features of the vehicle door handle; and determining a first operational feature point of the local point cloud based on a first operational feature point in the successfully registered local point cloud template, wherein the first operational feature point of the local point cloud is used to distinguish between different types of vehicle door handles.

[0008] In one possible embodiment, the method further includes at least one of the following: If the first operational feature point of the local point cloud is a press center, verify the press center based on a first condition, the first condition including: the first region is a concave region or a planar region and the force data corresponding to the first region exhibits continuous normal resistance characteristics; the first region is the region where the press center is located in the local point cloud, and the press center is used to operate a press-type vehicle door handle; If the first operational feature point of the local point cloud is a rotation axis, verify the rotation axis based on a second condition, the second condition including: the second region is axial and the force data of the second region exhibits torque response characteristics based on a central axis, and the rotation axis is used to operate an outward-pulling type vehicle door handle; If, based on the second operational feature point in the successfully registered local point cloud template, a second operational feature point of the local point cloud is determined and the second operational feature point of the local point cloud is a gripping point for gripping the vehicle door handle, verify the gripping point based on a third condition, the third condition including: the third region is located in the middle or end of the local point cloud; the third region is a continuous curved surface region and the force data corresponding to the third region exhibits multi-directional mechanical stability characteristics; the third region is the region where the gripping point is located in the local point cloud.

[0009] In one possible embodiment, the method further includes: determining the type of vehicle door handle based on the successfully registered local point cloud template; if the vehicle door handle is a push-button type, generating a first operation command, the first operation command instructing an actuator to apply a preset constant force on the vehicle door handle corresponding to the first region, detecting position changes of the actuator until a first signal indicating completion of the first operation command is generated; if the vehicle door handle is a pull-out type, generating a second operation command, the second operation command instructing an actuator to apply a preset torque on the vehicle door handle corresponding to the second region, detecting angle and torque feedback of the actuator to determine whether the second operation command is completed.

[0010] In one possible embodiment, controlling the robot's actuator to move toward the vehicle door handle from a first pose relative to the vehicle door handle includes: controlling the robot's actuator to move toward the vehicle door handle from the first pose relative to the vehicle door handle using a target path corresponding to the target vehicle door handle type, based on a preset target vehicle door handle type; registering a local point cloud template of a vehicle door handle of a preset type with a local point cloud based on the characteristics of the vehicle door handle includes: registering a local point cloud template of the target vehicle door handle type with a local point cloud based on the characteristics of the vehicle door handle; the method further includes: if registration fails, adjusting the target vehicle door handle type and returning to the step of controlling the robot's actuator to move toward the vehicle door handle from the first pose relative to the vehicle door handle using a preset target vehicle door handle type, along a target path corresponding to the target vehicle door handle type.

[0011] In one possible embodiment, controlling the movement of the actuator along the vehicle door handle includes: adjusting at least one of the actuator's pose, impedance, movement speed, and movement direction, so that the value of the force data of the actuator contacting the vehicle door handle during the movement of the actuator along the vehicle door handle is within a preset value range.

[0012] In one possible embodiment, before controlling the robot's actuator to move toward the vehicle door handle at a first pose relative to the vehicle door handle, the method further includes: acquiring a global point cloud of the vehicle door region; determining a region of interest (ROI) where the vehicle door handle is located based on the global point cloud; controlling the robot's actuator to move toward the vehicle door handle at the first pose relative to the vehicle door handle includes: controlling the actuator to move toward the vehicle door handle at the first pose relative to the vehicle door handle based on the ROI where the vehicle door handle is located.

[0013] In one possible embodiment, determining the region of interest (ROI) where the vehicle door handle is located based on the global point cloud includes: performing instance segmentation on the global point cloud using an instance segmentation model to determine the ROI where the vehicle door handle is located; wherein the training samples of the instance segmentation model include global point cloud samples of the contaminated vehicle door handle.

[0014] Secondly, this application provides a vehicle door handle modeling device, comprising: a control module, configured to control an actuator of a robot to move toward the vehicle door handle at a first pose relative to the vehicle door handle; further configured to, when force data indicating that the actuator is in contact with the vehicle door handle indicates that the actuator is in contact with the vehicle door handle, determine the initial contact point position based on the force data and the pose data of the actuator and control the actuator to move along the vehicle door handle to obtain multiple moving contact point positions; further configured to switch the first pose and / or the path of the actuator moving along the vehicle door handle at least once and obtain the corresponding initial contact point position and the corresponding multiple moving contact point positions; and further configured to generate a local point cloud of the vehicle door handle based on the initial contact point position and the multiple moving contact point positions.

[0015] Thirdly, this application provides a robot comprising: a memory, a processor, and an execution component; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any of the first aspects.

[0016] In one possible embodiment, the actuation component is mounted on the robot's first robotic arm, and the robot also includes a second robotic arm for assisting in supporting the vehicle where the door handle is located.

[0017] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0018] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0019] The vehicle door handle modeling method of this application, through proximity control, contact detection, path switching, and point cloud generation, enables the robot to continuously obtain reliable and more complete contact point positions even when there is insufficient visual information of the vehicle door handle, complex external environment, and large variation in vehicle door handle type. Based on this, it constructs a stable and more comprehensive local point cloud of the vehicle door handle, thereby improving the vehicle door handle modeling distortion problem, improving the coverage, accuracy, and geometric consistency of the local point cloud of the vehicle door handle, and providing a reliable foundation for subsequent operation of the vehicle door handle. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of the execution component of the robot according to an embodiment of this application;

[0022] Figure 2 This is a flowchart of a vehicle door handle modeling method according to an embodiment of this application;

[0023] Figure 3 A flowchart illustrating a vehicle door handle modeling method according to another embodiment of this application;

[0024] Figure 4 A flowchart illustrating a vehicle door handle modeling method according to another embodiment of this application;

[0025] Figure 5 This is a schematic diagram of the vehicle door handle modeling device according to an embodiment of this application.

[0026] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] In related technology 1, the global point cloud of the vehicle door handle is acquired using a Red Green Blue-Depth (RGB-D) camera or a Three Dimensional (3D) LiDAR, and the target area where the vehicle door handle is located is segmented based on an instance segmentation model. However, such solutions are prone to positioning errors in scenes with insufficient lighting, severe occlusion, or surface reflection, resulting in low modeling accuracy of the vehicle door handle.

[0029] Related technology 2 uses force sensors to detect changes in the contact force between the robot and the vehicle door handle, and combines this with a preset path to complete the modeling. However, this method relies on prior knowledge of the vehicle door handle (such as structural knowledge of the vehicle door handle and an accurate preset path), and when faced with new or non-standard vehicle door handles, it is prone to failure to accurately model the vehicle door handle due to improper path planning.

[0030] In view of this, the vehicle door handle modeling method of this application embodiment controls the robot's execution component to move towards the vehicle door handle from a first pose relative to the vehicle door handle; when the force data indicating that the execution component is in contact with the vehicle door handle indicates that the execution component is in contact with the vehicle door handle, the method determines the initial contact point position based on the force data and the pose data of the execution component, and controls the execution component to move along the vehicle door handle to obtain multiple moving contact point positions; the method switches the first pose and / or the path of the execution component moving along the vehicle door handle at least once, and obtains the corresponding initial contact point position and the corresponding multiple moving contact point positions; based on the initial contact point position and the multiple moving contact point positions, a local point cloud of the vehicle door handle is generated. That is, by repeatedly obtaining the contact point positions under different poses and different paths, and converging the multiple contact point positions into a local point cloud, the modeling accuracy of the vehicle door handle can be improved when the visual information of the door handle is insufficient or the structure of the vehicle door handle is complex.

[0031] The vehicle door handle modeling method of this application can be applied to robots. The robot is used to drive the actuators to approach, contact, and move along the surface of the door area in three-dimensional space according to control commands. This application will use a robot equipped with a robotic arm as an example for illustration; however, the robot can be of other types, and no specific limitation is made here.

[0032] Figure 1 This is a schematic diagram of the execution component 10 of the robot according to an embodiment of this application.

[0033] The actuator 10 can be a ball-head probe, a tactile probe, a gripper, a contact head covered with flexible material, etc., and its function is to establish measurable physical contact with the vehicle door handle. The actuator 10 can be located at the end effector of the robot. Figure 1 In the example, the execution component 10 is specifically a gripper.

[0034] In addition to the actuator 10, the robot also includes a sensor module and a control module. The sensor module may include a vision sensor 11 and a force sensor 12. Figure 1 In the example, vision sensor 11 and force sensor 12 are disposed on actuation component 10. The control module may include a processor and a memory, wherein code is stored in the memory, and the processor executes the code stored in the memory to perform the method of any embodiment of this application.

[0035] In one possible embodiment, the robot may include multiple robotic arms, specifically at least a first robotic arm and a second robotic arm. An actuation component is mounted on the first robotic arm, while the second robotic arm, distinct from the first, assists in supporting the vehicle where the door handle is located. Thus, during the execution of the steps described below, the second robotic arm, by supporting the vehicle where the door handle is located, reduces vehicle sway and helps the actuation component accurately perform the steps described below.

[0036] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0037] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk drive. Figure 1 In the example, the execution component 10 also includes a connector 13 for connection to the robotic arm, which may be a flange.

[0038] Figure 2 This is a flowchart illustrating a vehicle door handle modeling method according to an embodiment of this application. Figure 2As shown, the vehicle door handle modeling method of this application embodiment includes steps S201 to S204.

[0039] S201, The robot's actuator moves toward the vehicle door handle from its first position relative to the door handle.

[0040] The first posture can be understood as the initial pre-contact space state of the actuator relative to the vehicle door handle. This space state includes at least two parts: position and attitude. The position is used to define the relative distance between the actuator and the vehicle door handle, and the attitude is used to define the approach direction, normal orientation, and accessibility of the actuator when moving along the vehicle door handle.

[0041] The vehicle door handle can be a push-button type or a pull-out type, etc. The first pose in this application embodiment is not fixed, but can be determined according to the approximate spatial area of ​​the vehicle door handle, the normal direction of the door panel, and the distribution of surrounding obstacles.

[0042] In one possible embodiment, the robot may first receive coarse positioning information of the vehicle door handle from a sensor module or a priori database. The coarse positioning information includes the approximate three-dimensional coordinates of the center of the vehicle door handle, the length direction of the vehicle door handle, the installation height, or the boundary range of the door area where the vehicle door handle is located.

[0043] It should be noted that even when the visual information of the vehicle door handle is incomplete due to reflection, obstruction, or contamination, the robot can still estimate the area where the vehicle door handle is located based on preset vehicle templates, model templates, prior knowledge of the door structure, and historical parking postures.

[0044] For example, the robot can solve for a set of pre-contact trajectories that satisfy collision constraints and robot motion constraints based on the area where the vehicle door handle is located or coarse localization information, the robot's current pose, and obstacles within a preset range of the door. The endpoint of this pre-contact trajectory is the first pose, corresponding to a hovering state where the actuator maintains a predetermined safe distance before contact. This safe distance can be set in the range of millimeters to centimeters. By using this safe distance, collisions between the robot's actuator and the vehicle door handle can be avoided when there are deviations in the visual information of the vehicle door handle, while maintaining a sufficiently short approach distance to reduce the solution error of the subsequent first contact point.

[0045] For example, the robot can control the actuator to move towards the vehicle door handle at a low speed and with low impact based on the first pose. Low speed means the actuator's movement speed is below a preset threshold, so that at the moment of contact, the robot can promptly control the actuator to stop based on the force feedback from the vehicle door handle. Low impact means the force exerted by the actuator on the vehicle door handle is below a preset threshold, thereby reducing the risk of the actuator scratching or colliding with the surface of the vehicle door handle.

[0046] In one possible embodiment, the robot may employ a segmented approach strategy to move the actuator toward the vehicle door handle from a first pose. Specifically, the robot controls the actuator to first reach the vicinity of the first pose at a relatively fast speed, and then switch to reaching the first pose and moving toward the vehicle door handle at a lower speed.

[0047] In this embodiment of the world coordinate system, by first constructing the first pose and then controlling the movement of the actuator toward the vehicle door handle in the manner described above, the traditional method of relying on a single visual positioning to directly contact the vehicle door handle can be transformed into a gradual approach process with safety buffer and attitude planning, thereby providing a stable initial state for accurate modeling of the vehicle door handle even when visual information is unstable.

[0048] S202. When the force data indicating that the actuator is in contact with the vehicle door handle indicates that the actuator is in contact with the vehicle door handle, the position of the first contact point is determined based on the force data and the position data of the actuator, and the actuator is controlled to move along the vehicle door handle to obtain multiple moving contact point positions.

[0049] Combination Figure 1 For example, force data refers to data characterizing the contact state acquired by force sensors installed in the actuator. Force data can include torque, normal force, tangential force, resultant force, and other indicators calculated from these raw quantities. Pose data refers to the spatial position and orientation information of the actuator at the moment of contact and during its movement along the vehicle door handle. In the case of a robot including a robotic arm, the pose data of the actuator can be determined jointly by the joint encoder of the robotic arm and the coordinate system calibration relationship of the actuator. The initial contact point position refers to the three-dimensional coordinates of the first verifiable contact point formed on the surface of the vehicle door handle when the actuator transitions from a state of not contacting the vehicle door handle to a state of contacting the vehicle door handle. The moving contact point position refers to the three-dimensional coordinates of multiple contact points sequentially acquired during the movement of the actuator along the surface or boundary of the vehicle door handle while maintaining contact with the door handle.

[0050] In one possible embodiment, the robot continuously samples the output of the force sensor as it controls the actuator to approach the vehicle door handle to obtain force data. This force data is then filtered, zero-drift compensated, and contact event detected. Filtering can employ moving average filtering, median filtering, low-pass filtering, or Kalman filtering to reduce high-frequency fluctuations caused by mechanical vibration and electrical noise. Zero-drift compensation is used to deduct the effects of the force sensor's own weight, the actuator's gravity, and temperature drift on the force data. Contact event detection determines whether the actuator has contacted the vehicle door handle by comparing the current normal force, tangential force, torque, and their rate of change with preset thresholds. Specifically, when the force data indicates that the normal force jumps from near zero to exceeding the preset threshold and continues for several sampling cycles, the robot can determine that the actuator is in contact with the vehicle door handle.

[0051] In another possible embodiment, the robot can combine the torque abrupt change indicated by force data with the magnitude of the speed decrease of the actuator to determine whether the actuator is in contact with the vehicle door handle, thereby improving the robustness of identifying contact between the actuator and the vehicle door handle.

[0052] After confirming contact between the actuator and the vehicle door handle through the above example, the robot can register the force data at the moment of contact with the actuator pose data corresponding to the synchronization timestamp and solve for the initial contact point position. For example, if the actuator is a ball-head probe, the initial contact point position can be calculated based on the ball head center position, ball head radius, and normal contact direction. If the actuator is a planar contact or gripper, the initial contact point position of the planar contact or gripper contacting the vehicle door handle can be deduced from the geometric parameters of the planar contact or gripper and the current posture. The solution method for multiple moving contact point positions is similar and will not be elaborated here. It should be noted that the calculation of contact point positions, including the initial contact point position and moving contact point positions, usually relies on coordinate transformation relationships. That is, the contact point is first represented in the coordinate system corresponding to the actuator, and then the coordinate system where the contact point is located can be transformed to the world coordinate system through transformations from the coordinate system of the actuator to the coordinate system of the actuator and the robot's connecting parts, from the connecting parts to the robot's coordinate system, and from the robot's coordinate system to the world coordinate system.

[0053] In one possible embodiment, to suppress the instantaneous error of the actuator's single contact with the vehicle door handle, the robot can compare at least one of the following—pose increments, force data trends, and contact stiffness models—from multiple sampling moments before and after the actuator's contact with the vehicle door handle with a corresponding standard template to verify or adjust the initial contact point position. Here, the contact stiffness model characterizes the displacement and force relationship between the vehicle door handle surface and the actuator during the contact establishment phase. The standard template represents the pose increments, force data trends, and contact stiffness model corresponding to the actual contact of the actuator with the vehicle door handle, collected beforehand. By introducing the contact stiffness model, the deviation in the initial contact point position caused by the elastic deformation of the vehicle door handle surface and / or the actuator material can be corrected.

[0054] After the initial contact point is determined, the robot can control the actuators to move along a preset path. This preset path can be a path that matches the type of a preset vehicle door handle.

[0055] For example, the process of the actuator moving along a preset path can be achieved using a constant-speed movement method, a segmented movement method, or an impedance movement method. In the constant-speed movement method, the robot controls the actuator to continuously displace tangentially, and ensures that the normal force is maintained within a preset threshold range through a force closed loop; in the segmented movement method, the actuator pauses, samples, and confirms effective contact after moving one small step, and then continues to move the next small step to adapt to high-precision point cloud acquisition; in the impedance movement method, the robot controls the actuator to automatically adjust its posture to conform to the curvature change of the vehicle door handle.

[0056] For example, the positions of multiple moving contact points can be obtained by sampling at fixed time intervals, sampling at fixed displacement intervals, or adaptively increasing the sampling density of key areas based on curvature changes. For instance, when the robot detects a rapid change in the direction of the contact force, it indicates that the actuator may be passing through rounded corners, grooves, or boundary transition areas. In this case, the robot can control the actuator to reduce the movement step size and increase the sampling frequency to more accurately recover the geometry of the vehicle door handle.

[0057] As the actuator moves continuously along the vehicle door handle, the robot can repeatedly perform contact confirmation, pose reading, coordinate transformation, and contact point position calculation operations at each valid sampling moment, thereby obtaining multiple moving contact point positions arranged in a time sequence. If the force data at a certain moment indicates that the normal force is lower than a preset threshold, or the tangential force or torque exceeds an abnormal preset threshold, the robot can determine that the actuator has lost contact or become stuck with the surface of the vehicle door handle. At this time, the robot can trigger deceleration, retraction, or re-contact with the vehicle door handle.

[0058] In this embodiment, by utilizing force data to determine the actual moment of contact between the actuator and the vehicle door handle, the position of the first contact point and the positions of multiple subsequent moving contact points are deduced. This allows for the direct provision of the contact point positions of the vehicle door handle based on physical contact in reflective, dirty, and obstructed environments. These contact point positions serve as the basis for accurate subsequent modeling of the vehicle door handle.

[0059] S203, switch the first position at least once and / or the path along which the actuator moves along the vehicle door handle and obtain the corresponding first contact point position and the corresponding multiple moving contact point positions.

[0060] Switching the initial pose refers to changing the pre-contact space state when the actuator approaches the vehicle door handle again, such as changing the height or attitude angle of the actuator, so that the actuator contacts the vehicle door handle from different directions. Switching the path of the actuator along the vehicle door handle refers to changing the direction and / or path of the actuator's movement after contact is established, so that subsequent sampling points cover areas that were not previously fully sampled. The corresponding initial contact point position and multiple movement contact point positions refer to the sampling results of a set of contact points re-acquired under each new initial pose or new movement path. At least one switch means that after the robot has completed its initial exploration of the vehicle door handle, it is not limited to a single, fixed movement path and / or a single movement starting point, but actively performs one or more repeated explorations.

[0061] In one possible embodiment, after the robot's initial exploration of the vehicle door handle, it can first assess the initial contact point location and multiple moving contact point locations (hereinafter referred to as the initial exploration contact point set) to determine whether the distribution of each contact point location in the initial exploration contact point set is uniform, and whether there are empty areas, missing edge areas, or abnormally clustered areas. Specifically, this can be achieved by calculating the point cloud bounding box, local neighborhood density, curvature estimation stability, and spacing statistics between contact point locations using the initial exploration contact point set. For example, if the spacing between contact point locations in the neighborhood of a certain area of ​​the vehicle door handle is greater than the overall average, it can be determined that the area is undersampled. When multiple contact point locations are concentrated on the same local plane while contact point locations on the opposite curved surface are missing, it can be determined that the current movement path of the actuator does not cover the complete outline of the vehicle door handle. Through the analysis of the above results, if the direction in which the actuator approaches the vehicle door handle is unreasonable, the first pose can be changed so that the actuator re-contacts the vehicle door handle from another direction. If the direction of movement of the actuator along the vehicle door handle is singular, the robot can switch the path of movement of the actuator along the vehicle door handle while maintaining the same first pose. For example, the movement path can be changed from along the length direction to along the boundary contour direction, from left to right to right to left, or from single-layer scanning to multi-layer scanning at different heights.

[0062] In another possible embodiment, the path along which the switching execution component moves along the vehicle door handle can be: the robot's memory can pre-store several movement paths for different vehicle door handle types, and if the current vehicle door handle matches a certain vehicle door handle type, multiple switching operations can be performed in sequence according to the corresponding movement paths.

[0063] In one possible embodiment, after completing the initial exploration, the robot can control the actuator to retract to a preset safe distance position, and then plan the next path to contact the vehicle door handle based on the new first pose. This retraction mechanism can prevent the actuator from rubbing against the protruding area of ​​the vehicle door handle or the edge of the door panel, while reserving space for switching the first pose. Subsequently, the robot can control the actuator to move towards the vehicle door handle from the new first pose, and re-detect the contact between the actuator and the vehicle door handle, as well as solve the initial contact point position, the moving contact point position, etc., which will not be described in detail here.

[0064] In this embodiment, by introducing repeated tactile exploration with multiple first poses and multiple movement paths, the coverage blind spots and error accumulation problems caused by single movement paths or sampling based on a single pose are overcome. Since the surface of a vehicle door handle may contain curved surfaces, recesses, boundary transitions, and orientation differences due to different installation postures, using only a single pose and / or a single movement path can easily lead to problems such as unstable contact or unreachability in certain areas, ultimately resulting in sparse point clouds or distorted shapes of the vehicle door handle. This embodiment, by switching the first pose and / or the path along which the actuator moves along the vehicle door handle at least once, enables the robot to obtain diverse contact point positions from different directions and different movement paths, thereby expanding the coverage of contact points and improving the integrity and consistency of the geometric information of the vehicle door handle.

[0065] S204. Based on the initial contact point location and multiple moving contact point locations, generate a local point cloud of the vehicle door handle.

[0066] Local point cloud refers to a data set that represents the local geometric contour and surface morphology of a vehicle door handle in the form of a discrete three-dimensional coordinate point set.

[0067] Based on this, each contact point location in the local point cloud can be associated with corresponding force data. The force data indicates the normal force, tangential force, torque, etc., of the actuator contacting the vehicle door handle at that contact point location. In addition, each contact point location in the local point cloud can also be associated with the timestamp of the actuator contacting the vehicle door handle at that contact point location, the first pose, and / or the number of times the actuator switches along the path of movement of the vehicle door handle and its identifier.

[0068] In one possible embodiment, the robot can uniformly transform all initial contact point positions and all multiple moving contact point positions obtained in steps S102 and S103 into the same coordinate system. This coordinate system can be the robot coordinate system, the world coordinate system, etc. After the coordinate system is unified, the robot can preprocess the set of contact point positions composed of all initial contact point positions and all multiple moving contact point positions. Preprocessing may include using statistical outlier detection to remove abnormal contact point positions caused by contact jitter, misjudged collision, or instantaneous slippage; using radius neighborhood filtering to remove isolated contact point positions; and using temporal continuity constraints to delete contact point positions that jump. For multiple contact point positions that overlap or are close to each other in multiple rounds of exploration, they can be merged into a single contact point position according to a preset spatial distance threshold, or multiple overlapping or close contact point positions can be fused by weighted averaging after multiple switching of the first pose and / or the movement of the execution component along the vehicle door handle.

[0069] After obtaining the preprocessed set of contact point locations, the robot can generate key geometric parameters of the vehicle door handle, such as length, thickness, protrusion height, edge orientation, and operable area location, through curvature estimation, boundary extraction, and principal axis analysis. Next, the robot can perform interpolation, curve fitting, or local surface reconstruction on the preprocessed set of contact point locations to obtain a local point cloud of the vehicle door handle. For example, for multiple moving contact point locations obtained along the edge of the vehicle door handle, spline curve fitting of the boundary contour can be used; for moving contact point locations covering the surface of the vehicle door handle, local quadratic surface fitting, moving least squares methods, or Poisson reconstruction can be used to form a smoother local surface. If the preprocessed set of contact point locations contains sparse voids in certain areas, the robot can interpolate and fill in the voids by combining multiple switching of the first pose and / or neighborhood trends along the path of the actuator moving along the vehicle door handle. Alternatively, it can call upon a template library of vehicle door handles for geometric constraint completion. This template library includes 3D information of various types of vehicle door handles. The template library here is not intended to replace real sampling, but rather to provide structural continuity constraints for small missing areas, given that the main outline of the vehicle door handle has been determined based on physical contact information such as the preprocessed set of contact point locations, in order to improve the accuracy and completeness of local point cloud modeling.

[0070] In summary, the vehicle door handle modeling method of this application, through proximity control, contact detection, path switching, and point cloud generation, enables the robot to continuously obtain reliable and more complete contact point positions even when there is insufficient visual information of the vehicle door handle, a complex external environment, and significant variations in the type of vehicle door handle. Based on this, it constructs a stable and more comprehensive local point cloud of the vehicle door handle, thereby improving the vehicle door handle modeling distortion problem, increasing the coverage, accuracy, and geometric consistency of the local point cloud of the vehicle door handle, and providing a reliable foundation for subsequent operation of the vehicle door handle.

[0071] like Figure 2 As shown, in one possible embodiment, after generating the local point cloud of the vehicle door handle, the vehicle door handle modeling method further includes steps S205 to S207.

[0072] S205. Perform at least one of curvature calculation, normal vector analysis, and edge detection on the local point cloud to obtain the features of the vehicle door handle.

[0073] The characteristics of a vehicle door handle include at least one of the following: a region with curvature greater than a threshold, a planar region, a curved surface region, and a door handle boundary.

[0074] Understandably, local point clouds can characterize the geometric contours of local surfaces of vehicle door handles. In this embodiment, curvature calculation is used to characterize the degree of curvature within the neighborhood of each 3D point in the local point cloud, typically based on the neighborhood covariance matrix, surface fitting results, or principal curvature estimation, thereby identifying areas with significant local convexities, depressions, and bends, i.e., areas with curvature greater than a threshold. Normal vector analysis is used to extract the surface orientation of the neighborhood of each 3D point in the local point cloud. By comparing the consistency of the normals of the current 3D point with those of its neighbors, planar regions and curved surface regions can be distinguished. Edge detection is used to identify significant changes in density, abrupt changes in normals, or large changes in geometric gradients of 3D points in the local point cloud to determine the door handle boundary.

[0075] S206. Based on the characteristics of the vehicle door handle, register the local point cloud template of the vehicle door handle of the preset type with the local point cloud.

[0076] The local point cloud template of the preset type of vehicle door handle can be a standard geometric template for various types of vehicle door handles such as pull-out type and push-button type. This standard geometric template can be pre-stored in the memory and aligned with the acquired local point cloud in the same coordinate system.

[0077] In one possible embodiment, at least one corresponding set of information from regions with curvature greater than a threshold, planar regions, curved surface regions, and door handle boundaries can be converted into fixed-dimensional digital vectors, i.e., converted into feature descriptors. Based on the features of the vehicle door handle, the local point cloud template of the vehicle door handle of a preset type is registered with the local point cloud, i.e., registration based on feature descriptors.

[0078] In one possible embodiment, in addition to using feature descriptor-based registration, an iterative nearest-point registration method can also be used to align the boundary, planar, and surface features in the local point cloud template with the corresponding features in the actual local point cloud.

[0079] S207. Determine the first operational feature point of the local point cloud based on the first operational feature point in the successfully registered local point cloud template.

[0080] The first operational feature point of the local point cloud is used to distinguish the operation of various types of vehicle door handles.

[0081] After successful registration using the above method, the robot transforms the first operational feature point in the local point cloud template to the coordinate system of the local point cloud based on the transformation relationship between the local point cloud template and the actual local point cloud, thereby determining the first operational feature point of the local point cloud. The first operational feature point of the local point cloud is used to subsequently identify the operable positions of different types of vehicle door handles and to provide accurate action targets for the actuators.

[0082] For example, the first operational feature point in the local point cloud template is used as a key point for operating the vehicle door handle, such as a point for pressing or a reference point for rotation. The coordinates of the first operational feature point in the local point cloud template can be associated with the local point cloud template and mapped to the actual local point cloud after registration.

[0083] In this embodiment, the robot enhances the recognizability of the vehicle door handle's geometric structure by performing at least one of curvature calculation, normal vector analysis, and edge detection on the local point cloud. By registering a local point cloud template of a preset type of vehicle door handle with the local point cloud based on the characteristics of the vehicle door handle, the prior structure of the preset type of vehicle door handle can be associated with the actual local point cloud. By determining the first operational feature point of the local point cloud based on the first operational feature point in the successfully registered local point cloud template, the operation position can still be accurately identified even when the vehicle door handle type changes or is partially occluded, thereby improving the success rate of subsequent operations on the vehicle door handle.

[0084] like Figure 2 As shown, in one possible embodiment, the vehicle door handle modeling method further includes at least one of steps S208 to S210. Steps S208 and S209 may be performed after step S207.

[0085] S208. When the first operational feature point of the local point cloud is the pressing center, the pressing center is verified based on the first condition.

[0086] The first condition includes: the first region is a concave region or a planar region, and the force data corresponding to the first region exhibits continuous normal resistance characteristics. The first region is the area in the local point cloud where the pressing center is located, and the pressing center is used to operate a push-button type vehicle door handle.

[0087] The pressing center refers to the local location where normal pressing force is applied to a push-button type vehicle door handle. The area where the pressing center is located (i.e., the first area) is usually a concave area or a flat area and exhibits resistance characteristics after being pressed.

[0088] In one possible embodiment, the pressing center can be obtained by fitting a low-lying surface in a local point cloud. Specifically, the robot determines whether the first region satisfies the geometric characteristics of a concave or planar region based on data such as the curvature value of the local point cloud, the normal angle between adjacent 3D points, and the elevation difference. Then, it combines the normal force curve at the contact point position of the first region to determine whether there are characteristics of continuously rising, continuously maintained, or stably rebounding normal resistance, thereby verifying whether the first region is indeed the region where the pressing center is located and verifying whether the pressing center is correct.

[0089] S209. When the first operational feature point of the local point cloud is the rotation axis, verify the rotation axis based on the second condition.

[0090] The second condition includes: the second region is axial and the force data of the second region exhibits torque response characteristics based on the central axis. The rotating axis is used to operate the pull-out type vehicle door handle.

[0091] The rotation axis refers to the center position of the rotation reference formed by the pull-out type vehicle door handle during the opening process, which usually corresponds to the axis direction of the door handle.

[0092] In one possible embodiment, the rotation axis can be determined by axis fitting, principal direction analysis and axis regression. When the actuator rotates around the second region, the robot can simultaneously collect torque and angular displacement. If the force data corresponding to the contact point position of the second region shows the torque change law around the central axis, it can be determined that the second region is the actual location of the rotation axis and that the rotation axis is correct.

[0093] S210. Based on the second operational feature point in the successfully registered local point cloud template, determine the second operational feature point of the local point cloud. If the second operational feature point of the local point cloud is a gripping point for gripping the vehicle door handle, verify the gripping point based on the third condition.

[0094] The third condition includes: the third region is located in the middle or end of the local point cloud; the third region is a continuous curved surface region; and the force data corresponding to the third region exhibits multi-directional mechanical stability characteristics. The third region is the area in the local point cloud where the gripping point is located.

[0095] The gripping point is the area where the actuating component is clamped or covered. Its geometry usually has a continuous curved surface and can maintain stable contact in multiple force directions.

[0096] In one possible embodiment, the gripping points can be filtered based on the middle or end positions of the local point cloud. The robot can also confirm whether the third region is a continuous curved surface region by the surface continuity, whether the normal transition is smooth, and the envelope range of the third region. Subsequently, it collects the force value fluctuations indicated by the force data when force is applied in different directions to determine whether the force data corresponding to the third region exhibits multi-directional mechanical stability characteristics.

[0097] In this embodiment, based on the structure of different types of vehicle door handles, three types of operational feature points are respectively set: pressing center, rotation axis, and gripping point. Furthermore, at least one of the shape features, position features, and force features of the area where the operational feature point is located is used as a specific verification condition for each of the three types of operational feature points, thereby achieving accurate identification and verification of the above three types of operational feature points and improving the accuracy and stability of subsequent operations on the vehicle door handle.

[0098] In one possible embodiment, the vehicle door handle modeling method further includes: determining the type of vehicle door handle based on a successfully registered local point cloud template. Based on this, the vehicle door handle modeling method further includes: generating a first operation command if the vehicle door handle is a push-button type, and / or generating a second operation command if the vehicle door handle is a pull-out type.

[0099] For example, a local point cloud template can be associated with a vehicle door handle type label. After the local point cloud and the local point cloud template are successfully registered, it can be determined that the type of the vehicle door handle is consistent with the type of the vehicle door handle corresponding to the local point cloud template.

[0100] The first operation command instructs the actuator to apply a preset constant force to the vehicle door handle corresponding to the first area, detect changes in the position of the actuator, and generate a first signal indicating completion of the first operation command.

[0101] The second operation command instructs the actuator to apply a preset torque to the vehicle door handle corresponding to the second area, and detects the angle and torque feedback of the actuator to determine whether the second operation command has been completed.

[0102] It is understandable that the first area and the second area are the area where the pressing center is located and the area where the rotation axis is located, respectively, corresponding to the pressing contact surface and the pulling force application area, which can provide a clear range of action for the actuator to operate the vehicle door handle.

[0103] For example, for a push-button type vehicle door handle, the robot can control the actuator to continuously output constant pressure along the normal direction of the vehicle door handle after determining a first area, and detect changes in the position of the actuator by means of position sensors or the like. Taking operating the vehicle door handle to open or close the vehicle door as an example, the first signal may include the sound emitted by the door lock opening or closing.

[0104] For example, for an outward-pull type vehicle door handle, after determining the second area, the robot controls the actuator to apply torque around a predetermined axis. The robot can determine whether the rotation angle of the actuator operating the vehicle door handle has reached the set range and whether the load has changed by detecting the angle and torque feedback of the actuator, thereby determining the completion status of the second operation command. The second operation command may be operating the vehicle door handle to open or close the vehicle door.

[0105] In this embodiment, the vehicle door handle type is accurately determined based on the successfully registered local point cloud and local point cloud template. Based on this, and considering the type of vehicle door handle, the actuator can generate appropriate operating commands according to the structure of different types of vehicle door handles, and accurately determine the operating state of the vehicle door handle based on information such as changes in the position, angle, and force feedback of the actuator. This improves the adaptability and reliability of vehicle door handle operation.

[0106] Figure 3 This is a flowchart of a vehicle door handle modeling method according to another embodiment of this application.

[0107] like Figure 3 As shown, in one possible embodiment, step S201 includes: step S2011.

[0108] S2011. Based on the preset target vehicle door handle type, control the robot's actuator to move towards the vehicle door handle from its first position relative to the vehicle door handle along the target path corresponding to the target vehicle door handle type.

[0109] The preset target vehicle door handle type can be determined randomly or sequentially from a plurality of preset vehicle door handle types, which is equivalent to assuming that the type of vehicle door handle is the target vehicle door handle type. Alternatively, when the robot performs global point cloud modeling of the vehicle door area, the preset vehicle door handle type can be identified based on the global point cloud.

[0110] like Figure 3As shown, in one possible embodiment, step S206 includes: step S2061.

[0111] S2061. Based on the characteristics of the vehicle door handle, register the local point cloud template of the target vehicle door handle type with the local point cloud. The registration process has been explained above and will not be repeated here.

[0112] In one possible embodiment, the vehicle door handle modeling method further includes: in the event of registration failure, adjusting the target vehicle door handle type and returning to the preset target vehicle door handle type, and controlling the robot's execution component to move towards the vehicle door handle at the first pose relative to the vehicle door handle along the target path corresponding to the target vehicle door handle type.

[0113] The target vehicle door handle type to be adjusted can be another vehicle door handle type that is different from the previously tried vehicle door handle types selected from a set of preset vehicle door handle types. It should be noted that when adjusting the vehicle door handle types sequentially from multiple vehicle door handle types, the order of the vehicle door handle types can be matched with the frequency of historically successfully registered vehicle door handle types to improve the success rate of vehicle door handle type matching.

[0114] In this embodiment, the robot first drives the actuator to move towards the vehicle door handle along the target path corresponding to the target vehicle door handle type, and then registers the local point cloud template corresponding to the target vehicle door handle type with the local point cloud. This achieves the linkage between motion control of the actuator and recognition of the vehicle door handle type. If the initial target vehicle door handle type is inaccurate, the robot can automatically adjust the target vehicle door handle type according to the registration result and retry. This makes the process of the actuator moving towards the vehicle door handle and the local point cloud registration process form a closed loop, improving the robot's adaptability in complex scenarios and the accuracy of vehicle door handle operation in scenarios with insufficient or occluded visual information of the vehicle door handle.

[0115] In one possible embodiment, step S201 includes: adjusting at least one of the pose, impedance, moving speed, and moving direction of the actuator, so that the value of the force data of the actuator contacting the vehicle door handle during the actuator's movement along the vehicle door handle is within a preset value range.

[0116] Impedance can be used to characterize the compliance of the actuator during contact with the vehicle door handle, movement speed can be used to characterize the relative motion rate of the actuator along the surface of the vehicle door handle, and movement direction can be used to characterize the motion tendency of the actuator relative to the door handle contour. The preset numerical range can be predetermined by the robot based on the vehicle door handle structure, the safety threshold of the actuator contacting the vehicle door handle, etc.

[0117] In one possible embodiment, the robot can collect force data on the contact between the actuator and the vehicle door handle in real time using force sensors and compare this force data with a preset numerical range. When the force data deviates from the preset range, the robot fine-tunes the pose of the actuator to change the angle and area of ​​contact with the vehicle door handle. Alternatively, the robot can adjust the equivalent stiffness and damping of the actuator to improve compliance with the vehicle door. Specifically, the stiffness and damping matrices of the actuator can be dynamically adjusted using an impedance control algorithm, allowing the actuator to conform to the surface changes of the vehicle door handle during contact, reducing the risk of surface damage caused by rigid collisions. Alternatively, the robot can reduce its movement speed to minimize the instantaneous impact on the vehicle door handle. Alternatively, the robot can adjust its movement direction to allow the actuator to move smoothly along the edge, surface tangent, or local contour direction of the vehicle door handle. Furthermore, the robot can generate compensation control quantities based on the force data fed back from the force sensors, thereby achieving closed-loop control of the force exerted by the actuator on the vehicle door handle.

[0118] In this embodiment of the application, the above operation can ensure that the actuator maintains stable contact when moving along the vehicle door handle, avoiding damage to the surface of the vehicle door handle due to excessive force of the actuator contacting the vehicle door handle, and also avoiding interruption of contact point position acquisition and local point cloud distortion in subsequent modeling due to insufficient force of the actuator contacting the vehicle door handle.

[0119] When the robot's actuator is controlled to move towards the vehicle door handle from its first position relative to the vehicle door handle using a target path corresponding to the target vehicle door handle type, the robot in this embodiment can also adjust at least one of the actuator's pose, impedance, moving speed, and moving direction so that the force data of the actuator contacting the vehicle door handle during the actuator's movement along the target path is within a preset value range.

[0120] like Figure 3 As shown, in one possible embodiment, before step S201, the vehicle door handle modeling method further includes steps S211 to S212.

[0121] S211. Obtain the global point cloud of the vehicle door region.

[0122] For example, the global point cloud of the vehicle door region can be a three-dimensional point set covering the outer panel of the vehicle door, the edge of the vehicle door, the mounting area of ​​the vehicle door handle, and its surrounding space. The global point cloud of the vehicle door region can reflect the overall spatial relationship between the vehicle door handle and the vehicle door.

[0123] In one possible embodiment, the global point cloud can be acquired by a vision sensor. Alternatively, the robot can perform filtering and noise reduction, ground and background removal, and vehicle door plane extraction on the initial global point cloud acquired by the vision sensor to preserve the three-dimensional structural information of the vehicle door region.

[0124] S212. Based on the global point cloud, determine the region of interest where the vehicle door handle is located.

[0125] The region of interest (ROI) for the vehicle door handle is a local region related to the vehicle door handle selected from the global point cloud. This ROI may include the door handle body, its outer contour, and easily accessible adjacent surfaces, providing a coarse localization basis for the robot's actuators.

[0126] In one possible embodiment, the robot can determine the region of interest (ROI) of the vehicle door handle based on geometric features such as convexities, depressions, and abrupt edge changes in the global point cloud. This ROI can be output as a local point cloud block, a bounding box, or a spatial mask.

[0127] Step S201 includes controlling the actuator to move toward the vehicle door handle at a first pose relative to the vehicle door handle, based on the region of interest where the vehicle door handle is located.

[0128] For example, the region of interest can be converted into pose and motion constraints for the movement of the actuator toward the vehicle door handle.

[0129] For example, the robot can generate the first pose of the actuator relative to the vehicle door handle and / or the path of movement along the vehicle door handle based on the spatial center of the region of interest, the normal direction, and the relative distance to the surrounding vehicle body surface.

[0130] In this embodiment, before the actuator moves towards the vehicle door handle, a global point cloud of the vehicle door area is acquired and the region of interest where the door handle is located is determined. This allows the robot to contact or move along the door handle with a more accurate initial state, thereby reducing the error in locating the door handle and the risk of the actuator colliding with it. Simultaneously, this approach improves adaptability to reflections, occlusions, or changes in the type of door handle, making the movement of the actuator towards the door handle more stable.

[0131] In one possible embodiment, determining the region of interest (ROI) where the vehicle door handle is located based on the global point cloud includes: performing instance segmentation on the global point cloud using an instance segmentation model to determine the ROI where the vehicle door handle is located.

[0132] The training samples for the instance segmentation model include global point cloud samples of contaminated vehicle door handles.

[0133] For example, the training samples of the instance segmentation model include global point cloud samples of contaminated vehicle door handles, wherein the global point cloud samples of contaminated vehicle door handles may include global point cloud samples collected and labeled under conditions of dust, mud, water stains, snow particles, oil stains, reflective interference, or partial occlusion on the surface of the vehicle door handle.

[0134] For example, instance segmentation models can employ models such as PointNet++ and Mask R-CNN. Instance segmentation models can extract features from the input global point cloud, such as spatial coordinates, normal vectors, intensity values, or neighborhood geometric relationships, and output the instance category label and corresponding confidence score of each point, thereby distinguishing vehicle door handle instances from door body instances, window frame instances, and vehicle handle surrounding decorative parts instances, etc.

[0135] In one possible embodiment, in order to improve the robustness of the instance segmentation model to contaminated scenarios, noise injection, local occlusion simulation, point density perturbation, or reflection anomaly simulation can be applied to the global point cloud samples of the contaminated vehicle door handle during the training process, so that the trained instance segmentation model can accurately learn the features for accurately segmenting instances.

[0136] In one possible embodiment, the robot can first acquire a global point cloud of the vehicle door region and preprocess it. This preprocessing includes denoising, downsampling, coordinate normalization, and ground or background removal to reduce the interference of irrelevant points on the instance segmentation results. The robot can then input the preprocessed global point cloud into an instance segmentation model, which performs instance-level classification on each point in the preprocessed global point cloud, outputting the boundary of the point set corresponding to the vehicle door handle, thereby determining the region of interest where the vehicle door handle is located.

[0137] In this embodiment, by using global point cloud samples of contaminated vehicle door handles to train the instance segmentation model, the ability of the instance segmentation model to recognize vehicle door handles can be improved. This enables the stable recognition of vehicle door handle areas under conditions of reflection, dirt, and occlusion, thereby improving the positioning accuracy of vehicle door handles and the reliability of subsequent operations on vehicle door handles.

[0138] Figure 4 This is a flowchart of a vehicle door handle modeling method according to another embodiment of this application.

[0139] like Figure 4 As shown, the vehicle door handle modeling method of this application embodiment includes steps S301 to S318.

[0140] S301, The robot moves to the side of the vehicle.

[0141] S302. The vision sensor scans the vehicle door area and generates a global point cloud.

[0142] S303. Perform instance segmentation on the global point cloud to determine the region of interest where the vehicle door handle is located.

[0143] S304, The preset vehicle door handle type is the target vehicle door handle type.

[0144] S305, control the actuator to move to the first position.

[0145] S306. Load the target path corresponding to the target vehicle door handle type.

[0146] S307, The control actuator moves towards the vehicle door handle at a low speed.

[0147] S308. Check whether the actuator is in contact with the vehicle door handle.

[0148] S309. Control the execution unit to move along the target path.

[0149] In step S309, the robot can also adjust at least one of the pose, impedance, moving speed, and moving direction of the actuator so that the force data of the actuator contacting the vehicle door handle during the actuator's movement along the vehicle door handle is within a preset range.

[0150] S310. Record the initial contact point position and the positions of multiple moving contact points and their force data.

[0151] S311, switch the first pose and / or target path at least once.

[0152] It should be noted that the target path matches the target vehicle door handle type. Switching the target path here means switching to another movement path that is different from the target path. This other movement path is not necessarily a movement path that matches a certain vehicle door handle type.

[0153] After executing step S311, the robot can return to step S310. Without changing the first pose and the target path, the robot executes step S312.

[0154] S312. Generate a local point cloud of the vehicle door handle.

[0155] S313. Determine the features of the vehicle door handle. Specifically, the robot can perform at least one of curvature calculation, normal vector analysis, and edge detection on the local point cloud to obtain the features of the vehicle door handle.

[0156] S314. Based on the characteristics of the vehicle door handle, register the local point cloud template of the target vehicle door handle type with the local point cloud.

[0157] If registration fails, proceed with step S315 and then return to step S309. In this case, the target path in step S309 is the target path corresponding to the adjusted vehicle door handle type. If registration succeeds, proceed with step S316.

[0158] S315. Adjust the door handle type of the target vehicle.

[0159] S316. Determine the operational feature points of the local point cloud. These operational feature points include the first operational feature point and the second operational feature point of the local point cloud.

[0160] S317. Verify the operational feature points of the local point cloud based on the verification conditions. For example, if the first operational feature point of the local point cloud is the pressing center, the verification condition is the first condition.

[0161] If verification is successful, proceed to step S318. If verification fails, return to step S311. It should be noted that if the number of times step S311 is returned reaches a preset threshold, the robot may send a prompt message to the terminal device, indicating that manual intervention is required until the operational feature points of the local point cloud are successfully verified based on the verification conditions.

[0162] S318. Based on the type of the target vehicle door handle, generate a corresponding operation command to be executed by the actuating component. For example, if the vehicle door handle is a push-button type, generate a first operation command.

[0163] The steps described above have been explained in detail and will not be repeated here.

[0164] It should be noted that, in the above steps, S304 to S311 can be understood as the tactile scanning of the vehicle door handle and the force feedback stage of the tactile scanning of the vehicle door handle. Steps S311 to S312 can be understood as the stage of accurately generating local point clouds. Steps S317 to S318 can be understood as the vehicle door handle confirmation stage.

[0165] In summary, the vehicle door handle modeling method of this application embodiment has at least one of the following technical effects:

[0166] 1. In scenarios where visual information is missing or occluded, accurately construct local point clouds of vehicle door handles through tactile exploration to ensure the reliability of vehicle door handle operation.

[0167] 2. By extracting the features of vehicle door handles and verifying the operational feature points of local point clouds, different types of vehicle door handles, such as push-button and pull-out types, can be accurately identified, adapting to the complex geometric structures of different vehicle door handle types.

[0168] 3. Force feedback and control can effectively avoid hard collisions between the actuators and the vehicle door handle, ensuring the safety of the actuator operation.

[0169] 4. Based on the global point cloud and the region of interest where the vehicle door handle is located, the vehicle door handle can be coarsely located, improving the accuracy of subsequent operations.

[0170] Figure 5 This is a schematic diagram of the vehicle door handle modeling device according to an embodiment of this application. Figure 5 As shown, the vehicle door handle modeling device provided in this application embodiment includes: a control module 410.

[0171] The control module 410 is used to control the robot's actuators to move toward the vehicle door handle at a first position relative to the vehicle door handle.

[0172] The control module 410 is also used to determine the initial contact point position based on the force data indicating that the actuator is in contact with the vehicle door handle, and to control the actuator to move along the vehicle door handle to obtain multiple moving contact point positions.

[0173] The control module 410 is also used to switch the first position at least once and / or the path of the actuator moving along the vehicle door handle and obtain the corresponding first contact point position and the corresponding multiple moving contact point positions.

[0174] The control module 410 is also used to generate a local point cloud of the vehicle door handle based on the initial contact point location and multiple moving contact point locations.

[0175] In one possible embodiment, the control module 410 is further configured to perform at least one of curvature calculation, normal vector analysis, and edge detection on the local point cloud to obtain the features of the vehicle door handle. The features of the vehicle door handle include at least one of a region with curvature greater than a threshold, a planar region, a curved surface region, and a door handle boundary. Based on the features of the vehicle door handle, the control module 410 registers a local point cloud template of a preset type of vehicle door handle with the local point cloud. Based on the first operation feature point in the successfully registered local point cloud template, the control module 410 determines the first operation feature point of the local point cloud. The first operation feature point of the local point cloud is used to distinguish between different types of vehicle door handles.

[0176] In one possible embodiment, the control module 410 is further configured to perform at least one of the following: If the first operational feature point of the local point cloud is a press center, verify the press center based on a first condition, the first condition including: the first region is a concave region or a planar region and the force data corresponding to the first region exhibits a continuous normal resistance characteristic; the first region is the region in the local point cloud where the press center is used to operate a press-type vehicle door handle; If the first operational feature point of the local point cloud is a rotation axis, verify the rotation axis based on a second condition, the second condition including: the second region is axial and... The force data in the second region exhibits torque response characteristics based on the central axis, with the rotation axis used to operate the pull-out type vehicle door handle. Based on the second operational feature point in the successfully registered local point cloud template, and assuming the second operational feature point of the local point cloud is a gripping point for grasping the vehicle door handle, the gripping point is verified based on the third condition. The third condition includes: the third region is located in the middle or end of the local point cloud; the third region is a continuous curved surface region; and the force data corresponding to the third region exhibits multi-directional mechanical stability characteristics. The third region is the region where the gripping point is located in the local point cloud.

[0177] In one possible embodiment, the control module 410 is further configured to determine the type of vehicle door handle based on the successfully registered local point cloud template; if the vehicle door handle is a push-button type, generate a first operation command, the first operation command instructing the actuator to apply a preset constant force on the vehicle door handle corresponding to the first area, detect the position change of the actuator until a first signal indicating completion of the first operation command is generated; if the vehicle door handle is a pull-out type, generate a second operation command, the second operation command instructing the actuator to apply a preset torque on the vehicle door handle corresponding to the second area, detect the angle and torque feedback of the actuator to determine whether the second operation command is completed.

[0178] In one possible embodiment, the control module 410 is specifically configured to control the robot's actuator to move towards the vehicle door handle from a first pose relative to the vehicle door handle using a preset target vehicle door handle type, following a target path corresponding to the target vehicle door handle type; and to register the local point cloud template of the target vehicle door handle type with the local point cloud based on the characteristics of the vehicle door handle. The control module 410 is further configured to, in the event of registration failure, adjust the target vehicle door handle type and return to controlling the robot's actuator to move towards the vehicle door handle from a first pose relative to the vehicle door handle using a preset target vehicle door handle type, following a target path corresponding to the target vehicle door handle type.

[0179] In one possible embodiment, the control module 410 is specifically used to adjust at least one of the pose, impedance, moving speed, and moving direction of the actuator, so that the value of the force data of the actuator contacting the vehicle door handle during the actuator's movement along the vehicle door handle is within a preset value range.

[0180] In one possible embodiment, the control module 410 is further configured to acquire a global point cloud of the vehicle door region; and based on the global point cloud, determine the region of interest where the vehicle door handle is located. Specifically, the control module 410 is further configured to control an actuator to move toward the vehicle door handle at a first pose relative to the vehicle door handle, based on the region of interest where the vehicle door handle is located.

[0181] In one possible embodiment, the control module 410 is specifically used to perform instance segmentation on the global point cloud using an instance segmentation model to determine the region of interest where the vehicle door handle is located; wherein, the training samples of the instance segmentation model include global point cloud samples of the contaminated vehicle door handle.

[0182] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the above-described method embodiments.

[0183] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0184] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0185] This application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the embodiments described above.

[0186] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0187] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0188] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0189] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0190] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0191] If the integrated unit / module is implemented as a software program module and sold or used as an independent financial product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software financial product. This computer software financial product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0192] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0193] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0194] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for modeling vehicle door handles, characterized in that, The method includes: The robot's actuators move toward the vehicle door handle from a first position relative to the vehicle door handle; When the force data indicating that the actuator is in contact with the vehicle door handle indicates that the actuator is in contact with the vehicle door handle, the position of the first contact point is determined based on the force data and the position data of the actuator, and the actuator is controlled to move along the vehicle door handle to obtain multiple moving contact point positions; The first pose is switched at least once and / or the path along which the actuator moves along the vehicle door handle is obtained, and the corresponding first contact point position and the corresponding multiple moving contact point positions are obtained. A local point cloud of the vehicle door handle is generated based on all initial contact point positions and multiple moving contact point positions obtained after at least one switch of the first pose and / or the path along which the actuator moves along the vehicle door handle, as well as the initial contact point positions and multiple moving contact point positions obtained before the switch.

2. The method according to claim 1, characterized in that, After generating the local point cloud of the vehicle door handle, the method further includes: At least one of curvature calculation, normal vector analysis, and edge detection is performed on the local point cloud to obtain the features of the vehicle door handle. The features of the vehicle door handle include at least one of the following: a region with curvature greater than a threshold, a planar region, a curved surface region, and a door handle boundary. Based on the characteristics of the vehicle door handle, a local point cloud template of a preset type of vehicle door handle is registered with the local point cloud. Based on the first operational feature point in the successfully registered local point cloud template, the first operational feature point of the local point cloud is determined. The first operational feature point of the local point cloud is used to distinguish the operation of various types of vehicle door handles.

3. The method according to claim 2, characterized in that, The method further includes at least one of the following: When the first operational feature point of the local point cloud is the pressing center, the pressing center is verified based on a first condition. The first condition includes: the first region is a concave region or a planar region and the force data corresponding to the first region shows a continuous normal resistance characteristic; the first region is the region of the pressing center in the local point cloud, and the pressing center is used to operate a pressing type vehicle door handle. When the first operational feature point of the local point cloud is the axis of rotation, the axis of rotation is verified based on the second condition. The second condition includes: the second region is axial and the force data of the second region exhibits torque response characteristics based on the central axis. The axis of rotation is used to operate the pull-out type vehicle door handle. Based on the second operational feature point in the successfully registered local point cloud template, if the second operational feature point of the local point cloud is determined to be a gripping point for gripping the vehicle door handle, the gripping point is verified based on a third condition. The third condition includes: a third region is located in the middle or end of the local point cloud; the third region is a continuous curved surface region; and the force data corresponding to the third region exhibits multi-directional mechanical stability characteristics. The third region is the region of the gripping point in the local point cloud.

4. The method according to claim 3, characterized in that, The method further includes: The type of the vehicle door handle is determined based on the successfully registered local point cloud template; When the vehicle door handle is a push-button type, a first operation command is generated. The first operation command instructs the actuator to apply a preset constant force on the vehicle door handle corresponding to the first area, detect the position change of the actuator, and generate a first signal indicating that the first operation command has been completed. When the vehicle door handle is an outward-pulling type, a second operation command is generated. The second operation command instructs the actuator to apply a preset torque on the vehicle door handle corresponding to the second area, and detects the angle and torque feedback of the actuator to determine whether the second operation command has been completed.

5. The method according to claim 2, characterized in that, The movement of the robot's actuator toward the vehicle door handle from its first pose relative to the vehicle door handle includes: moving the robot's actuator toward the vehicle door handle from its first pose relative to the vehicle door handle along a target path corresponding to the target vehicle door handle type, based on a preset target vehicle door handle type. The step of registering a local point cloud template of a preset type of vehicle door handle with the local point cloud based on the features of the vehicle door handle includes: registering a local point cloud template of the target vehicle door handle type with the local point cloud based on the features of the vehicle door handle. The method further includes: In the event of registration failure, the target vehicle door handle type is adjusted, and the robot's actuator is controlled to move towards the vehicle door handle from the first pose relative to the vehicle door handle using the preset target vehicle door handle type and the target path corresponding to the target vehicle door handle type.

6. The method according to any one of claims 1-5, characterized in that, The control of the actuator to move along the vehicle door handle includes: adjusting at least one of the actuator's pose, impedance, moving speed, and moving direction, so that during the actuator's movement along the vehicle door handle, the value of the force data of the actuator in contact with the vehicle door handle is within a preset value range.

7. The method according to any one of claims 1-5, characterized in that, Before the actuator of the controlled robot moves toward the vehicle door handle at a first pose relative to the vehicle door handle, the method further includes: Obtain the global point cloud of the vehicle door region; Based on the global point cloud, the region of interest where the vehicle door handle is located is determined; The control of the robot's actuator to move toward the vehicle door handle at a first pose relative to the vehicle door handle includes: controlling the actuator to move toward the vehicle door handle at a first pose relative to the vehicle door handle based on the region of interest where the vehicle door handle is located.

8. The method according to claim 7, characterized in that, The determination of the region of interest where the vehicle door handle is located based on the global point cloud includes: The global point cloud is segmented using an instance segmentation model to determine the region of interest where the vehicle door handle is located; wherein, the training samples of the instance segmentation model include global point cloud samples of the contaminated vehicle door handle.

9. A robot, characterized in that, include: Memory, processor, and execution unit; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

10. The robot according to claim 9, characterized in that, The actuator is mounted on the first robotic arm of the robot, which also includes a second robotic arm for assisting in supporting the vehicle where the door handle is located.

Images