Mechanical arm positioning taking and placing method and system for ultrasonic welding of dosing device

Abstract

The invention provides a mechanical arm positioning taking and placing method and system for ultrasonic welding of a drug delivery device, and relates to the technical field of mechanical arm positioning taking and placing. Image data of the drug delivery device are obtained and preprocessed, feature extraction is conducted based on the preprocessed image data of the drug delivery device, and three-dimensional positioning and posture analysis are conducted on the drug delivery device based on the extracted features; and planning a mechanical arm clamping path based on the obtained attitude analysis result and the three-dimensional positioning result. Three-dimensional positioning of the drug delivery device is achieved through image data preprocessing and posture analysis, the clamping and welding path of the mechanical arm is optimized through path planning, the accuracy of the welding process is ensured, the welding quality is detected and fed back in real time through the image monitoring system, the welding quality and the production efficiency are effectively improved, and the production cost is reduced. The automation level and the operation precision of the whole method are improved, and manual intervention and the defective product rate are remarkably reduced.

Description

Technical Field

[0001] The invention relates to the technical field of mechanical arm positioning, picking and placing, and in particular to a mechanical arm positioning, picking and placing method and system for ultrasonic welding of a drug delivery device. Background Art

[0002] With the continuous development of the medical device industry, applicators, as an auxiliary medical device, have higher and higher requirements for assembly and welding accuracy during the production process. The existing applicator assembly and welding process usually uses ultrasonic welding technology to ensure the stability of the welding parts. On complex production lines, how to effectively improve assembly accuracy and welding efficiency, especially the need for positioning, picking and placing, and welding with the help of automated equipment, is becoming increasingly urgent. The application of robotic arms in production has gradually been widely promoted, and automated operation can be achieved through its flexible operating performance.

[0003] At present, the robot arm usually needs to rely on sensors and image processing technology for precise positioning and control during the automated welding process of the dispenser. However, the existing technology has many shortcomings; First, during the delivery and positioning of the dispenser, due to inaccurate or delayed data acquisition by the sensor, the robot arm is prone to deviation when performing positioning and pick-and-place operations; Secondly, the real-time monitoring of the welding process is also relatively weak, and it is impossible to fully grasp the welding quality, which may cause problems such as loose welding or uneven welds; In addition, the lack of effective path planning and motion control technology makes it difficult for the robot to achieve a balanced speed and accuracy during gripping, transfer and welding operations.

[0004] Therefore, it is urgent to propose a method and system for positioning and placing a robotic arm for ultrasonic welding of a drug delivery device. Summary of the invention

[0005] The present invention provides a method and system for positioning and placing a robotic arm for ultrasonic welding of a drug dispenser, aiming to solve the practical problem in the related technology that the robotic arm is prone to deviation when performing positioning and placing operations, circumvent the technical problem of loose product welding caused by lack of real-time monitoring during welding, and avoid the subsequent problem of poor precision of the robotic arm due to lack of effective path planning and motion control technology.

[0006] In order to achieve the above purpose, the embodiments of the present application are implemented in the following manner: In a first aspect, an embodiment of the present application provides a method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device, characterized in that it includes the following: Step S100, based on the conveyor belt conveying the applicator, acquiring and preprocessing the image data of the applicator; Step S200, extracting features based on the preprocessed image data of the drug delivery device, and performing three-dimensional positioning and posture analysis on the drug delivery device based on the extracted features; Step S300, planning a gripping path of the robot arm based on the obtained posture analysis results and three-dimensional positioning results; Step S400: Based on the planned clamping path, the robot arm clamps the drug dispenser and transports it to the mechanical turntable for ultrasonic welding, and performs image monitoring on the drug dispenser during the welding process; Step S500, based on the image data of the applicator after welding, path planning is performed for the robot arm at the other end; Step S600: The robot arm at the other end grabs the processed applicator based on the planned path and transfers it to the conveyor belt, which transports the applicator at a uniform speed.

[0007] Based on the conveyor belt conveying the drug dispenser, the drug dispenser image data is obtained and preprocessed, feature extraction is performed based on the preprocessed drug dispenser image data, three-dimensional positioning and posture analysis of the drug dispenser is performed based on the extracted features, and the robot arm clamping path is planned based on the posture analysis results and three-dimensional positioning results. Based on the planned clamping path, the robot arm clamps the drug dispenser and transports it to the mechanical turntable for ultrasonic welding. During the welding process, the drug dispenser is image-monitored, and based on the image data of the drug dispenser after welding, the path planning is performed for the robot arm at the other end. The robot arm at the other end clamps the processed drug dispenser based on the planned path and transfers it to the conveyor belt, and the conveyor belt transports the drug dispenser at a uniform speed.

[0008] In combination with the first aspect, in some specific embodiments, based on the conveyor belt conveying the applicator, acquiring the applicator image data and preprocessing it specifically includes: Step S101, acquiring image data of the dispenser in a conveying state, including a material discharge process and a photographing process; Step S101.1, material discharge processing, manually placing the pre-assembled dispenser on the conveyor belt, and the conveyor belt is used to transport it at a uniform speed; Step S101.2, obtaining image data of the dispenser in a conveying state, by presetting a shooting area above the conveying area of ​​the dispenser, installing an industrial camera above the shooting area, shooting the dispenser, and obtaining continuous image data; The camera’s shooting frequency is adjusted according to the conveyor belt’s speed and the size of the dispenser to ensure that a clear and complete image of the dispenser is captured. The conveyor belt is transported to the designated shooting area; Step S102, preprocessing the image data of the applicator, the preprocessing including image denoising, grayscale processing, edge detection and image enhancement; Step S102.1, image denoising, using a Gaussian filter combined with a median filter to smooth the image, and obtain the denoised image data of the drug delivery device; Step S102.2, grayscale processing, converting the color image after the denoising process into a grayscale image, using a weighted average method to perform grayscale processing on the weights of the three channels, and obtaining the grayscale processed image data of the applicator; Step S102.3, edge detection, using the Canny edge detection algorithm to extract the edge of the grayscale processed image to highlight the outline of the applicator image data; Step S102.4: image enhancement, using histogram equalization to enhance the contrast of the image and the contour of the applicator image data; Step S102.5: Obtain pre-processed applicator image data based on steps S102.1 to S102.4.

[0009] In combination with the first aspect, in some specific embodiments, feature extraction is performed based on the preprocessed image data of the applicator, and three-dimensional positioning and posture analysis of the applicator are performed based on the extracted features, specifically including: Step S201, feature extraction, based on the detection result obtained in step S102.3, extract the complete contour information in the applicator image data, and identify the edge feature point position of the applicator image data; Step S202, three-dimensional positioning includes correction processing and conversion processing; Step S202.1, correction processing, obtaining the camera's internal parameters and distortion coefficients through camera calibration, and performing distortion correction until the geometric shape of the applicator in the image is consistent with the actual one; Step S202.2, conversion processing, using the PnP algorithm to convert the edge feature point coordinates on the applicator image data plane into coordinates in the three-dimensional world coordinate system; Step S202.3, obtaining the three-dimensional coordinate data of the drug delivery device image data based on step S202.1 and step S202.2; Step S203: posture analysis: based on the three-dimensional coordinate positioning data, the rotation matrix and translation vector of the applicator relative to the camera are solved by the EPnP algorithm to determine the posture of the applicator in the three-dimensional space; Step S203.1: Input processing, through the three-dimensional point set shows the position of the applicator in the world coordinate system, where , the corresponding two-dimensional image point set Represents the projection of a 3D point on the camera image plane; Step S203.2: Solving the camera intrinsic parameter matrix , specifically: In the formula, and The images are and The focal length of the direction, and are the coordinates of the center of the image; The EPnP algorithm constructs a system of equations based on the following projection relationship and solves it by minimizing the reprojection error and Specifically: In the formula, is the homogeneous coordinate of the image coordinate point, expressed as , yes The rotation matrix represents the rotation of the camera relative to the world coordinate system. yes The translation vector represents the displacement of the camera relative to the world coordinate system; Step S203.3: Output processing, output rotation matrix and translation vectors , specifically: Represents rotation in three-dimensional space; Represents the translation of the camera relative to the world coordinate system; Step S204, converting the rotation matrix obtained in step S203 into a quaternion representation; Step S204.1, calculate the quaternion, by calculating each component of the quaternion, specifically: In the formula, is a rotation matrix whose elements Represents the rotation matrix Line The elements of the column, Represents the vector part of the quaternion, describing the direction of the rotation axis, is the scalar part of the quaternion, describing the cosine half angle of the rotation angle, Step S204.2: Convert the output to obtain a quaternion , and the obtained quaternion is used for the subsequent attitude control of the robotic arm.

[0010] In combination with the first aspect, in some specific implementations, planning a gripping path of a robotic arm based on the obtained posture analysis results and three-dimensional positioning results specifically includes: Step S301, planning the gripping path of the robot arm includes inputting quaternion, three-dimensional positioning results and posture analysis results, inverse kinematics solution, trajectory generation and kinematics simulation; Step S302, input quaternion, three-dimensional positioning result and posture analysis result; Input quaternion, quaternion Used to describe the rotation information of the applicator in three-dimensional space; Input 3D positioning results ,The 3D positioning result is used to describe the position coordinates of the applicator in the world coordinate system; Inputting the posture analysis result, the posture analysis result includes the rotation axis direction of the drug delivery device and its angle relative to the camera; Step S303, inverse kinematics solution includes establishing a kinematic model of the robot arm and solving each joint angle; Step S303.1: Establish a kinematic model of the robot arm, and use DH parameters to describe the joints and connecting rods of the robot arm, specifically: In the formula, is the rotation angle, is the torsion angle of the connecting rod, is the length of the connecting rod, is the offset; Step S303.2, solve the angles of each joint and use the Jacobian matrix to solve so that the end of the robot arm reaches the position of the drug dispenser The joint angles of are: In the formula, Represents the relationship between the position change of the end effector and the change of the robot arm joint angle. is the angle of each joint; Step S304, trajectory generation includes posture interpolation and smooth path planning; Step S304.1, posture interpolation, using the quaternion interpolation method to generate a smooth rotation path between the starting and target postures, interpolation is performed based on the quaternion value obtained in S204.2, specifically: In the formula, is the angle between quaternions, and are the starting and target quaternions, is the interpolation factor, The value range is ; Step S304.2, smooth path planning, based on the posture interpolation results and the joint angles solved by inverse kinematics, a smooth clamping path is generated through B-spline interpolation, specifically: In the formula, Indicates that the robot arm is in parameter The spatial position at a given moment, Represents control points, each control point Corresponding to the three-dimensional coordinates of the robot at a specific position, Indicates control points Order B-spline basis function, which means that the parameters The influence of this control point on the curve shape; Among them, the use of B-spline interpolation can make the movement process of the robot arm smooth and continuous, ensuring that the path meets the gripping accuracy while avoiding exceeding the joint angle limit of the robot arm; Step S305, kinematic simulation, after the path is generated, dynamic simulation is performed to check whether the path meets the speed and acceleration limits of the robot arm, and the Newton-Euler method is used for dynamic analysis to determine the torque of each joint of the robot arm during the movement, specifically: By performing recursive operations on each joint, the forces and torques of each joint of the robotic arm are solved. The robotic arm is regarded as a connection system of multiple rigid bodies, and the motion state of each joint is derived through the relationship between force and acceleration. In the formula, Indicates The combined force of the connecting rods, Indicates The mass of the connecting rod, Indicates The acceleration of the connecting rod, Indicates The torque of the joint, Indicates The moment of inertia of the connecting rod, represents the angular acceleration, represents the angular velocity, and represent external force and external moment respectively; Among them, step S305 can ensure the smooth operation of the robot arm during the actual operation by selecting the above method to avoid excessive impact force.

[0011] Step S306: Based on steps S301 to S305, the angle change and posture rotation information of each joint of the robot arm are obtained to generate a complete motion instruction.

[0012] In combination with the first aspect, in some specific embodiments, based on the planned clamping path, the robot arm clamps the applicator and transports it to the mechanical turntable, performs ultrasonic welding, and performs image monitoring on the applicator during the welding process, specifically including: Step S401, generating and issuing a path instruction, based on the gripping path generated in step S300 and the angle change information of each joint of the robot arm, converting the data into a specific motion instruction, the motion instruction including a joint angle instruction, a speed and acceleration instruction, and a posture adjustment instruction; Among them, the joint angle command is used to indicate the target angle change of each joint of the robot arm during the gripping process; The speed and acceleration instructions are used to set the speed and acceleration of the robot during movement to ensure smooth path execution without excessive shock or vibration. The posture adjustment instruction is based on the quaternion posture information to adjust the posture of the end effector of the robot arm to ensure that the correct direction and angle of the applicator are maintained during the gripping process; Step S402: Robot arm motion control: when the robot arm executes the motion instruction, the robot arm performs real-time feedback control based on the built-in sensor; The sensors include position sensors and torque sensors, which ensure that the robot arm remains on the planned path during movement based on real-time feedback. If a deviation between the joint angle and the preset instruction is detected during execution, error correction is automatically performed to adjust the joint angle to ensure that the robot arm reaches the target position accurately. Step S403, performing a clamping action, when the end of the robot arm reaches the position of the applicator, the end clamp performs a clamping action; The opening and closing of the clamp is controlled by a servo motor. The specific steps are as follows: A1. Gripping and positioning: Based on the posture analysis results, the posture of the end effector is adjusted to align with the gripping point of the applicator; A2. Force control adjustment: During the clamp closing process, the force sensor is used to adjust the clamping force in real time to ensure that the appropriate clamping force is applied to the applicator to avoid damage.

[0013] Step S404: transport to the mechanical turntable. After the clamping is completed, the robot arm moves the applicator to the mechanical turntable according to the planned path, and releases the applicator at a predetermined position to prepare for ultrasonic welding; Step S405, ultrasonic welding: after the robot arm accurately positions the applicator on the convex engaging block of the welding table, the ultrasonic welding head applies pressure to the welding position of the applicator and transmits high-frequency vibration energy, so that the welding surface of the applicator is partially melted to achieve welding; Step S406: Image monitoring: an industrial camera is installed above the welding mechanical turntable to monitor the status of the dispenser during the welding process; Among them, the camera is connected to the system and can collect image data of the welding process in real time; Step S407, welding quality inspection, through image monitoring and edge detection algorithm, the weld of the dispenser is inspected to identify whether there is any defect in the weld; Among them, the edge detection algorithm uses the Canny edge detection algorithm. After the edge detection is completed, the extracted weld contour is compared with the ideal weld standard to detect whether there are any defects in the weld, and the weld continuity and weld uniformity are detected respectively; Weld continuity, by analyzing the edge continuity, detecting whether there is weld fracture or porosity; Weld uniformity, by measuring the change in weld width, to determine whether the welding is uniform; In the formula, is the weld width, and The upper and lower edge points of the weld are The coordinates of the location, is the number of edge points.

[0014] If there is no abnormality, the robot arm confirms the welding quality through the image monitoring system. If the inspection is qualified, it sends a completion signal to the control system and transfers the dispenser to the next station; If the test fails, the abnormal information is recorded, the welding position is readjusted and an alarm signal is sent to prompt the operator to check and intervene manually.

[0015] Among them, in step S400, the robot arm receives path planning instructions to achieve accurate clamping of the applicator and transport it to the welding position. During the welding process, the welding quality and position status are detected in real time through the image monitoring system to ensure the stability and reliability of the welding process. Finally, the robot arm makes feedback adjustments based on the results of image monitoring or transports the welded applicator to the next process.

[0016] In combination with the first aspect, in some specific implementations, path planning is performed on the robot arm at the other end based on the image data of the applicator after welding, specifically including: Step S501, image acquisition and analysis: after welding is completed, the image data of the applicator after welding is acquired through an industrial camera, and feature extraction and analysis of the image data are performed, including identifying the welding state, position and spatial posture of the applicator relative to the other end of the robot arm; Step S502: Based on the position and posture of the applicator after welding, the path planning of the robot arm at the other end is performed using inverse kinematics. The path planning method is consistent with the process in step S300, including inverse kinematics solution based on image data and B-spline interpolation to generate a smooth path. For details, refer to the specific process in steps S301 to S306. Step S503: Generate a robot arm motion instruction through path planning and output it to the robot arm at the other end, so that the robot arm reaches the position of the drug dispenser and completes the subsequent gripping and transfer operations.

[0017] In combination with the first aspect, in some specific embodiments, the robotic arm at the other end grasps the processed applicator based on the planned path and transfers it to the conveyor belt, and the conveyor belt transports the applicator at a uniform speed, specifically including: Step S601, clamping operation. The clamping operation in this step is consistent with the clamping operation method in step S400. For details, refer to the description of the clamping process in step S400. Step S602, transfer processing, after the other end of the robot arm completes the clamping, the robot arm moves the drug dispenser to the specified position of the conveyor belt based on the planned path, and the robot arm transports the drug dispenser to the conveyor belt based on the posture analysis result and the path interpolation data; Step S603, uniform speed conveying. After the robot arm at the other end reaches the specified position above the conveyor belt, it controls the clamp to gradually loosen to ensure that the drug dispenser falls smoothly on the conveyor belt. After the drug dispenser falls on the conveyor belt, the conveyor belt starts at a uniform speed and transports the drug dispenser to the next process position based on the preset speed.

[0018] In a second aspect, the present application further provides a robotic arm positioning and placing system for ultrasonic welding of a drug delivery device, comprising the following modules: A preprocessing module, used for conveying the applicator on a conveyor belt, acquiring image data of the applicator and preprocessing the image data; An image processing module is used to extract features based on the preprocessed image data of the applicator, and perform three-dimensional positioning and posture analysis of the applicator based on the extracted features; A first path planning module, used for planning a gripping path of the robot arm based on the obtained posture analysis results and three-dimensional positioning results; Ultrasonic welding module, which is used to clamp the applicator based on the planned clamping path, and transport it to the mechanical turntable for ultrasonic welding, and monitor the image of the applicator during the welding process; A second path planning module is used to plan the path of the robot arm at the other end based on the image data of the applicator after welding; The transfer module is used for the robotic arm at the other end to grasp the processed applicator based on the planned path and transfer it to the conveyor belt, which transports the applicator at a uniform speed.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention realizes the three-dimensional positioning of the applicator through precise image data preprocessing and posture analysis, optimizes the clamping and welding path of the robot arm through path planning, ensures the smoothness and precision of the welding process, and performs real-time detection and feedback of the welding quality through the image monitoring system, effectively improving the welding quality and production efficiency, improving the automation level and operating accuracy of the overall method, and significantly reducing manual intervention and defective product rate.

[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A flow chart of a method for positioning and placing a robotic arm for ultrasonic welding of a drug delivery device provided in an embodiment of the present application.

[0023] Figure 2 A schematic diagram of a robotic arm positioning and placing system for ultrasonic welding of a drug dispenser provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0025] See also Figure 1 , Figure 1 A flow chart of a method for positioning and placing a robot arm for ultrasonic welding of a drug dispenser is provided for the example of the present application.

[0026] In this embodiment, a method for positioning and placing a drug delivery device by a robot arm for ultrasonic welding may include steps S100, S200, S300, S400, S500, and S600.

[0027] Step S100: Based on the conveyor belt conveying the applicator, image data of the applicator is acquired and pre-processed.

[0028] Here, specifically include: Step S101, acquiring image data of the dispenser in a conveying state, including a material discharge process and a photographing process; Step S101.1, material discharge processing, manually placing the pre-assembled dispenser on the conveyor belt, and the conveyor belt is used to transport it at a uniform speed; Step S101.2, obtaining image data of the dispenser in a conveying state, by presetting a shooting area above the conveying area of ​​the dispenser, installing an industrial camera above the shooting area, shooting the dispenser, and obtaining continuous image data; Step S102, preprocessing the image data of the applicator, the preprocessing including image denoising, grayscale processing, edge detection and image enhancement; Step S102.1, image denoising, using a Gaussian filter combined with a median filter to smooth the image, and obtain the denoised image data of the drug delivery device; Step S102.2, grayscale processing, converting the color image after the denoising process into a grayscale image, using a weighted average method to perform grayscale processing on the weights of the three channels, and obtaining the grayscale processed image data of the applicator; Step S102.3, edge detection, using the Canny edge detection algorithm to extract the edge of the grayscale processed image to highlight the outline of the applicator image data; Step S102.4: image enhancement, using histogram equalization to enhance the contrast of the image and the contour of the applicator image data; Step S102.5: Obtain pre-processed applicator image data based on steps S102.1 to S102.4.

[0029] Step S200: extract features based on the preprocessed image data of the drug dispenser, and perform three-dimensional positioning and posture analysis on the drug dispenser based on the extracted features.

[0030] Here, specifically include: Step S201, feature extraction, based on the detection result obtained in step S102.3, extract the complete contour information in the applicator image data, and identify the edge feature point position of the applicator image data; Step S202, three-dimensional positioning includes correction processing and conversion processing; Step S202.1, correction processing, obtaining the camera's internal parameters and distortion coefficients through camera calibration, and performing distortion correction until the geometric shape of the applicator in the image is consistent with the actual one; Step S202.2, conversion processing, using the PnP algorithm to convert the edge feature point coordinates on the applicator image data plane into coordinates in the three-dimensional world coordinate system; Step S202.3, obtaining the three-dimensional coordinate data of the drug delivery device image data based on step S202.1 and step S202.2; Step S203: posture analysis: based on the three-dimensional coordinate positioning data, the rotation matrix and translation vector of the applicator relative to the camera are solved by the EPnP algorithm to determine the posture of the applicator in the three-dimensional space; Step S203.1: Input processing, through the three-dimensional point set shows the position of the applicator in the world coordinate system, where , the corresponding two-dimensional image point set Represents the projection of a 3D point on the camera image plane; Step S203.2: Solving the camera intrinsic parameter matrix , specifically: In the formula, and The images are and The focal length of the direction, and are the coordinates of the center of the image; The EPnP algorithm constructs a system of equations based on the following projection relationship and solves it by minimizing the reprojection error and Specifically: In the formula, is the homogeneous coordinate of the image coordinate point, expressed as , yes The rotation matrix represents the rotation of the camera relative to the world coordinate system. yes The translation vector represents the displacement of the camera relative to the world coordinate system; Step S203.3: Output processing, output rotation matrix and translation vectors , specifically: Represents rotation in three-dimensional space; Represents the translation of the camera relative to the world coordinate system; Step S204, converting the rotation matrix obtained in step S203 into a quaternion representation; Step S204.1, calculate the quaternion, by calculating each component of the quaternion, specifically: In the formula, is a rotation matrix whose elements Represents the rotation matrix Line The elements of the column, Represents the vector part of the quaternion, describing the direction of the rotation axis, is the scalar part of the quaternion, describing the cosine half angle of the rotation angle, Step S204.2: Convert the output to obtain a quaternion , and the obtained quaternion is used for the subsequent attitude control of the robotic arm.

[0031] Step S300: planning a gripping path of the robot arm based on the obtained posture analysis results and three-dimensional positioning results.

[0032] Here, specifically include: Step S301, planning the gripping path of the robot arm includes inputting quaternion, three-dimensional positioning results and posture analysis results, inverse kinematics solution, trajectory generation and kinematics simulation; Step S302, input quaternion, three-dimensional positioning result and posture analysis result; Input quaternion, quaternion Used to describe the rotation information of the applicator in three-dimensional space; Input 3D positioning results ,The 3D positioning result is used to describe the position coordinates of the applicator in the world coordinate system; Inputting the posture analysis result, the posture analysis result includes the rotation axis direction of the drug delivery device and its angle relative to the camera; Step S303, inverse kinematics solution includes establishing a kinematic model of the robot arm and solving each joint angle; Step S303.1: Establish a kinematic model of the robot arm, and use DH parameters to describe the joints and connecting rods of the robot arm, specifically: In the formula, is the rotation angle, is the torsion angle of the connecting rod, is the length of the connecting rod, is the offset; Step S303.2, solve the angles of each joint and use the Jacobian matrix to solve so that the end of the robot arm reaches the position of the drug dispenser The joint angles of are: In the formula, Represents the relationship between the position change of the end effector and the change of the robot arm joint angle. is the angle of each joint; Step S304, trajectory generation includes posture interpolation and smooth path planning; Step S304.1, posture interpolation, using the quaternion interpolation method to generate a smooth rotation path between the starting and target postures, interpolation is performed based on the quaternion value obtained in S204.2, specifically: In the formula, is the angle between quaternions, and are the starting and target quaternions, is the interpolation factor, The value range is ; Step S304.2, smooth path planning, based on the posture interpolation results and the joint angles solved by inverse kinematics, a smooth clamping path is generated through B-spline interpolation, specifically: In the formula, Indicates that the robot arm is in parameter The spatial position at a given moment, Represents control points, each control point Corresponding to the three-dimensional coordinates of the robot at a specific position, Indicates control points Order B-spline basis function, which means that the parameters The influence of this control point on the curve shape; Step S305, kinematic simulation, after the path is generated, dynamic simulation is performed to check whether the path meets the speed and acceleration limits of the robot arm, and the Newton-Euler method is used for dynamic analysis to determine the torque of each joint of the robot arm during the movement, specifically: By performing recursive operations on each joint, the forces and torques of each joint of the robotic arm are solved. The robotic arm is regarded as a connection system of multiple rigid bodies, and the motion state of each joint is derived through the relationship between force and acceleration. In the formula, Indicates The combined force of the connecting rods, Indicates The mass of the connecting rod, Indicates The acceleration of the connecting rod, Indicates The torque of the joint, Indicates The moment of inertia of the connecting rod, represents the angular acceleration, represents the angular velocity, and represent external force and external moment respectively; Step S306: Based on steps S301 to S305, the angle change and posture rotation information of each joint of the robot arm are obtained to generate a complete motion instruction.

[0033] Step S400: Based on the planned clamping path, the robot arm clamps the drug dispenser and transports it to the mechanical turntable for ultrasonic welding, and performs image monitoring of the drug dispenser during the welding process.

[0034] Here, specifically include: Step S401, generating and issuing a path instruction, based on the gripping path generated in step S300 and the angle change information of each joint of the robot arm, converting the data into a specific motion instruction, the motion instruction including a joint angle instruction, a speed and acceleration instruction, and a posture adjustment instruction; Among them, the joint angle command is used to indicate the target angle change of each joint of the robot arm during the gripping process; The speed and acceleration instructions are used to set the speed and acceleration of the robot during movement to ensure smooth path execution without excessive shock or vibration. The posture adjustment instruction is based on the quaternion posture information to adjust the posture of the end effector of the robot arm to ensure that the correct direction and angle of the applicator are maintained during the gripping process; Step S402: Robot arm motion control: when the robot arm executes the motion instruction, the robot arm performs real-time feedback control based on the built-in sensor; The sensors include position sensors and torque sensors, which ensure that the robot arm remains on the planned path during movement based on real-time feedback. If a deviation between the joint angle and the preset instruction is detected during execution, error correction is automatically performed to adjust the joint angle to ensure that the robot arm reaches the target position accurately. Step S403, performing a clamping action, when the end of the robot arm reaches the position of the applicator, the end clamp performs a clamping action;

[0035] Step S404: transport to the mechanical turntable. After the clamping is completed, the robot arm moves the applicator to the mechanical turntable according to the planned path, and releases the applicator at a predetermined position to prepare for ultrasonic welding; Step S405, ultrasonic welding: after the robot arm accurately positions the applicator on the convex engaging block of the welding table, the ultrasonic welding head applies pressure to the welding position of the applicator and transmits high-frequency vibration energy, so that the welding surface of the applicator is partially melted to achieve welding; Step S406: Image monitoring: an industrial camera is installed above the welding mechanical turntable to monitor the status of the dispenser during the welding process; Step S407, welding quality inspection, through image monitoring and edge detection algorithm, the weld of the dispenser is inspected to identify whether there is any defect in the weld; Among them, the edge detection algorithm uses the Canny edge detection algorithm. After the edge detection is completed, the extracted weld contour is compared with the ideal weld standard to detect whether there are any defects in the weld, and the weld continuity and weld uniformity are detected respectively; Weld continuity, by analyzing the edge continuity, detecting whether there is weld fracture or porosity; Weld uniformity, by measuring the change in weld width, to determine whether the welding is uniform; In the formula, is the weld width, and The upper and lower edge points of the weld are The coordinates of the location, is the number of edge points.

[0036] If there is no abnormality, the robot arm confirms the welding quality through the image monitoring system. If the inspection is qualified, it sends a completion signal to the control system and transfers the dispenser to the next station; If the test fails, the abnormal information is recorded, the welding position is readjusted and an alarm signal is sent to prompt the operator to check and intervene manually.

[0037] Step S500: Based on the image data of the applicator after welding, path planning is performed for the robot arm at the other end.

[0038] Here, specifically include: Step S501, image acquisition and analysis: after welding is completed, the image data of the applicator after welding is acquired through an industrial camera, and feature extraction and analysis of the image data are performed, including identifying the welding state, position and spatial posture of the applicator relative to the other end of the robot arm; Step S502: Based on the position and posture of the applicator after welding, the path planning of the robot arm at the other end is performed using inverse kinematics. The path planning method is consistent with the process in step S300, including inverse kinematics solution based on image data and B-spline interpolation to generate a smooth path. For details, refer to the specific process in steps S301 to S306. Step S503: Generate a robot arm motion instruction through path planning and output it to the robot arm at the other end, so that the robot arm reaches the position of the drug dispenser and completes the subsequent gripping and transfer operations.

[0039] Step S600: The robot arm at the other end grabs the processed applicator based on the planned path and transfers it to the conveyor belt, which transports the applicator at a uniform speed.

[0040] Here, specifically include: Step S601, clamping operation. The clamping operation in this step is consistent with the clamping operation method in step S400. For details, refer to the description of the clamping process in step S400. Step S602, transfer processing, after the other end of the robot arm completes the clamping, the robot arm moves the drug dispenser to the specified position of the conveyor belt based on the planned path, and the robot arm transports the drug dispenser to the conveyor belt based on the posture analysis result and the path interpolation data; Step S603, uniform speed conveying. After the robot arm at the other end reaches the specified position above the conveyor belt, it controls the clamp to gradually loosen to ensure that the drug dispenser falls smoothly on the conveyor belt. After the drug dispenser falls on the conveyor belt, the conveyor belt starts at a uniform speed and transports the drug dispenser to the next process position based on the preset speed.

[0041] When the method is actually used, first, the applicator is transported by a conveyor belt, and the pre-assembled applicator is manually placed on the conveyor belt, and the conveyor belt transports it to the shooting area at a uniform speed. The industrial camera is installed above the conveyor belt to capture continuous images of the applicator. The shooting frequency of the camera is adjusted according to the speed of the conveyor belt and the size of the applicator to ensure that the image is clear and complete; Subsequently, the acquired image data is processed, and the preprocessing includes: removing noise by Gaussian filtering and median filtering, graying the image, using the Canny algorithm to perform edge detection to extract the outline of the applicator, and enhancing the image contrast by histogram equalization to generate preprocessed image data; After the preprocessing, the complete outline of the applicator is extracted and its edge feature points are identified based on the results of edge detection. The PnP algorithm is used to convert the extracted two-dimensional feature points into three-dimensional coordinates. The position of the applicator in space is obtained through camera calibration and correction. The EPnP algorithm is then used to calculate the rotation matrix and translation vector of the applicator to describe its posture in three-dimensional space. The rotation matrix is ​​converted into a quaternion through a formula to describe the rotation direction and angle of the applicator.

[0042] Afterwards, based on the obtained three-dimensional positioning results and posture information, the quaternion and position information are input, and then the kinematic model of the robot arm is established according to the DH parameters. The Jacobian matrix is ​​used to solve the angles of each joint of the robot arm to ensure that the end effector of the robot arm can accurately reach the position of the drug dispenser. The B-spline interpolation method is used to generate a continuous and smooth clamping path based on the solved joint angles and smooth path planning. Then, the Newton-Euler method is used for dynamic simulation to check whether the path meets the speed and acceleration limits of the robot arm to ensure stable operation of the robot arm.

[0043] At this time, the robot arm will grasp the applicator based on the planned path and accurately transfer it to the mechanical turntable. During the process, the joint angle and posture are adjusted in real time. On the mechanical turntable, the robot arm accurately positions the applicator, and the ultrasonic welding head applies pressure and transmits high-frequency vibration energy to weld the contact surface of the applicator. The industrial camera installed on the welding table monitors the welding process in real time, detects the quality of the weld, and determines whether the welding is qualified through image processing algorithms.

[0044] After welding is completed, the industrial camera obtains the image data of the applicator after welding and analyzes it, including the welding status, position and posture. Based on the analysis results, the robotic arm at the other end solves the path through inverse kinematics and uses B-spline interpolation to generate a smooth path to ensure the stability of the clamping process.

[0045] The robotic arm at the other end accurately grasps the processed applicator based on the planned path and transfers it to the conveyor belt. After the robotic arm releases the applicator, the conveyor belt transports the applicator to the next process at a uniform speed.

[0046] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a robotic arm positioning and picking system for ultrasonic welding of a drug delivery device in the present application.

[0047] A robotic arm positioning and placing system for ultrasonic welding of a drug delivery device, comprising the following modules: A preprocessing module, used for conveying the applicator on a conveyor belt, acquiring image data of the applicator and preprocessing the image data; An image processing module is used to extract features based on the preprocessed image data of the applicator, and perform three-dimensional positioning and posture analysis of the applicator based on the extracted features; A first path planning module, used for planning a gripping path of the robot arm based on the obtained posture analysis results and three-dimensional positioning results; Ultrasonic welding module, which is used to clamp the applicator based on the planned clamping path, and transport it to the mechanical turntable for ultrasonic welding, and monitor the image of the applicator during the welding process; A second path planning module is used to plan the path of the robot arm at the other end based on the image data of the applicator after welding; The transfer module is used for the robotic arm at the other end to pick up the processed applicator based on the planned path and transfer it to the conveyor belt, which transports the applicator at a uniform speed. The above description is only a specific implementation mode of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present invention, which should be covered within the protection scope of the embodiments of the present invention.

Claims

1. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device, characterized in that: These include: S100, based on the conveyor belt conveying the applicator, acquiring and preprocessing the image data of the applicator; S200, extracting features based on the preprocessed image data of the drug delivery device, and performing three-dimensional positioning and posture analysis on the drug delivery device based on the extracted features; S300, planning a gripping path for the robotic arm based on the obtained posture analysis results and three-dimensional positioning results; S400, based on the planned gripping path, the robotic arm grips the drug dispenser and transports it to the mechanical turntable for ultrasonic welding, and performs image monitoring on the drug dispenser during the welding process; S500, based on the image data of the drug dispenser after welding, path planning is performed on the robot arm at the other end; S600, the robotic arm at the other end grabs the processed dispenser based on the planned path and transfers it to the conveyor belt, which transports the dispenser at a uniform speed.

2. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 1, characterized in that: Based on the conveyor belt conveying the applicator, the applicator image data is obtained and preprocessed, including: S101, acquiring image data of the dispenser in a conveying state, including a material discharge process and a photographing process; S101.1, material discharge processing, manually place the pre-assembled dispenser on the conveyor belt, and the conveyor belt transports it at a uniform speed; S101.2, obtaining image data of the dispenser in the conveying state, by presetting a shooting area above the conveying area of ​​the dispenser, installing an industrial camera above the shooting area, shooting the dispenser, and obtaining continuous image data; S102, preprocessing of the image data of the applicator, including image denoising, grayscale processing, edge detection and image enhancement; S102.1, image denoising, using a Gaussian filter combined with a median filter to smooth the image, and obtaining the denoised image data of the drug delivery device; S102.2, grayscale processing, converting the color image after denoising into a grayscale image, using weighted average method to perform grayscale processing on the weights of three channels, and obtaining the image data of the applicator after grayscale processing; S102.3, edge detection, using the Canny edge detection algorithm to extract the edge of the image after grayscale processing, and highlight the outline of the applicator image data; S102.4, image enhancement, using histogram equalization to enhance the contrast of the image and enhance the contour of the applicator image data; S102.

5. Obtain pre-processed applicator image data based on S102.1 to S102.

4.

3. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 2, characterized in that: Feature extraction is performed based on the preprocessed image data of the applicator, and three-dimensional positioning and posture analysis of the applicator is performed based on the extracted features, specifically including: S201, feature extraction, based on the detection results obtained in S102.3, extract the complete contour information in the applicator image data, and identify the edge feature point positions of the applicator image data; S202, three-dimensional positioning includes correction processing and conversion processing; S202.1, correction processing, obtaining the camera's internal parameters and distortion coefficients through camera calibration, and performing distortion correction until the geometric shape of the applicator in the image is consistent with the actual one; S202.2, conversion processing, using the PnP algorithm to convert the edge feature point coordinates on the applicator image data plane into coordinates in the three-dimensional world coordinate system; S202.3, based on S202.1 and S202.2, obtain the three-dimensional coordinate data of the image data of the drug delivery device; S203, posture analysis, based on the three-dimensional coordinate positioning data, the rotation matrix and translation vector of the applicator relative to the camera are solved by the EPnP algorithm, so as to determine the posture of the applicator in the three-dimensional space; S203.

1. Input processing, through three-dimensional point set shows the position of the applicator in the world coordinate system, where , the corresponding two-dimensional image point set Represents the projection of a 3D point on the camera image plane; S203.2, solution processing, first solve the camera internal parameter matrix , specifically: In the formula, and The images are and The focal length of the direction, and are the coordinates of the center of the image; The EPnP algorithm constructs a system of equations based on the following projection relationship and solves it by minimizing the reprojection error and Specifically: In the formula, is the homogeneous coordinate of the image coordinate point, expressed as , yes The rotation matrix represents the rotation of the camera relative to the world coordinate system. yes The translation vector represents the displacement of the camera relative to the world coordinate system; S203.

3. Output processing, output rotation matrix and translation vectors , specifically: Represents rotation in three-dimensional space; Represents the translation of the camera relative to the world coordinate system; S204, converting the rotation matrix obtained in S203 into a quaternion representation; S204.

1. Calculate the quaternion by calculating the components of the quaternion, specifically: In the formula, is a rotation matrix whose elements Represents the rotation matrix Line The elements of the column, Represents the vector part of the quaternion, describing the direction of the rotation axis, is the scalar part of the quaternion, describing the cosine half angle of the rotation angle, S204.

2. Convert the output to get the quaternion , and the obtained quaternion is used for the subsequent attitude control of the robotic arm.

4. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 3, characterized in that: Based on the posture analysis results and three-dimensional positioning results, the robot arm gripping path is planned, including: S301, planning the gripping path of the robot arm includes inputting quaternion, three-dimensional positioning results and posture analysis results, inverse kinematics solution, trajectory generation and kinematics simulation; S302, input quaternion, three-dimensional positioning result and posture analysis result; Input quaternion, quaternion Used to describe the rotation information of the applicator in three-dimensional space; Input 3D positioning results ,The 3D positioning result is used to describe the position coordinates of the applicator in the world coordinate system; Inputting the posture analysis result, the posture analysis result includes the rotation axis direction of the drug delivery device and its angle relative to the camera; S303, inverse kinematics solution includes establishing a kinematic model of the robot arm and solving each joint angle; S303.

1. Establish the kinematic model of the robot arm and use DH parameters to describe the joints and connecting rods of the robot arm. Specifically: In the formula, is the rotation angle, is the torsion angle of the connecting rod, is the length of the connecting rod, is the offset; S303.

2. Solve the angles of each joint and use the Jacobian matrix to make the end of the robot reach the position of the drug dispenser The joint angles of are: In the formula, Represents the relationship between the position change of the end effector and the change of the robot arm joint angle. is the angle of each joint; S304, trajectory generation includes posture interpolation and smooth path planning; S304.1, posture interpolation, using the quaternion interpolation method to generate a smooth rotation path between the starting and target postures, based on the quaternion values ​​obtained in S204.2, interpolation is performed, specifically: In the formula, is the angle between quaternions, and are the starting and target quaternions, is the interpolation factor, The value range is ; S304.2, smooth path planning, based on the obtained posture interpolation results and the joint angles solved by inverse kinematics, a smooth clamping path is generated through B-spline interpolation, specifically: In the formula, Indicates that the robot arm is in parameter The spatial position at a given moment, Represents control points, each control point Corresponding to the three-dimensional coordinates of the robot at a specific position, Indicates control points Order B-spline basis function, which means that the parameters The influence of this control point on the curve shape; S305, kinematic simulation, after the path is generated, dynamic simulation is performed to check whether the path meets the speed and acceleration limits of the robot arm, and the Newton-Euler method is used for dynamic analysis to determine the torque of each joint of the robot arm during the movement process, specifically: By performing recursive operations on each joint, the forces and torques of each joint of the robotic arm are solved. The robotic arm is regarded as a connection system of multiple rigid bodies, and the motion state of each joint is derived through the relationship between force and acceleration. In the formula, Indicates The combined force of the connecting rods, Indicates The mass of the connecting rod, Indicates The acceleration of the connecting rod, Indicates The torque of the joint, Indicates The moment of inertia of the connecting rod, represents the angular acceleration, represents the angular velocity, and represent external force and external moment respectively; S306 , based on the angle change and posture rotation information of each joint of the robot arm obtained from S301 to S305 , a complete motion instruction is generated.

5. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 1, characterized in that: Based on the planned gripping path, the robot arm grips the applicator and transports it to the mechanical turntable for ultrasonic welding. During the welding process, the robot monitors the applicator, including: S401, generating and issuing path instructions, based on the gripping path generated in S300 and the angle change information of each joint of the robot arm, converting the data into specific motion instructions, the motion instructions including joint angle instructions, speed and acceleration instructions, and posture adjustment instructions; S402, robot arm motion control, when the robot arm executes the motion instruction, the robot arm performs real-time feedback control based on the built-in sensor; S403, performing a clamping action, when the end of the robot arm reaches the position of the applicator, the end clamp performs a clamping action; S404, transporting to the mechanical turntable, after the clamping is completed, the robotic arm moves the applicator to the mechanical turntable according to the planned path, and releases the applicator at a predetermined position, preparing for ultrasonic welding; S405, ultrasonic welding: after the robot arm accurately positions the applicator on the convex engaging block of the welding table, the ultrasonic welding head applies pressure to the welding position of the applicator and transmits high-frequency vibration energy, so that the welding surface of the applicator is partially melted to achieve welding; S406, image monitoring, an industrial camera is installed above the mechanical turntable for welding to monitor the status of the dispenser during the welding process; S407, welding quality inspection, through image monitoring and edge detection algorithm, the weld of the dispenser after welding is inspected to identify whether there is any defect in the weld; If there is no abnormality, the robot arm confirms the welding quality through the image monitoring system. If the inspection is qualified, it sends a completion signal to the control system and transfers the dispenser to the next station; If the test fails, the abnormal information is recorded, the welding position is readjusted and an alarm signal is sent to prompt the operator to check and intervene manually.

6. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 4, characterized in that: Based on the image data of the dispenser after welding, the path planning of the robot arm at the other end is carried out, including: S501, image acquisition and analysis: after welding is completed, the image data of the applicator after welding is acquired through an industrial camera, and feature extraction and analysis of the image data are performed, including identifying the welding state, position and spatial posture of the applicator relative to the other end of the robot arm; S502, based on the position and posture of the applicator after welding, the path planning of the robot arm at the other end is performed by inverse kinematics, and the path planning method is consistent with the process in S300, including inverse kinematics solution based on image data and B-spline interpolation to generate a smooth path, and specific reference is made to the specific processes in S301 to S306; S503. Generate a robot arm motion command through path planning and output it to the robot arm at the other end, so that the robot arm reaches the position of the drug dispenser and completes the subsequent gripping and transfer operations.

7. A method for positioning and placing a robot arm for ultrasonic welding of a drug delivery device as claimed in claim 1, characterized in that: The robotic arm at the other end grabs the processed applicator based on the planned path and transfers it to the conveyor belt, which transports the applicator at a constant speed, including: S601, clamping operation. The clamping operation in this step is consistent with the clamping operation method in step S400. For details, refer to the description of the clamping process in S400. S602, transfer processing, after the other end of the robot arm completes the clamping, the robot arm moves the drug dispenser to the specified position of the conveyor belt based on the planned path, and the robot arm transports the drug dispenser to the conveyor belt based on the posture analysis result and the path interpolation data; S603, uniform speed conveying. After the robot arm at the other end reaches the specified position above the conveyor belt, it controls the clamp to gradually loosen to ensure that the dispenser falls smoothly on the conveyor belt. After the dispenser falls on the conveyor belt, the conveyor belt starts at a uniform speed and transports the dispenser to the next process position based on the preset speed.

8. A mechanical arm positioning and placing system for ultrasonic welding of a drug delivery device, using a mechanical arm positioning and placing method for ultrasonic welding of a drug delivery device as claimed in any one of claims 1 to 7, characterized in that: Includes the following modules: A preprocessing module, used for conveying the applicator on a conveyor belt, acquiring image data of the applicator and preprocessing the image data; An image processing module is used to extract features based on the preprocessed image data of the applicator, and perform three-dimensional positioning and posture analysis of the applicator based on the extracted features; A first path planning module, used for planning a gripping path of the robot arm based on the obtained posture analysis results and three-dimensional positioning results; Ultrasonic welding module, which is used to clamp the applicator based on the planned clamping path, and transport it to the mechanical turntable for ultrasonic welding, and monitor the image of the applicator during the welding process; A second path planning module is used to plan the path of the robot arm at the other end based on the image data of the applicator after welding; The transfer module is used for the robotic arm at the other end to grasp the processed applicator based on the planned path and transfer it to the conveyor belt, which transports the applicator at a uniform speed.

Images