Control method and system of molten iron sampling robot

Abstract

The invention provides a control method and system for a molten iron sampling robot, and the method comprises the steps: determining a hand-eye conversion matrix between a camera coordinate system and a base coordinate system of the molten iron sampling robot through a hand-eye calibration method; according to the molten iron container image, the position of the molten iron container under the base coordinate system is determined in combination with the hand-eye conversion matrix; according to the D-H parameters, the transformation relation between joint coordinate systems of a mechanical arm of the molten iron sampling robot is analyzed, and a mechanical arm total transformation matrix of the tail end of the mechanical arm of the molten iron sampling robot relative to a base of the molten iron sampling robot is established; according to the total transformation matrix and the position of the molten iron container, the joint angle and the joint telescopic length of each joint of the mechanical arm are solved through kinematics inverse operation, then the molten iron sampling robot is controlled, the molten iron sampling robot is controlled, and accurate control over the molten iron sampling robot is achieved. The accuracy and the reliability of molten iron sampling are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to robots, and in particular relates to a control method and system for a molten iron sampling robot. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the steel production process, accurate and efficient molten iron sampling and silicon measurement are key links to ensure product quality and production process control. Traditional molten iron sampling and silicon measurement methods mainly rely on manual operation or simple mechanical devices, which have many shortcomings.

[0004] With the increasing intelligence of steel enterprises, more and more molten iron sampling is achieved by robots. However, the molten iron sampling robot has multiple joints, and the adjacent joints are linked joints. The existing control method may make it impossible to accurately determine the adjustment angle and length of the joints, increase the control difficulty, and cause problems such as reduced reliability and limited adaptability. Therefore, how to control the molten iron sampling robot to improve the accuracy of the joint adjustment of the molten iron sampling robot's mechanical arm, thereby achieving accurate sampling of molten iron and improving the reliability of control is a problem that needs to be solved at present. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a control method and system for a molten iron sampling robot, which improves the accuracy of adjusting the joints of the mechanical arm of the molten iron sampling robot and realizes precise control of the molten iron sampling robot.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a control method for a molten iron sampling robot, comprising:

[0008] The hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the molten iron sampling robot's mechanical arm is determined by the hand-eye calibration method;

[0009] Acquire an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the coordinate system of the end of the mechanical arm of the molten iron sampling robot according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot;

[0010] According to the DH parameters, the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot is analyzed, and the total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot is established;

[0011] According to the established total transformation matrix of the robot arm and the determined position of the molten iron container, the joint angles and joint extension lengths of each joint of the robot arm are solved by using inverse kinematics operation;

[0012] The movement of the molten iron sampling robot is controlled according to the obtained joint angles and joint extension lengths of each joint.

[0013] In a second aspect, the present invention provides a control system for a molten iron sampling robot, comprising:

[0014] The first determination module is configured to: determine the hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the mechanical arm of the molten iron sampling robot by a hand-eye calibration method;

[0015] The second determination module is configured to: obtain an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the coordinate system of the end of the robot mechanical arm according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot;

[0016] A transformation matrix establishment module is configured to: analyze the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot according to the DH parameters, and establish a total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot;

[0017] A solution module is configured to: solve the joint angle and joint extension length of each joint of the robot arm by using inverse kinematics operation according to the established total transformation matrix of the robot arm and the determined position of the molten iron container;

[0018] The control module is configured to control the movement of the molten iron sampling robot according to the obtained joint angles and joint extension lengths of each joint.

[0019] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0021] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0022] One or more of the above technical solutions have the following beneficial effects:

[0023] In the present invention, a hand-eye conversion matrix between a camera coordinate system and a coordinate system of a terminal end of a mechanical arm of a molten iron sampling robot is determined by a hand-eye calibration method; the position of the molten iron container in the coordinate system of the terminal end of the mechanical arm is determined according to an image of the molten iron container in combination with the hand-eye conversion matrix; the transformation relationship between the coordinate systems of each joint of the mechanical arm of the molten iron sampling robot is analyzed according to DH parameters, and a total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the terminal end of the mechanical arm of the molten iron sampling robot is established; according to the total transformation matrix and the position of the molten iron container, the joint angles and joint extension lengths of each joint of the mechanical arm are solved by inverse kinematic operations, and then the molten iron sampling robot is controlled, and the joint adjustment angles and extension lengths of each joint can be accurately obtained, thereby achieving precise control of the molten iron sampling robot and improving the accuracy and reliability of molten iron sampling.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a schematic diagram of the overall structure of the molten iron vision robot in the first embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of an improved sampling spoon in Example 1 of the present invention;

[0028] Figure 3 This is a hardware architecture diagram of the molten iron vision robot in the first embodiment of the present invention;

[0029] Figure 4 This is a hardware architecture diagram of a motion control system in Embodiment 1 of the present invention;

[0030] Figure 5 This is a schematic diagram of the working process of the molten iron vision robot in the second embodiment of the present invention;

[0031] Figure 6 Schematic diagram of the connecting rod coordinate system in the second embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of a silicon measurement system in Embodiment 1 of the present invention;

[0033] In the figure, 1. walking mechanism, 2. operating platform, 3. waist rotating mechanism, 4. upper arm pitching mechanism, 5. lower arm telescopic mechanism, 6. pitching mechanism, 7. sampling spoon clamping device. DETAILED DESCRIPTION

[0034] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0035] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0036] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0037] Embodiment 1

[0038] like Figure 1-Figure 5 As shown, this embodiment provides a molten iron visual robot, including three parts: a mechanical system, an operating platform, and a visual recognition software system. The mechanical system is composed of a multi-degree-of-freedom mechanical arm, a mobile chassis, and a sample collection mechanism; the operating platform includes a cooling module, a cutting module, a grinding module, and a silicon measurement module; the visual recognition software system includes a camera information reading module and a recognition module. Through the comprehensive application of this system, the efficiency and data reliability of molten iron sampling and silicon measurement can be significantly improved.

[0039] Specifically, the multi-degree-of-freedom robot arm is composed of a waist rotation mechanism 3, a large arm pitch mechanism 4, a small arm telescopic mechanism 5, and a pitch and swing mechanism of a clamping mechanism, and is fixedly installed on a mobile chassis. The power module includes drive components such as a servo motor; the load module is composed of an output end bearing support frame, a load motor, and a load motor support frame. In addition, the sample collection module includes a visual sensor and a sampling spoon mechanism installed at the end of the robot arm.

[0040] The round handle of the traditional sampling spoon is prone to instability during the mechanical grasping process, which may cause the molten iron to spill out and affect the accuracy of the experimental results. To solve this problem, the present embodiment improves the sampling spoon and designs two grooves at the end of the handle. This design enables the mechanical gripper to insert the grippers on both sides into the grooves during grasping, thereby significantly improving the grasping stability, preventing the occurrence of molten iron spilling, and ensuring the reliability and stability of the experimental results.

[0041] Based on the functional requirements of sampling and silicon measurement of the molten iron visual robot, the hardware design of this embodiment revolves around the mobile chassis platform and the multi-degree-of-freedom robotic arm, and adopts a modular design architecture. The power module provides power to the mobile chassis platform and the multi-degree-of-freedom robotic arm respectively to ensure good electrical isolation between the two. In addition, the mobile chassis platform is also equipped with a backup power supply to cope with the situation where the main battery is insufficiently powered. The hardware of the mobile chassis platform and the multi-degree-of-freedom robotic arm mainly includes a drive module, an information acquisition module and an actuator.

[0042] The host computer not only builds the basic motion control and visual display interface, but also receives information from various sensors in real time, and determines the corresponding position of the target iron ladle through data processing of the visual sensor and the use of target recognition and positioning algorithms. Subsequently, the motion sequence of each joint of the robot is calculated and the motion instructions are sent to the motion controller.

[0043] The motion controller controls the drive module to execute motion instructions, so that the robot arm reaches the specified position. At the same time, the sensors on board can confirm whether the movements of each joint are in place, and perform closed-loop processing on the feedback information to ensure the efficiency and accuracy of the system.

[0044] The motion control system is mainly responsible for controlling the movement of the robot arm and the movement of the mobile chassis platform. The host computer uniformly controls the robot arm, mobile chassis platform and sampling spoon, and parses and processes the data fed back by each sensor to obtain the robot's operating status information in real time. The host computer sends the control command to the motion controller through TCP Ethernet communication. After receiving the command, the motion controller outputs a voltage signal and controls the movement of the servo valve through the servo amplifier, thereby driving the motor to realize the motion control of the sampling and silicon measuring robot.

[0045] In the workpiece operation platform, the information flow between the modules (including cooling module, cutting module, grinding module, silicon measuring module and conveyor belt) is realized through the TCP / IP protocol. Each module is equipped with a network interface to transmit data to the central control system through the TCP / IP protocol. The control system receives and processes real-time data from different modules through TCP connection to coordinate the operation of each module. For example, the status information of the cutting module can be transmitted to the control system in real time, and the control system then sends instructions to the cooling module through the TCP protocol to adjust the grinding intensity. All communications are carried out through the standardized TCP / IP protocol to ensure the reliability and consistency of data transmission, thereby achieving efficient information flow and system coordination.

[0046] The workpiece operating platform is composed of modules such as a cooling module, a cutting module, a grinding module, a silicon measuring module and a conveyor belt. Multiple modules are arranged in sequence on the conveyor belt of the operating platform at intervals, and each group of modules is adapted to the structural configuration of the workpiece to be conveyed.

[0047] The cooling conveying mechanism is used in conjunction with the small water reservoir to carry and convey the iron block sample to the small water reservoir. The mobile robot drives the mechanical arm to take the iron block sample out of the small water reservoir and place it on the conveyor belt, which transports the sample to the cutting machine for cutting operation.

[0048] Subsequently, the mobile robot continues to drive the mechanical arm to take the cut iron block sample out of the cutting machine and place it on the conveyor belt, which then transports the sample to the grinder for grinding. Finally, the mobile robot drives the mechanical arm to take the polished iron block sample out of the grinder and place it on the conveyor belt, which transports the sample to the bottom of the small spectrometer for silicon measurement, and feeds back the silicon content result to the PC.

[0049] like Figure 7 As shown, the cooling conveying mechanism is equipped with a small water reservoir, which is used to carry and convey the iron block sample into the small water reservoir; the mobile robot drives the mechanical arm to take the iron block sample out of the small water reservoir and place it on the conveyor belt, which conveys the sample to the position of the cutting machine for cutting; the mobile robot drives the mechanical arm again to place the cut iron block sample on the conveyor belt, and conveys it to the position of the grinder for grinding; the iron block sample is taken out and placed on the conveyor belt, and conveyed to the bottom of the small spectrometer for silicon measurement; the measured silicon content result is fed back to the PC. The robot returns to its original position, and the measurement ends.

[0050] Embodiment 2

[0051] This embodiment provides a control method for a molten iron sampling robot, comprising:

[0052] The hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the molten iron sampling robot's mechanical arm is determined by the hand-eye calibration method;

[0053] Acquire an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the coordinate system of the end of the robot mechanical arm according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot;

[0054] According to the DH parameters, the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot is analyzed, and the total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot is established;

[0055] According to the established total transformation matrix of the robot arm and the determined position of the molten iron container, the joint angles and joint extension lengths of each joint of the robot arm are solved by using inverse kinematics operation;

[0056] The movement of the molten iron sampling robot is controlled according to the obtained joint angles and joint extension lengths of each joint.

[0057] This embodiment uses the molten iron sampling robot in the first embodiment. After the robot system is started, the high-resolution camera starts to capture the image of the molten iron surface in real time. The Zhang Zhengyou chessboard calibration method is used to calibrate the camera of the molten iron tank, and the size of each chessboard and the number of rows and columns are set to complete the camera calibration process. The qualified indicators of camera calibration include four aspects: x-axis direction, y-axis direction, distance and rotation angle.

[0058] Then, stereo calibration and distortion correction are performed to ensure that the imaging planes of the two lenses of the binocular camera are on the same plane and that the optical axes of the two cameras are parallel. The eye-to-hand calibration method is used to fix the camera in a static position. The coordinate relationship between the camera coordinate system (Camera) and the end coordinate system (End) of the robotic arm is solved through two movements.

[0059] After reading the camera information, the input image is first preprocessed, including operations such as letterBox and normalization. Then, the decoding process is performed through the FBox branch and Cls branch of the inference module. Finally, the NMS module and Scaleboxes are used for post-processing to complete the recognition of molten iron and thus determine the target object.

[0060] Convert the two-dimensional pixel coordinates (x, y) to the coordinates (X C ,Y C ,Z C ), convert the pixel angle θ into the position of the molten iron container (α1, β1, γ1). C ,Y C ,Z C ) is converted into the coordinates (X W ,Y W ,Z W ); According to the position of the molten iron container (α1, β1, γ1), the robot end effector reaches (X W ,Y W ,Z W ) when (α2, β2, γ2); using the inverse kinematics solution, through (X W ,Y W ,Z W ), (α2, β2, γ2) these 6-dimensional known data can be used to calculate the angle j that each motor of the robot arm needs to rotate; a series of intermediate points can be planned according to the environment and actual needs (such as the presence or absence of obstacles, the shortest path requirement, the shortest time requirement, etc.). Through the calculation of the inverse kinematics principle, the angle value signal of the sampling spoon reaching the middle point of the molten iron tank is obtained.

[0061] After obtaining the angle value signal, the sampling robot starts to move down to perform sampling operations. Then, each motor is controlled to drive the robot arm to rotate to the middle point above the molten iron ladle. The sampling arm swings to a predetermined position to perform sampling operations.

[0062] During the sampling process, the sampling spoon is inserted into the molten iron surface for 5 seconds before being pulled out to ensure the effectiveness and accuracy of sample collection.

[0063] Combine the following Figure 5-Figure 6 The control method of the molten iron vision robot proposed in this embodiment is described in detail:

[0064] Step 1: System boot.

[0065] Step 2: Read the camera information.

[0066] Step 3: In order to make the image appear realistically in the picture and facilitate the subsequent operation of the robotic arm, the camera needs to be calibrated for distortion.

[0067] The specific formula is as follows:

[0068]

[0069] in, is the camera internal parameter, is the camera external parameter; {u,v} is the pixel coordinate system, the origin is the upper left corner of the image; {X W ,Y W ,Z W} is the world coordinate system, describing the camera position; f is the focal length of the camera; R represents the rotation matrix; T represents the offset vector, R and T are called external parameters.

[0070] This embodiment aims to solve the problem of converting the relative position relationship between the molten iron ladle, the camera and the robotic arm, and proposes a hand-eye calibration method. After the camera determines the position of the molten iron ladle, the position needs to be converted into coordinates relative to the robotic arm to ensure the effective operation of the robotic arm.

[0071] This embodiment adopts the "eye on hand" hand-eye calibration technology to obtain the coordinate relationship between the camera coordinate system and the robot coordinate system through two movements, thereby achieving precise positioning and efficient operation.

[0072] Let G be the robot end tool hand coordinate system, B be the calibration plate coordinate system, C be the camera coordinate system, and W be the robot base coordinate system. The transformation relationship is:

[0073]

[0074] in, is the position relationship of the calibration plate relative to the robot base. When the calibration plate is stationary, To keep the quantity constant; It is the position relationship of the robot end effector relative to the robot base, which is given by the robot arm controller; is the position relationship between the camera and the robot end effector, which is the quantity required for hand-eye calibration; It is the position and posture relationship of the calibration plate relative to the camera, that is, the external parameter obtained by camera calibration.

[0075] Keep the calibration plate still and the robot moves to two different positions:

[0076]

[0077] so:

[0078]

[0079] It can be simplified to AX = XB, where:

[0080]

[0081] Among them, A is the transformation matrix corresponding to the two movements of the tool hand, and B is the transformation matrix corresponding to the two movements of the camera. Among them, A and B are both known quantities. By performing multiple transformations on the end position of the robot arm, the hand-eye calibration equation AX=XB can be solved to obtain the value of the hand-eye conversion matrix X.

[0082] A camera is used to obtain image information of the grasped object, and the position A of the object is usually identified through image processing technology (such as edge detection, feature matching, etc.).

[0083] Convert the object position A in the camera coordinate system to the target position in the robot coordinate system: B = X -1 A. Perform an inverse transform on X to convert the object position in the camera coordinate system to the position in the robot coordinate system.

[0084] According to the spatial position and posture of the end of the robot arm, the corresponding joint angle is solved.

[0085] The DH parameter method is usually used in the kinematic analysis of the robot arm. Through this method, a specified reference coordinate system is established for each joint, and the transformation relationship between any two adjacent coordinate systems is established, so as to derive the total transformation matrix of the robot arm.

[0086] The robot arm of this embodiment has five degrees of freedom, which are waist rotation, upper arm pitch, lower arm extension, sampling spoon platform, swing and pitch. Figure 6The connecting rod coordinate system shown takes the rotation center of the robot base as the base coordinate origin, and takes the joints of each degree of freedom as the origin of the joint coordinate system. The rotation axis is along the B-axis direction, and the X-axis coordinate system is established along the positive direction of the mobile chassis to achieve an accurate description of the movement of the robot arm.

[0087] The DH parameter values ​​of the robot arm designed in this embodiment are shown in Table 1, where θ i represents the joint angle, d i Indicates the connecting rod offset, a i-1 represents the connecting rod length, α i-1 represents the torsion angle of the connecting rod, σ=0 means that the joint is a revolute joint, θ i is a variable, and the other parameters are fixed values. σ=1 means that the joint is a sliding joint, d i is a variable, and the rest of the parameters are fixed values.

[0088] Table 1:

[0089]

[0090] According to the DH parameter method, the transformation relationship between two adjacent coordinate systems {i-1} and {i} can be obtained by using the matrix transformation method. The general transformation formula is as follows:

[0091]

[0092] Where cθ i = cosθ i , sθ i = sinθ i , cα i-1 =cosα i-1 , sα i-1 = sinα i-1 .

[0093] Substituting the parameters in Table 1 into formula (6), we can obtain the transformation matrices of five adjacent coordinate systems, whose formulas are as follows:

[0094]

[0095] Therefore, the total transformation matrix of the sampling spoon coordinate system at the end of the robot arm relative to the base coordinate system of the mobile chassis platform is:

[0096]

[0097] in,

[0098] r 11 =cθ5(cθ1(cθ2cθ4+sθ2sθ4))+sθ5sθ1

[0099] r12 =sθ5(-cθ1(cθ2cθ4+sθ2sθ4))+cθ5sθ1

[0100] r 13 =-sθ4cθ1cθ2+cθ1sθ2cθ4

[0101] p x =a4(cθ1cθ2+cθ2sθ4)+d5(-sθ4cθ1cθ2+cθ1sθ2cθ4)+cθ1(d3sθ2+a1)

[0102] r 21 =cθ5(sθ1(cθ2cθ4+sθ2sθ4))-sθ5cθ1

[0103] r 22 =-sθ5(sθ1(cθ2cθ4-sθ2sθ4))-cθ1cθ5

[0104] r 23 =-sθ4sθ1cθ2+sθ1sθ2cθ4

[0105] p y =a4(cθ4sθ1cθ2+sθ1sθ2sθ4)+d5(-sθ4sθ1cθ2+sθ1sθ2cθ4)+sθ1(d3sθ2+a1)

[0106] r 31 =cθ5(sθ2cθ4-cθ2sθ4)

[0107] r 32 = -sθ5(sθ2cθ4-cθ2sθ4)

[0108] r 33 =-sθ2cθ4-cθ2sθ4

[0109] p z =a4(sθ2cθ4-cθ2sθ4)-d5(sθ2sθ4+cθ2cθ4)-d3cθ2 (8)

[0110] Inverse kinematics analysis is to solve the angles of each rotating joint and the length of the sliding joint by knowing the posture information of the end sampling spoon.

[0111] The position of the molten iron container obtained by the camera is expressed as (X A ,Y A ,Z A ), and its posture is defined by the rotation angle (α1, β1, γ1). The target position (X w ,Y w ,Z w) and posture (α2, β2, γ2).

[0112] The location of the molten iron container (X A ,Y A ,Z A ) and posture (α1, β1, γ1) to construct a 4×4 transformation matrix T A , combining translation and rotation:

[0113]

[0114] Among them, R(α1,β1,γ1) is the rotation matrix generated by Euler angles, P A =[X A ,Y A ,Z A ] T is the position vector of the object.

[0115] Use the hand-eye transformation matrix X to transform the pose in the camera coordinate system to the robot arm end coordinate system: T B =X·T A , T B Represents the pose matrix in the robot end effector coordinate system.

[0116] From T B Extract the target position (X w ,Y w ,Z w ) and posture (α2, β2, γ2):

[0117]

[0118] From T B Extract the rotation part R B , and then convert it into Euler angle form to get (α2, β2, γ2).

[0119] This can achieve a complete conversion from the camera coordinate system of the molten iron container to the coordinate system of the robot end effector to obtain the required target position and posture.

[0120] This embodiment uses an analytical method to multiply the unknown joint angle by both sides of the above formula (8) to separate the angle values, thereby calculating the angle values ​​of each joint.

[0121] Solve for the joint angle θ1:

[0122]

[0123] Further we can get:

[0124]

[0125] By calculation, we can get:

[0126]

[0127] From the equality of the elements (2,4) on both sides of formula (10), that is, the corresponding elements in the second row and fourth column, we can obtain:

[0128] p y sθ1-p x cθ1=0 (12)

[0129] Therefore, the angle of θ1 can be obtained as:

[0130]

[0131] Solve for the joint angle θ5:

[0132] Similarly, from the corresponding equality of the elements (2,1) and (2,2) on both sides of formula (10), we can obtain:

[0133]

[0134] Substituting θ1 into formula (14), the angle of θ5 can be obtained as:

[0135]

[0136] Solve for the joint angle θ2:

[0137] From the equality of the elements (1,3) and (3,3) on both sides of equation (10), we can get:

[0138]

[0139] Then we can get:

[0140]

[0141] Multiply both sides of formula (10) The resulting transformation matrix is ​​as follows:

[0142]

[0143] Further we can get:

[0144]

[0145] By calculation, we can get:

[0146]

[0147] From the correspondence between the elements (1,1) and (2,3) on both sides of equation (20), we can obtain:

[0148]

[0149] Then the angle of θ2 can be obtained as:

[0150]

[0151] Solve for the joint angle θ4:

[0152] The angle of θ4 can be obtained from formulas (18) and (23):

[0153]

[0154] Solve the joint extension length d3:

[0155] From the equation (11), the elements (1,4) on both sides are equal, we can get:

[0156] α1+α4c(θ2-θ4)+d5s(θ2-θ4)+d3sθ2=p x cθ1+p y sθ1 (24)

[0157] Then the length of d3 can be obtained as:

[0158]

[0159] In this embodiment, after the end position of the sampling spoon is calibrated with the center point of the molten iron located by vision, the sampling robot starts to operate and drives the mechanical arm to perform the sampling operation. The robot obtains the angle and length of each joint through inverse kinematics calculation, thereby completing the sample extraction in the molten iron.

[0160] In this embodiment, the image acquired by the camera is preprocessed, specifically including the following operations: first, letterBox processing is performed to convert the size of the original image (720, 1280) to the network input size (640, 640); second, the color channel order of the image is converted from BGR to RGB; then, the image data format is converted from BHWC to BCHW; finally, the image pixel value is divided by 255 for normalization, because the pixel value of the image is usually between 0 and 255. By dividing by 255, the pixel value can be mapped to the range of 0 to 1, thereby improving the efficiency and accuracy of subsequent processing.

[0161] The preprocessed image is decoded using the FBox branch and Cls, and then post-processed using non-maximum suppression (NMS) and scale boxes to identify the molten iron and thus determine the target object.

[0162] Embodiment 3

[0163] The purpose of this embodiment is to provide a control system for a molten iron sampling robot, comprising:

[0164] A first determination module is configured to: determine a hand-eye conversion matrix between a camera coordinate system and an end effector coordinate system of a molten iron sampling robot by a hand-eye calibration method;

[0165] The second determination module is configured to: obtain an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the robot end effector coordinate system according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot;

[0166] A transformation matrix establishment module is configured to: analyze the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot according to the DH parameters, and establish a total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot;

[0167] A solution module is configured to: solve the joint angle and joint extension length of each joint of the robot arm by using inverse kinematics operation according to the established total transformation matrix of the robot arm and the determined position of the molten iron container;

[0168] The control module is configured to control the movement of the molten iron sampling robot according to the obtained joint angles and joint extension lengths of each joint.

[0169] In further embodiments, there is also provided:

[0170] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, no further description is given here.

[0171] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0172] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0173] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0174] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0175] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0176] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0177] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.

[0178] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0179] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0180] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A control method for a molten iron sampling robot, characterized in that: include: The hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the molten iron sampling robot's mechanical arm is determined by the hand-eye calibration method; Acquire an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the coordinate system of the end of the robot mechanical arm according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot; According to the DH parameters, the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot is analyzed, and the total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot is established; According to the established total transformation matrix of the robot arm and the determined position of the molten iron container, the joint angles and joint extension lengths of each joint of the robot arm are solved by using inverse kinematics operation; The movement of the molten iron sampling robot is controlled according to the obtained joint angles and joint extension lengths of each joint.

2. The control method of a molten iron sampling robot according to claim 1, characterized in that: The hand-eye calibration method is used to determine the hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the molten iron sampling robot, which is: Based on the posture relationship of the calibration plate relative to the camera, the posture relationship of the end of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot, and the conversion relationship between the posture relationship of the camera relative to the end of the mechanical arm of the molten iron sampling robot and the posture relationship of the calibration plate relative to the base of the molten iron sampling robot, when the end of the mechanical arm of the robot moves to different positions, the calibration plate is kept stationary, and the posture relationship of the calibration plate relative to the base of the molten iron sampling robot is kept quantitatively unchanged, and the hand-eye calibration equation is constructed; wherein, the calibration plate is fixed at the end of the mechanical arm of the molten iron sampling robot; The hand-eye calibration equation is solved to obtain a hand-eye conversion matrix between the camera coordinate system and the base coordinate system of the molten iron sampling robot.

3. The control method of a molten iron sampling robot according to claim 1, characterized in that: According to the DH parameter analysis, the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot is established, and the total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot is established, which is specifically: Determine the DH parameter values ​​of the molten iron sampling robot including the joint angles, connecting rod offset, connecting rod length and connecting rod torsion angle of each joint of the molten iron sampling robot mechanical arm; According to the DH parameter method, the transformation relationship of the adjacent joint coordinate system is obtained by matrix transformation. The determined DH parameter value of the molten iron sampling robot is substituted into the transformation relationship of the adjacent joint coordinate system to obtain the transformation matrix of each joint adjacent coordinate system: According to the transformation matrix of the adjacent coordinate system of each joint, the total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot is obtained.

4. The control method of a molten iron sampling robot according to claim 1, characterized in that: According to the established total transformation matrix of the robotic arm and the determined position of the end of the robotic arm of the molten iron sampling robot, the joint angles and joint extension lengths of each joint of the robotic arm are solved by using the inverse kinematic operation, which is as follows: According to the equation relationship between the multiplication result of the transformation matrix of the adjacent coordinate systems of each joint and the total transformation matrix of the robot; The joint angle to be solved is multiplied by both sides of the equation respectively, and the corresponding elements on both sides of the equation are equal after the multiplication, so as to obtain the solution of the joint angle to be solved.

5. The control method of a molten iron sampling robot as claimed in claim 1, characterized in that: The image of the molten iron container is obtained, and the position of the molten iron container corresponding to the image of the molten iron container in the robot arm end coordinate system is determined according to the hand-eye conversion matrix, specifically: Construct a transformation matrix based on the position and posture of the molten iron container in the camera coordinates; The hand-eye conversion matrix is ​​used to convert the position and posture in the camera coordinate system to the robot manipulator end coordinate system, thereby obtaining the position of the molten iron container in the robot manipulator end coordinate system.

6. The control method of a molten iron sampling robot according to claim 1, characterized in that: Also includes: The acquired molten iron container image is preprocessed, and the preprocessed molten iron container image is decoded using the FBox branch and the Cls branch. The molten iron container is identified and determined through non-maximum suppression and scale box processing.

7. A control system for a molten iron sampling robot, characterized in that: include: The first determination module is configured to: determine the hand-eye conversion matrix between the camera coordinate system and the end coordinate system of the mechanical arm of the molten iron sampling robot by a hand-eye calibration method; The second determination module is configured to: obtain an image of a molten iron container, and determine the position of the molten iron container corresponding to the image of the molten iron container in the coordinate system of the end of the robot mechanical arm according to the hand-eye conversion matrix; the image of the molten iron container is obtained by photographing the molten iron container by a camera installed at the end of the mechanical arm of the molten iron sampling robot; A transformation matrix establishment module is configured to: analyze the transformation relationship between the joint coordinate systems of the mechanical arm of the molten iron sampling robot according to the DH parameters, and establish a total transformation matrix of the mechanical arm of the molten iron sampling robot relative to the base of the molten iron sampling robot; A solution module is configured to: solve the joint angle and joint extension length of each joint of the robot arm by using inverse kinematics operation according to the established total transformation matrix of the robot arm and the determined position of the molten iron container; The control module is configured to control the movement of the molten iron sampling robot according to the obtained joint angles and joint extension lengths of each joint.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images