A visual servoing-based slag feeding machine screw conveying pipe trajectory correction method

Abstract

The present application relates to the field of automatic residue adding control, and particularly relates to a residue adding machine spiral conveying pipe trajectory correction method based on visual servoing, the present application collects the image of the moving area of the residue adding machine spiral conveying pipe, inputs the obtained image into a YOLOv5 target detection model, obtains the labeled image of the spiral conveying pipe, inputs the labeled image into a SURF feature matching model, obtains the accurate coordinates of the spiral conveying pipe, determines the offset error of the spiral conveying pipe, controls the movement trajectory of the spiral conveying pipe of the mechanical arm based on the offset error, the present application corrects the residue adding trajectory in real time, has small calculation amount in the correction process, has high positioning accuracy, has good real-time performance, avoids the phenomenon that the residue adding cannot be corrected in time and the residue adding is uneven when the spiral conveying pipe deviates from the planned path during the residue adding process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic slagging control, in particular to a spiral conveying pipe trajectory correction method of a slagging machine based on visual servoing. BACKGROUND

[0002] Continuous casting technology has become one of the most important technologies in slab casting technology, and has been widely used in major steel producing countries. The level of slab continuous casting process is one of the important indicators to measure the level of a country's steel industry. It is a process of heating molten steel in a crystallizer, solidifying and forming, and then hot rolling or cold working to produce various steel products. In order to solve the problem of lubrication and heat transfer between the continuous casting shell and the crystallizer, the application technology of related protective slag is valued. Protective slag is an indispensable auxiliary material that is scattered on the surface of molten steel in the crystallizer, which makes the lubrication and heat transfer of the crystallizer better, the shell thicker, and the drawing resistance smaller, thereby reducing the risk of leakage. Therefore, related protective slag feeding equipment and systems have emerged.

[0003] For example, Chinese patent publication No. CN109531542A discloses a robot slagging system, which includes a perception subsystem for real-time detection of the state of the crystallizer, the state information of the surrounding environment, and the operation information of the robot, and inputs the perceived information to the decision subsystem; the decision subsystem is used for teaching operation of the slagging trajectory of the robot; the state information of the crystallizer and the surrounding environment during the teaching process is detected in real time, and the state information and the operation information are analyzed to abstract the mapping relationship between the state information and the operation; according to the abstracted mapping relationship and the state information of the crystallizer and the surrounding environment, a control signal of the robot operation trajectory is generated; the execution subsystem is used for completing the slagging operation according to the control signal and the planned trajectory. The present application overcomes the shortcomings of the prior art and realizes the intelligentization of the crystallizer slagging system.

[0004] However, the prior art still has the following problems,

[0005] The current slagging robot is prone to deviation error when the spiral conveying pipe advances according to the planned path during the slagging process, which cannot be corrected in time and will cause uneven slagging. SUMMARY

[0006] To solve the above problems, the present application provides a spiral conveying pipe trajectory correction method of a slagging robot based on visual servoing, which includes:

[0007] Step S1, collecting the image of the moving area of the screw conveying pipe of the residue adding machine, including fixing an image collection unit on one side of the crystallizer, calibrating the image collection unit, and covering the collection field of view of the image collection unit with the moving area of the screw conveying pipe of the residue adding machine to collect the image of the moving area of the screw conveying pipe of the residue adding machine;

[0008] Step S2, determining the accurate coordinates of the screw conveying pipe, including inputting the obtained image into a YOLOv5 target detection model to obtain a labeled image of the screw conveying pipe, and inputting the labeled image into a SURF feature matching model to obtain the accurate coordinates of the screw conveying pipe;

[0009] Step S3, determining the offset error of the screw conveying pipe, including comparing the accurate coordinates with preset standard trajectory coordinates, and determining the offset error of the screw conveying pipe according to the comparison result;

[0010] Step S4, correcting the moving trajectory of the screw conveying pipe, and based on the offset error, controlling the mechanical arm to correct the moving trajectory of the screw conveying pipe.

[0011] Further, the step S1 further includes adjusting the running parameters of the image collection unit, wherein,

[0012] The saturation, definition, brightness and size of the obtained image of the image collection unit are adjusted to meet the data requirements of the target detection model.

[0013] Further, in the step S1, the calibration of the image collection unit adopts a traditional camera calibration method, wherein a coordinate system is established with the center of the crystallizer as the origin, the coordinates of the screw conveying pipe are labeled in the coordinate system, a plane type marker is made by using a plurality of images, and the correspondence between the known coordinate points on the plane type calibration object and the points in the image is established.

[0014] Further, in the step S2, the architecture of the YOLOv5 target detection model includes a CSP bottleneck layer, a spatial pyramid layer, a merging array layer, a bottleneck layer, an up-sampling layer and a convolution layer.

[0015] Further, in the step S2, the YOLOv5 target detection model is obtained by pre-training, and the training process includes: step S21, pre-shooting a plurality of working images of the screw conveying pipe, step S22, pre-processing part of the working images, including adding labels to the screw conveying pipe in each working image to obtain a training data set, and step S23, inputting the training data set into the target detection model to train the target detection model, and when the detection accuracy of the target detection model reaches a preset threshold, the target detection model is fixed, and the training is completed.

[0016] Further, in the step S3, the accurate coordinates are compared with preset standard track coordinates, and the deviation error of the spiral conveying pipe is determined according to the comparison result, wherein,

[0017] The deviation error is the deviation amount and the deviation direction of each accurate coordinate. When the deviation error is determined, a vector is constructed with the standard track coordinates as the starting point and the accurate coordinates as the ending point, the length and the direction of the vector are determined, the length is determined as the deviation amount, and the direction of the vector is determined as the deviation direction.

[0018] Further, the step S3 further includes setting a data recording unit to record the deviation error.

[0019] Further, in the step S4, the movement track of the spiral conveying pipe is corrected by the mechanical arm based on the deviation error, wherein,

[0020] The movement parameters of the mechanical arm are determined, the movement parameters include each movement coordinate of the end node of the mechanical arm, each movement coordinate is corrected to obtain a corrected coordinate, and the mechanical arm is controlled to operate with the corrected coordinate, and the corrected coordinate is obtained by moving the original movement coordinate in the opposite direction of the vector of the corresponding accurate coordinate by a corresponding deviation amount.

[0021] Further, the step S4 further includes determining whether to issue a warning information based on the deviation error, wherein,

[0022] The deviation amount in the deviation error is compared with a preset deviation amount comparison threshold,

[0023] Under the preset comparison condition, it is determined that the warning information needs to be issued to prompt the mechanical arm failure;

[0024] The preset comparison condition is that the deviation amount is greater than the preset deviation amount comparison threshold.

[0025] The application also provides a slagging machine spiral conveying pipe track correction method based on visual servoing.

[0026] The conveying device is used to output the protective slag and includes a storage bin and a spiral conveying pipe connected with the storage bin, so that the spiral conveying pipe outputs the protective slag in the storage bin;

[0027] The mechanical arm is arranged on one side of the conveying device and is used to clamp the spiral conveying pipe to move on the surface of the crystallization pool to lay the protective slag.

[0028] Compared with the prior art, by collecting the image of the moving area of the screw conveying pipe of the residue adding machine, inputting the obtained image into the YOLOv5 target detection model, obtaining the labeled image of the screw conveying pipe, inputting the labeled image into the SURF feature matching model, obtaining the accurate coordinates of the screw conveying pipe, determining the offset error of the screw conveying pipe, and based on the offset error, the movement trajectory of the screw conveying pipe is corrected by the mechanical arm, the residue adding trajectory is corrected in real time, the calculation amount in the correction process is small, the positioning accuracy is high, the real-time performance is good, and when the screw conveying pipe deviates from the planned path during the residue adding process, the residue adding phenomenon of the screw conveying pipe cannot be corrected in time.

[0029] Especially, in step S1, the image acquisition unit is fixed on one side of the crystallizer, in actual situations, because the temperature of the continuous casting machine is very high and a lot of gas and dust will be generated, so the image acquisition unit cannot be directly fixed above the screw conveying pipe, therefore, the image acquisition unit is fixed on one side of the crystallizer, which can not only ensure the effect of machine vision, but also avoid many interference factors.

[0030] Especially, in step S2, the obtained image is first input into the YOLOv5 target detection model to obtain the labeled image of the screw conveying pipe, and then the labeled image is input into the SURF feature matching model to obtain the accurate coordinates of the screw conveying pipe, through the above-mentioned manner, the accuracy and recognition precision of the screw conveying feature recognition in the image can be improved.

[0031] Especially, in step S4, the servo motor is controlled to correct the trajectory of the screw conveying pipe, which has good real-time performance, and can avoid the phenomenon that the screw conveying pipe cannot be corrected in time when it deviates from the planned path during the residue adding process. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The figure is a step diagram of the residue adding machine screw conveying pipe trajectory correction method based on visual servoing of the embodiment of the application;

[0033] Figure 2 The figure is a schematic diagram of the residue adding equipment of the embodiment of the application;

[0034] Figure 3 The figure is a schematic diagram of the residue adding trajectory of the screw conveying pipe of the embodiment of the application;

[0035] In the figure: 1: crystallizer, 2: screw conveying pipe, 21: discharge port, 3: mechanical arm, 4: residue adding movement trajectory. DETAILED DESCRIPTION

[0036] In order to make the objects, technical schemes and advantages of the present application clearer, the following further describes the present application with reference to the embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0037] The preferred embodiments of the present application are described below with reference to the drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and not to limit the protection scope of the present application.

[0038] It should be noted that in the description of the present application, the terms "upper", "lower", "left", "right", "inner", "outer" and the like indicate the direction or positional relationship terms based on the direction or positional relationship shown in the drawings, which are only for the convenience of description and do not indicate or imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0039] In addition, it should also be noted that in the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. Those skilled in the art can understand the specific meaning of the above terms in the present application according to the specific circumstances.

[0040] Please refer to Figure 1 and Figure 2 shown, Figure 1 a trajectory correction method for a screw conveying pipe of a residue adding machine based on visual servoing according to an embodiment of the present application, Figure 2 a schematic diagram of a residue adding device according to an embodiment of the present application, the trajectory correction method for a screw conveying pipe of a residue adding machine based on visual servoing according to the present application comprises:

[0041] Step S1, collecting images of a residue adding moving track 4 area of a screw conveying pipe 2 of a residue adding machine, including fixing an image collection unit on one side of a crystallizer 1, calibrating the image collection unit, and making the collection field of view of the image collection unit cover the residue adding moving track 4 area of the screw conveying pipe 2 of the residue adding machine, so as to collect images of the residue adding moving track 4 area of the screw conveying pipe 2 of the residue adding machine;

[0042] Step S2, determining the accurate coordinates of a discharge port 21 of the screw conveying pipe 2, including inputting the obtained images into a YOLOv5 target detection model to obtain labeled images of the screw conveying pipe 2, and inputting the labeled images into a SURF feature matching model to obtain the accurate coordinates of the screw conveying pipe 2;

[0043] Step S3, determining the offset error of the spiral conveying pipe 2, including comparing the accurate coordinates with the preset standard slag adding moving track 4 coordinates, and determining the offset error of the spiral conveying pipe 2 according to the comparison result;

[0044] Step S4, correcting the slag adding moving track 4 of the spiral conveying pipe 2, and correcting the slag adding moving track 4 of the spiral conveying pipe 2 based on the offset error by controlling the mechanical arm 3.

[0045] Specifically, the specific structure of the image acquisition unit is not limited in the present application, which can be an industrial CCD camera, a photographic device, etc., a device capable of acquiring images. Of course, preferably, the image acquisition unit can include an industrial CCD camera and an information sending module to send the images acquired by the industrial CCD camera to the processing end for analysis and processing. The setting mode of the image acquisition unit is also not limited in the present application, which can be set on one side of the crystallizer through a fixing frame, a slide rail or the like, as long as it can complete shooting.

[0046] Specifically, when the image or data needs to be processed in steps S2, S3 and S4, a logic component can be used to complete the processing. The logic component includes a field programmable processor, a computer and a microprocessor in the computer, etc.

[0047] Specifically, in step S1, the calibration process of the image acquisition unit includes establishing a coordinate system with the position of the crystallizer 1 as a reference, marking the coordinates of the spiral conveying pipe 2 in the coordinate system, and establishing the correspondence between the points with known coordinates on the calibration object and their image points by using a plurality of images to make a planar marker.

[0048] Specifically, in step S1, the adjustment of the operating parameters of the image acquisition unit includes adjusting the saturation, clarity, brightness and size of the images acquired by the image acquisition unit to meet the data requirements of the target detection model.

[0049] Specifically, in step S1, the industrial CMOS camera shoots the working images of the spiral conveying pipe 2 advancing along the planned path according to the preset shooting period. The working images shot by the industrial camera should clearly show the working environment around the spiral conveying pipe 2, so as to find the analysis objects such as the spiral conveying pipe 2 and the crystallizer 1 from the working images.

[0050] Specifically, in step S1, the image acquisition unit is fixed on one side of the crystallizer 1. In actual situations, since the temperature of the continuous casting machine is very high and a lot of gas and dust will be generated, the image acquisition unit cannot be directly fixed above the spiral conveying pipe 2. Therefore, the image acquisition unit is fixed on one side of the crystallizer 1, which can not only ensure the effect of machine vision, but also avoid many interference factors.

[0051] Specifically, in step S2, the architecture of the YOLOv5 target detection model includes a CSP bottleneck layer (BottleNeckCSP), a spatial pyramid layer (SPP), a merging array layer (concat), a bottleneck layer (CSP), an upsampling layer (upsample), and a convolution layer (conv).

[0052] Specifically, in step S2, the YOLOv5 target detection model is pre-trained, and the training process includes: step S21, pre-shooting working images of the screw conveying pipe 2, step S22, extracting part of the working images for preprocessing, including adding labels to the screw conveying pipe 2 in each working image, obtaining a training data set, step S23, inputting the training data set into the target detection model to train the target detection model, and when the detection accuracy of the target detection model reaches a preset threshold, fixing the target detection model, completing the training. First, input the obtained image into the YOLOv5 target detection model to obtain the labeled image of the screw conveying pipe 2, and then input the labeled image into the SURF feature matching model to obtain the accurate coordinates of the screw conveying pipe 2. The positioning accuracy is high, and the labeled image is an image labeled by the YOLOv5 target detection model.

[0053] Specifically, in step S3, the accurate coordinates are compared with the preset standard trajectory coordinates, and the offset error of the screw conveying pipe 2 is determined according to the comparison result. The offset error is the offset amount and the offset direction of each accurate coordinate. When determining the offset error, the standard trajectory coordinates are taken as the starting point and the accurate coordinates are taken as the ending point to construct a vector, and the length and direction of the vector are determined. The length is determined as the offset amount, and the direction of the vector is determined as the offset direction.

[0054] Specifically, in step S3, the determination of the offset error of the screw conveying pipe 2 includes setting a data recording unit to record the offset error.

[0055] Specifically, in step S4, the correction of the screw conveying pipe 2 includes determining the movement parameters of the mechanical arm 3, including the movement coordinates of the end node of the mechanical arm 3, correcting each movement coordinate to obtain a corrected coordinate, and controlling the mechanical arm 3 to operate with the corrected coordinate. The corrected coordinate is obtained by moving the original movement coordinate in the opposite direction of the corresponding vector of the corresponding accurate coordinate by a corresponding offset amount.

[0056] Specifically, in step S4, the trajectory of the screw conveying pipe 2 is corrected by controlling the servo motor, which has good real-time performance and avoids the phenomenon of uneven addition of slag when the screw conveying pipe 2 deviates from the planned path during the addition of slag.

[0057] Specifically, in step S4, the correction of the residue feeding movement track 4 of the screw feeding pipe 2 includes determining whether to issue a warning information based on the offset error, comparing the offset amount in the offset error with a preset offset amount comparison threshold, and determining to issue a warning information to prompt the mechanical arm 3 to fail under a preset comparison condition. The preset comparison condition is that the offset amount is greater than the preset offset amount comparison threshold.

[0058] The application also provides a device for the residue feeding machine screw feeding pipe track correction method based on visual servo, which comprises:

[0059] The feeding device is used to output the protective slag and comprises a storage bin and a screw feeding pipe 2 connected with the storage bin, so that the screw feeding pipe 2 outputs the protective slag in the storage bin;

[0060] The mechanical arm 3 is arranged on one side of the feeding device and is used to clamp the screw feeding pipe 2 to move on the surface of the crystallization pool to lay the protective slag.

[0061] Specifically, the specific structure of the mechanical arm is not limited, the degrees of freedom and joint settings of the mechanical arm can be adjusted by the person skilled in the art according to the specific application scene, which is the prior art and will not be repeated.

[0062] So far, the technical solutions of the application have been described in combination with the preferred embodiments shown in the drawings, but the person skilled in the art can easily understand that the protection scope of the application is obviously not limited to these specific embodiments. The person skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the application, and the technical solutions after the changes or replacements will fall within the protection scope of the application.

Claims

1. A visual servoing-based trajectory correction method for a screw conveyor of a slag feeder, characterized in that, The method comprises the following steps: Step S1, acquiring an image of the moving area of the screw conveying pipe of the slag adding machine, comprising fixing an image acquisition unit on one side of the crystallizer, calibrating the image acquisition unit, and covering the collection field of view of the image acquisition unit with the moving area of the screw conveying pipe of the slag adding machine to acquire the image of the moving area of the screw conveying pipe of the slag adding machine; Step S2, determining the accurate coordinates of the screw conveying pipe, comprising inputting the acquired image into a YOLOv5 target detection model to obtain a labeled image of the screw conveying pipe, and inputting the labeled image into a SURF feature matching model to obtain the accurate coordinates of the screw conveying pipe; Step S3, determining the offset error of the screw conveying pipe, comprising comparing the accurate coordinates with the preset standard trajectory coordinates, and determining the offset error of the screw conveying pipe according to the comparison result; Step S4, correcting the moving trajectory of the screw conveying pipe, comprising controlling the mechanical arm to correct the moving trajectory of the screw conveying pipe based on the offset error.

2. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, In the step S1, the running parameters of the image acquisition unit are also adjusted, wherein, the saturation, definition, brightness and size of the acquired image of the image acquisition unit are adjusted to meet the data requirements of the target detection model.

3. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, In the step S1, the calibration of the image acquisition unit adopts a traditional camera calibration method, wherein a coordinate system is established with the center of the crystallizer as the origin, the coordinates of the screw conveying pipe are labeled in the coordinate system, a plane type marker is made by using a plurality of images, and the correspondence between the known coordinate points on the plane type calibration object and the points in the image is established.

4. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, In the step S2, the architecture of the YOLOv5 target detection model comprises a CSP bottleneck layer, a spatial pyramid layer, a merging array layer, a bottleneck layer, an up-sampling layer and a convolution layer.

5. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, In step S2, the YOLOv5 target detection model is obtained by pre-training, and the training process comprises the following steps: step S21, pre-shooting a plurality of working images of the screw conveying pipe, step S22, pre-processing part of the working images, comprising adding labels to the screw conveying pipe in each working image to obtain a training data set, and step S23, inputting the training data set into the target detection model to train the target detection model, and when the detection accuracy of the target detection model reaches a preset threshold, the target detection model is fixed to complete the training.

6. The visual servoing based reclaimer screw conveyor trajectory correction method of claim 1, wherein, In the step S3, the accurate coordinates are compared with the preset standard trajectory coordinates, and the offset error of the screw conveying pipe is determined according to the comparison result, wherein, the offset error is the offset amount and the offset direction of each accurate coordinate, a vector is constructed with the standard trajectory coordinates as the starting point and the accurate coordinates as the ending point when the offset error is determined, the length and direction of the vector are determined, the length is determined as the offset amount, and the direction of the vector is determined as the offset direction.

7. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, In the step S3, a data recording unit is also set to record the offset error.

8. The visual servoing based reclaimer screw conveyor trajectory correction method of claim 1, wherein, In the step S4, the moving trajectory of the screw conveying pipe is corrected by the mechanical arm based on the offset error, wherein, The movement parameters of the mechanical arm are determined, the movement parameters including movement coordinates of the end node of the mechanical arm, the movement coordinates are corrected to obtain corrected coordinates, and the mechanical arm is controlled to operate with the corrected coordinates, wherein the corrected coordinates are obtained by moving the original movement coordinates in the opposite direction of the vector from the original movement coordinates to the corresponding accurate coordinates by a corresponding offset.

9. The visual servoing based trajectory correction method for screw conveyor of a residue charging machine according to claim 1, wherein, The step S4 further includes determining whether to issue a warning information based on the offset error, wherein, The offset in the offset error is compared with a preset offset comparison threshold value, Under a preset comparison condition, it is determined that the warning information needs to be issued to prompt the mechanical arm failure; The preset comparison condition is that the offset is greater than the preset offset comparison threshold value.

10. A slagging device applying the method according to any one of claims 1 to 9, characterized in that Comprise: A material conveying device is used to output the protective slag, which comprises a storage bin and a spiral conveying pipe connected with the storage bin, so that the spiral conveying pipe outputs the protective slag in the storage bin; A mechanical arm is arranged on one side of the material conveying device to clamp the spiral conveying pipe and move on the surface of the crystallization pool to lay the protective slag.

Images