Steel coil recognition and grasping method and system based on machine vision

Abstract

The application provides a steel coil recognition and grabbing method and system based on machine vision, and comprises the following steps: obtaining a swing period of a crane clamp; determining a time when the crane clamp swings to a highest point; obtaining a closing time of the crane clamp mechanism according to the time when the crane clamp swings to the highest point and the swing period of the crane clamp, and controlling the crane clamp to close to complete the coil clamping when the crane clamp mechanism closes. Through the machine vision algorithm, the movement state of the clamp is recognized and judged, so that the coil clamping action in the swing state of the clamp is realized, the time for waiting for the angle reduction is greatly shortened, and the overall production efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision recognition, and more specifically, to a machine vision-based method and system for recognizing and grasping steel coils. Background Technology

[0002] In the field of metallurgical automation, the operating efficiency of unmanned overhead cranes has a significant impact on the overall production rhythm. When hoisting steel coils, the clamps often swing back and forth near the coil, failing to grasp it immediately. The existing solution is to wait for the swing amplitude of the clamps to decrease before grasping it. However, when the swing angle is too large, the waiting time also increases, which greatly affects production efficiency.

[0003] To achieve steel coil gripping even when the clamp is wobbling, it is necessary to accurately identify the clamp's movement state. To achieve this, technologies such as angle sensors, laser scanning, and image recognition can be used.

[0004] For angle sensors, the unevenness of the train's running track results in varying angle data collected at different locations, significantly impacting accurate angle extraction. Furthermore, the angle values ​​are affected by the clamp's height; therefore, relying solely on angle sensors makes it difficult to obtain accurate clamp movement information.

[0005] For laser scanning, the position change of the gripper needs to be determined in a very short time during the grasping process, and the data processing volume of the laser point cloud is large, which affects the real-time nature of the status information. Therefore, laser scanning is not suitable for determining the movement status of the gripper.

[0006] Patent document CNCN106839985A discloses an automatic identification and positioning method for steel coil grasping by an unmanned overhead crane, including fixing the position of a frame vehicle carrying a steel coil; acquiring scene point cloud data of the frame vehicle carrying the steel coil using a laser scanner; separating steel coil data from the scene point cloud data; treating the steel coil as a cylinder and performing parameter fitting to obtain the physical coordinates of the steel coil; sending the physical coordinates of the steel coil to the unmanned overhead crane, which then grasps the steel coil based on the physical coordinates.

[0007] However, the technical solution in patent document CNCN106839985A uses a laser scanner to identify the specific location of the steel coil and grasp it, but it does not consider the grasping problem when the clamp is shaking, which has certain limitations. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method and system for steel coil identification and grasping based on machine vision.

[0009] A machine vision-based steel coil identification and grasping method according to the present invention includes:

[0010] Step S1: Obtain the swing period of the crane caliper;

[0011] Step S2: Determine when the crane caliper swings to its highest point;

[0012] Step S3: Based on the timing of the crane clamp swinging to its highest point and the swing period of the crane clamp, obtain the closing timing of the crane clamp mechanism, and control the crane clamp to close and complete the clamping when the crane clamp mechanism closes.

[0013] Preferably, in step S1, when the crane clamp swings, the crane clamp is regarded as a simple pendulum motion, and the swing period of the crane clamp is calculated according to the formula of the period of simple pendulum motion and the current height of the crane clamp.

[0014] Preferably, in step S2, the timing of the gantry clamp swinging to the highest point can be determined based on the gantry motion state obtained through previous image processing, at which point the gantry clamp speed is 0.

[0015] The vehicle motion state includes the motion state of the steel coil; the image processing to obtain the vehicle motion state includes: extracting the outline of the steel coil and mapping it to the world coordinate system to obtain the position of the steel coil in the real world; after obtaining the position of the steel coil in the real world, the motion state of the vehicle clamp relative to the steel coil is obtained by processing and comparing the continuously captured images.

[0016] Preferably, in step S3, when the crane clamp swings to its highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from its highest point to the center, i.e., half of the swing period. When the crane clamp mechanism moves to the calculated closing timing, the clamp is controlled to close, thereby completing the winding.

[0017] The opening and closing of the crane clamp mechanism and its opening and closing speed are controlled by action commands.

[0018] Preferably, the method further includes the following steps:

[0019] Image acquisition steps: The industrial camera is instructed to start acquiring images of the crane, crane clamp, and steel coil. The frequency and resolution of image acquisition are specified by the crane controller. When the crane starts working, the industrial camera takes pictures at fixed time intervals for image processing to obtain the crane's motion status.

[0020] A steel coil identification and grasping system based on machine vision according to the present invention includes:

[0021] Module M1: Obtains the swing period of the crane clamp;

[0022] Module M2: Determines when the crane clamp swings to its highest point;

[0023] Module M3: Based on the timing of the crane clamp swinging to its highest point and the swing period of the crane clamp, the closing timing of the crane clamp mechanism is obtained, and the crane clamp is controlled to close and complete the clamping when the crane clamp mechanism closes.

[0024] Preferably, in module M1, when the crane clamp swings, the crane clamp is regarded as a simple pendulum motion, and the swing period of the crane clamp is calculated according to the formula of the period of simple pendulum motion and the current height of the crane clamp.

[0025] Preferably, in module M2, the timing of the crane clamp swinging to the highest point can be determined based on the crane motion state obtained through previous image processing, at which point the crane clamp speed is 0.

[0026] The vehicle motion state includes the motion state of the steel coil; the image processing to obtain the vehicle motion state includes: extracting the outline of the steel coil and mapping it to the world coordinate system to obtain the position of the steel coil in the real world; after obtaining the position of the steel coil in the real world, the motion state of the vehicle clamp relative to the steel coil is obtained by processing and comparing the continuously captured images.

[0027] Preferably, in module M3, when the crane clamp swings to its highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from its highest point to the center, i.e., half of the swing period. When the crane clamp mechanism reaches the calculated closing timing, the clamp is controlled to close, thereby completing the winding.

[0028] The opening and closing of the crane clamp mechanism and its opening and closing speed are controlled by action commands.

[0029] Preferably, the method further includes the following steps:

[0030] Image acquisition steps: The industrial camera is instructed to start acquiring images of the crane, crane clamp, and steel coil. The frequency and resolution of image acquisition are specified by the crane controller. When the crane starts working, the industrial camera takes pictures at fixed time intervals for image processing to obtain the crane's motion status.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. This invention uses machine vision algorithms to identify and judge the movement state of the clamps, thereby realizing the clamping and rolling action when the clamps are wobbling, which greatly shortens the waiting time for the angle to decrease and improves the overall production efficiency.

[0033] 2. This invention solves the technical problem that relying solely on angle sensors cannot accurately obtain the motion state of the clamp by using machine vision recognition to extract the outline of the steel coil and calculate the motion state of the clamp.

[0034] 3. This invention uses a control algorithm combined with clamp motion state information to accurately calculate the clamp closing timing, thus solving the problem of having to wait too long to grip the roll when it is swaying. Attached Figure Description

[0035] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0036] Figure 1 This is a flowchart for visual steel coil recognition and grasping.

[0037] Figure 2 This is a schematic diagram of the components of a visual steel coil recognition and grasping system.

[0038] Figure 3 This is a schematic diagram of a visual steel coil recognition and grasping system.

[0039] Figure 4 This is another schematic diagram of a visual steel coil recognition and grasping system.

[0040] The diagram shows:

[0041] Crane clamp 1

[0042] Industrial Camera 2

[0043] Steel coil 3 Detailed Implementation

[0044] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0045] A steel coil identification and grasping system based on machine vision according to the present invention includes:

[0046] Image acquisition module: This includes an industrial camera and a communication connection module between the industrial camera and the crane controller. Upon receiving instructions from the crane controller, the industrial camera begins acquiring images. The acquisition frequency and resolution are specified by the crane controller. At startup, the industrial camera takes pictures at fixed time intervals. After each successful image acquisition, the image data is sent to the crane controller via the communication connection.

[0047] The gripping control module includes the crane clamping mechanism and a communication connection module between the clamping mechanism and the crane controller. The crane controller sends action commands to the clamping mechanism to control its opening and closing, as well as the opening and closing speed. After receiving the command from the crane controller, the clamping mechanism opens and closes at a preset speed and sends the actual opening and closing speed and the opening and closing completion signal to the crane controller.

[0048] The crane control module, also known as the crane controller, is responsible for issuing corresponding instructions to the image acquisition module and the gripping control module. After the system starts, the crane controller calls the image acquisition module to acquire images. After acquiring the images, it uses machine vision algorithms to extract the outline of the steel coil and map it to the world coordinate system, thereby obtaining the position of the steel coil in the real world. After obtaining the position information of the steel coil, the motion state of the clamp relative to the steel coil can be obtained by processing and comparing the continuously captured images. After obtaining the motion state, the control algorithm is implemented according to the following specific steps: (1) When the crane clamp swings, it can be regarded as a simple pendulum motion. The swing period of the crane clamp can be calculated based on the simple pendulum motion period formula and the current height of the crane clamp; (2) Based on the crane motion state obtained through previous image processing, the timing of the crane clamp swinging to the highest point can be determined. At the highest point, the speed of the crane clamp is 0; (3) When the crane clamp swings to the highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from the highest point to the center, which is half of the swing period. When the crane clamp mechanism moves to the calculated closing timing, the clamp is controlled to close, thereby completing the winding.

[0049] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0050] The present invention also provides a machine vision-based method for recognizing and grasping steel coils, comprising:

[0051] Step S1: Obtain the swing period of the crane clamp; specifically, in step S1, when the crane clamp swings, the crane clamp is regarded as a simple pendulum motion, and the swing period of the crane clamp is calculated according to the simple pendulum motion period formula and the current height of the crane clamp.

[0052] Step S2: Determine the timing when the crane clamp swings to its highest point; specifically, in step S2, based on the crane motion state obtained through previous image processing, the timing when the crane clamp swings to its highest point can be determined. At the highest point, the crane clamp speed is 0; wherein, the crane motion state includes the motion state of the steel coil; obtaining the crane motion state through image processing includes: extracting the outline of the steel coil and mapping it to the world coordinate system, thereby obtaining the position of the steel coil in the real world; after obtaining the position of the steel coil in the real world, the motion state of the crane clamp relative to the steel coil is obtained by processing and comparing the continuously captured images.

[0053] Step S3: Based on the timing of the crane clamp swinging to its highest point and the swing period of the crane clamp, the closing timing of the crane clamp mechanism is obtained, and the crane clamp is controlled to close to complete the clamping and winding when the crane clamp mechanism closes. Specifically, in step S3, when the crane clamp swings to its highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from the highest point to the center, which is half of the swing period. When the crane clamp mechanism moves to the calculated closing timing, the clamp is controlled to close, thereby completing the clamping and winding; wherein, the opening and closing of the crane clamp mechanism and the opening and closing speed are controlled by action commands.

[0054] Image acquisition steps: The industrial camera is instructed to start acquiring images of the crane, crane clamp, and steel coil. The frequency and resolution of image acquisition are specified by the crane controller. When the crane starts working, the industrial camera takes pictures at fixed time intervals for image processing to obtain the crane's motion status.

[0055] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for recognizing and grasping steel coils based on machine vision, characterized in that, include: Step S1: Obtain the swing cycle of the crane caliper; Step S2: Determine when the crane caliper swings to its highest point; Step S3: Based on the timing of the crane clamp swinging to the highest point and the swing period of the crane clamp, obtain the closing timing of the crane clamp mechanism, and control the crane clamp to close and complete the clamping when the crane clamp mechanism closes; In step S2, based on the vehicle motion state obtained through previous image processing, the timing of the vehicle caliper swinging to the highest point can be determined. At the highest point, the vehicle caliper speed is 0. The vehicle motion state includes the motion state of the steel coil; the image processing to obtain the vehicle motion state includes: extracting the outline of the steel coil and mapping it to the world coordinate system, thereby obtaining the position of the steel coil in the real world; after obtaining the position of the steel coil in the real world, the motion state of the vehicle clamp relative to the steel coil is obtained by processing and comparing the continuously captured images. In step S3, when the crane clamp swings to the highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from the highest point to the center, which is half of the swing period; when the crane clamp mechanism moves to the calculated closing timing, the clamp is controlled to close, thereby completing the clamping and winding. Among them, the opening and closing of the crane clamp mechanism and the opening and closing speed are controlled by action commands; It also includes the following steps: Image acquisition steps: The industrial camera is instructed to start acquiring images of the crane, crane clamp, and steel coil. The frequency and resolution of image acquisition are specified by the crane controller. When the crane starts working, the industrial camera takes pictures at fixed time intervals for image processing to obtain the crane's motion status.

2. The steel coil identification and grasping method based on machine vision according to claim 1, characterized in that, In step S1, when the crane clamp swings, it is regarded as a simple pendulum motion. The swing period of the crane clamp is calculated according to the formula of the period of simple pendulum motion and the current height of the crane clamp.

3. A system for implementing the machine vision-based steel coil identification and grasping method of claim 1, characterized in that, include: Module M1: Obtains the swing period of the crane clamp; Module M2: Determines when the crane clamp swings to its highest point; Module M3: Based on the timing of the crane clamp swinging to the highest point and the swing period of the crane clamp, the closing timing of the crane clamp mechanism is obtained, and the crane clamp is controlled to close to complete the clamping when the crane clamp mechanism closes; In module M2, the timing of the gantry clamp swinging to its highest point can be determined based on the gantry motion state obtained through previous image processing. At the highest point, the gantry clamp speed is 0. The vehicle motion state includes the motion state of the steel coil; the image processing to obtain the vehicle motion state includes: extracting the outline of the steel coil and mapping it to the world coordinate system, thereby obtaining the position of the steel coil in the real world; after obtaining the position of the steel coil in the real world, the motion state of the vehicle clamp relative to the steel coil is obtained by processing and comparing the continuously captured images. In module M3, when the crane clamp swings to the highest point, the closing timing of the crane clamp mechanism can be calculated based on the timing required for the crane clamp to close and the time it takes for the crane clamp to swing from the highest point to the center, which is half of the swing period. When the crane clamp mechanism moves to the calculated closing timing, the clamp is controlled to close, thereby completing the clamping. The opening and closing of the crane clamp mechanism and its opening and closing speed are controlled by action commands.

4. The machine vision-based steel coil identification and grasping system according to claim 3, characterized in that, In module M1, when the crane clamp swings, it is regarded as a simple pendulum motion. The swing period of the crane clamp is calculated according to the formula of the period of simple pendulum motion and the current height of the crane clamp.

5. The machine vision-based steel coil identification and grasping system according to claim 3, characterized in that, It also includes the following steps: Image acquisition steps: The industrial camera is instructed to start acquiring images of the crane, crane clamp, and steel coil. The frequency and resolution of image acquisition are specified by the crane controller. When the crane starts working, the industrial camera takes pictures at fixed time intervals for image processing to obtain the crane's motion status.

Images