An automated steel coil forklift control method based on image recognition

Abstract

The present invention discloses an automatic steel coil forklift control method based on image recognition, which comprises the following steps: acquiring a steel coil surface image, removing rust and oil pollution interference by adopting an image enhancement algorithm, and obtaining a clear steel coil surface image; extracting edge feature points from the clear steel coil surface image, detecting the steel coil contour by adopting a Hough transform algorithm, and determining the position of the steel coil axis centerline; calculating the angle between the steel coil axis centerline and a horizontal plane according to the position of the steel coil axis centerline, and obtaining a multi-dimensional inclination angle component; adopting a three-dimensional reconstruction algorithm, mapping the steel coil axis centerline position and the inclination angle component to a three-dimensional space coordinate system, and obtaining the actual space position of the steel coil; calculating the adjustment amount of the forklift travel direction and height according to the actual space position of the steel coil and in combination with a preset forklift path, and generating a dynamic path planning result.

Description

Technical Field

[0001] The present invention relates to the field of image recognition control, and in particular to an automatic steel coil forklift control method based on image recognition. Background Art

[0002] In the steel coil handling operation, the forklift needs to accurately adjust the moving path and the steel coil fork barrel angle according to the spatial position and posture of the steel coil to ensure smooth insertion into the steel coil axis. However, in actual scenarios, the steel coil may be irregularly placed or partially tilted, resulting in the surface normal direction not matching the standard path of the forklift. At this time, the traditional method relying on manual experience or simple sensors is difficult to accurately identify the tilt angle and spatial position of the steel coil, which in turn affects the forklift's path planning and steel coil fork barrel adjustment.

[0003] Specifically, when the steel coil is tilted, the angle between its axis and the horizontal plane will change, and the forklift needs to dynamically adjust its height and the angle of the steel coil fork according to this angle to maintain the best alignment with the axis of the steel coil. However, since the inclination angle of the steel coil may have components in different directions (such as front-to-back inclination, left-to-right inclination, or compound inclination), the forklift needs to consider the influence of multi-dimensional inclination angles at the same time. In addition, there may be rust, oil stains, or uneven lighting on the surface of the steel coil. These factors will interfere with the image recognition system's accurate capture of the normal direction of the steel coil, further increasing the complexity of the forklift's path planning.

[0004] In addition, during the movement of the forklift, it is necessary to dynamically adjust the route according to the real-time position of the steel coil to avoid collisions with other equipment or obstacles. However, due to the deviation between the actual position of the steel coil and the initial detection position caused by the tilt of the steel coil, the forklift may need to recalculate the path or even temporarily stop moving for recalibration in some cases. This dynamic adjustment process not only increases the computing burden of the system, but may also prolong the operation time and affect the overall efficiency.

[0005] Therefore, how to accurately capture the multi-dimensional inclination angle of the steel coil through image recognition technology and match it with the moving path of the forklift and the angle of the steel coil fork in real time has become a key technical issue in the steel coil handling operation. Summary of the invention

[0006] The purpose of the present invention is to solve the above-mentioned problems and provide an automatic steel coil forklift control method based on image recognition.

[0007] The technical solution of the present invention is achieved in this way:

[0008] The present invention provides an automated steel coil forklift control method based on image recognition, the method relies on a steel coil transport assembly equipped with a camera; the transport assembly includes a camera assembly installed on a door frame; the camera assembly is used to obtain a steel coil surface image; a positioning system is also installed on the forklift, and the control method includes: a processor obtains a steel coil surface image, uses an image enhancement algorithm to remove rust and oil pollution interference, and obtains a clear steel coil surface image; extracts edge feature points from the clear steel coil surface image, uses a Hough transform algorithm to detect the steel coil contour, and determines the position of the steel coil axis centerline; calculates the angle between the steel coil axis centerline and the horizontal plane according to the position of the steel coil axis centerline, and obtains a multi-dimensional inclination component; uses a three-dimensional reconstruction algorithm to reconstruct the steel coil axis centerline. The line position and inclination components are mapped to the three-dimensional space coordinate system to obtain the actual spatial position of the steel coil; according to the actual spatial position of the steel coil and combined with the preset forklift path, the adjustment amount of the forklift's travel direction and height is calculated to generate the result of dynamic path planning; based on the result of dynamic path planning, the steel coil fork barrel angle adjustment amount is calculated to obtain the optimal alignment angle between the steel coil fork barrel and the steel coil axis; according to the dynamic path planning and the steel coil fork barrel angle adjustment amount, a forklift control instruction is generated and sent to the forklift control system for execution; the real-time position information of the forklift is obtained and compared with the result of dynamic path planning. If the deviation exceeds the preset threshold, the adjustment amount is recalculated and the control instruction is updated; the above steps are executed repeatedly until the forklift completes the steel coil handling task.

[0009] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0010] The present invention discloses a method for handling steel coils with an intelligent forklift. In order to solve the problems of rust and oil pollution on the surface of the steel coil, a clear surface image of the steel coil is obtained by an image enhancement algorithm. The Hough transform algorithm is used to detect the contour of the steel coil, determine the position of the axis centerline, and calculate its angle with the horizontal plane. A three-dimensional reconstruction algorithm is used to map the position of the steel coil to a spatial coordinate system, and a dynamic planning result is generated in combination with a preset path. The angle adjustment amount of the steel coil fork barrel is calculated according to the planning result to achieve the best alignment between the steel coil fork barrel and the axis of the steel coil. The position of the forklift is monitored in real time and the control instructions are dynamically updated to ensure the accuracy of the handling process. The present invention can effectively solve the problems of positioning, path planning and steel coil fork barrel alignment in the handling of steel coils, improve the handling efficiency and safety, and realize the intelligence and automation of forklift operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings illustrate exemplary embodiments of the present invention and together with the description serve to explain the principles of the present invention. These drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification.

[0012] Figure 1 It is a structural schematic diagram of the transport assembly of the present invention;

[0013] Figure 2 It is a schematic diagram of the cooperation between the steel drum and the steel coil of the present invention;

[0014] Figure 3 It is a flow chart of a steel coil forklift control method of the present invention;

[0015] Figure 4 A flow chart of obtaining a steel coil surface texture image in a steel coil forklift control method of the present invention;

[0016] Figure 5 is a second schematic diagram of a steel coil forklift control method of the present invention;

[0017] Reference numerals:

[0018] 10. Mast frame; 11. Camera assembly; 20. Steel coil fork cylinder; 30. Steel coil. DETAILED DESCRIPTION

[0019] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0020] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] It should be understood that the term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not intended to limit the scope of these messages or information.

[0024] An automated steel coil forklift control method based on image recognition, the method relies on a steel coil transport assembly equipped with a camera; the transport assembly includes a camera assembly 11 installed on a door frame 10; the camera assembly 11 is used to obtain a surface image of a steel coil 30;

[0025] Control methods include:

[0026] Step S101, obtaining a surface image of the steel coil 30, and using an image enhancement algorithm to remove interference from rust and oil pollution to obtain a clear surface image of the steel coil 30.

[0027] The processor obtains the surface image of the steel coil 30, uses the adaptive median filter algorithm to remove the noise points in the image, determines the filter window size to be 3x3 pixels, and obtains the denoised image. According to the pre-established texture feature threshold, the grayscale co-occurrence matrix is ​​used to extract the texture feature area in the image and separate the background and target area. If the image contrast is lower than the preset threshold of 50, the histogram equalization algorithm is used to enhance the image contrast. The Canny edge detection algorithm is used to identify the edge features in the image and determine the surface defect area of ​​the steel coil 30. Combining the surface features and texture analysis results, the sample-based image repair algorithm is used to repair the defect area. Finally, a clear surface image of the steel coil 30 is obtained, and the surface quality inspection is completed. The image clarity and quality inspection standard are judged as the defect area area is less than 1% of the total area.

[0028] Specifically, obtaining the surface image of the steel coil 30 is the first step in quality inspection. The adaptive median filter algorithm can effectively remove noise points in the image and retain edge details. Taking a 3x3 pixel filter window as an example, the algorithm dynamically adjusts the filter intensity according to the distribution of surrounding pixel values, performs stronger filtering in smooth areas, and retains details in edge areas. The grayscale co-occurrence matrix is ​​an effective tool for extracting texture features. By calculating the grayscale relationship of pixel pairs, statistical features reflecting the texture can be obtained. For example, for a smooth steel coil 30 surface, the co-occurrence matrix will show a concentrated diagonal feature; for a rough surface, a discrete distribution will appear.

[0029] According to the pre-established threshold of the texture feature, the background and the target area can be effectively separated. Image contrast is crucial for defect detection. When the image contrast is lower than the preset threshold of 50, the histogram equalization algorithm can significantly improve the image quality. The algorithm redistributes the grayscale values ​​to make the dark areas brighter and the bright areas darker, thereby enhancing the overall contrast. This is particularly important for detecting tiny defects, as they are often hidden in low-contrast areas. The Canny edge detection algorithm is a key step in identifying surface defects of the steel coil 30. The algorithm first performs Gaussian smoothing on the image, then calculates the gradient amplitude and direction, and obtains the edge through non-maximum suppression and double thresholding. For linear defects such as cracks and scratches on the surface of the steel coil 30, the Canny algorithm can accurately locate their boundaries, providing a basis for subsequent defect analysis. The sample-based image repair algorithm can effectively repair the identified defect area. The algorithm analyzes the normal area around the defect, finds similar texture samples, and then fills the defect area with these samples. This method can not only repair defects, but also maintain the continuity of the surface texture, which is very important for maintaining the overall texture of the surface of the steel coil 30. Finally, a clear surface image of the steel coil 30 can be obtained by comprehensively analyzing the surface features and textures. The standard for judging the image clarity and quality is that the defect area is less than 1% of the total area. This standard not only ensures the overall quality of the steel coil 30, but also allows a certain degree of minor defects, reflecting the acceptable range in actual production. For example, for a 1000x1000 pixel surface image of the steel coil 30, the maximum allowable defect area is 10,000 pixels, which is approximately equal to a square area of ​​100x100 pixels. This process can not only effectively detect and repair surface defects of the steel coil 30, but also provide quantitative quality assessment. By adjusting the parameters of each step, such as the filter window size, texture feature threshold, etc., it can adapt to different types of steel coils 30 and different detection requirements, demonstrating the flexibility and practicality of the method.

[0030] Step S102 , for the clear surface image of the steel coil 30 , edge feature points are extracted, the contour of the steel coil 30 is detected using the Hough transform algorithm, and the position of the axis of the steel coil 30 is determined.

[0031] Obtain a clear surface image of the steel coil 30, use the Hough transform algorithm to extract the contour feature points in the image, and determine the axis centerline position of the steel coil 30. According to the pre-established texture feature threshold, use the grayscale co-occurrence matrix to analyze the texture distribution in the image and separate the target area from the background area. If the image contrast is lower than the preset threshold of 50, the histogram equalization algorithm is used to adjust the image grayscale distribution and improve the image contrast. For the target area after texture separation, the Canny edge detection algorithm is used to identify the edge features in the image and mark the surface defect area of ​​the steel coil 30. Combining the contour feature points and edge detection results, the sample-based image repair algorithm is used to repair the marked defective area. Through the repaired image, the ratio of the defective area area to the total area is calculated to determine whether the defective area is less than the preset threshold of 1%. According to the result of the defective area area judgment, the surface quality inspection report of the steel coil 30 is output.

[0032] Specifically, after obtaining a clear surface image of the steel coil 30, the edge feature points are extracted by the Canny edge detection algorithm. The algorithm uses a Gaussian smoothing parameter σ=5 and the gradient amplitude threshold is set to 50 and 150 to ensure that the edge information of the surface of the steel coil 30 can be accurately captured. Taking a steel coil 30 image with a resolution of 1000x1000 pixels as an example, the Canny algorithm can effectively identify the boundary contour of the steel coil 30 and generate a binary edge image. Then, the Hough transform algorithm is used to detect the straight line of the edge image. By setting the angle resolution to 1 degree and the distance resolution to 1 pixel, the contour straight line of the steel coil 30 can be accurately detected. For the detected straight line, the axis of the steel coil 30 is fitted by calculating its slope and midpoint coordinates and combining the least squares method. For example, when two parallel straight lines are detected, if the slope difference is less than 0.1, it can be determined that they are the upper and lower edges of the steel coil 30, and the axis of the steel coil 30 is determined by calculating the center line position of the two straight lines. This process can not only accurately locate the geometric center of the steel coil 30, but also provide a reliable basis for subsequent size measurement and position calibration. By comprehensively analyzing the edge features and the axis centerline position, the geometric accuracy of the steel coil 30 can be further evaluated, providing data support for production quality control.

[0033] According to the threshold of the surface texture feature of the steel coil 30, the gray level co-occurrence matrix is ​​used to analyze the texture distribution to determine whether there are surface defects that affect the quality of the steel coil 30. By detecting the area ratio of the defective area, the surface quality detection result of the steel coil 30 is obtained.

[0034] The surface texture image of the steel coil 30 is obtained, and the texture distribution is analyzed using the gray level co-occurrence matrix to extract the texture eigenvalue. According to the preset texture feature threshold, it is determined whether the texture eigenvalue exceeds the threshold. If it exceeds the threshold, it is marked as a defective area. For the marked defective area, the region growing algorithm is used to segment the boundary of the defective area and calculate the area of ​​the defective area:

[0035] ;

[0036] represents the total surface area of ​​the steel coil 30, Indicates the diameter of the steel coil 30, Indicates the length of the steel coil 30, Represents pi. This formula calculates the side area of ​​the cylinder plus the circular area of ​​the two end faces. Obtain the total surface area of ​​the steel coil 30, and use the area ratio formula to calculate the ratio of the defect area area to the total area. According to the preset area ratio threshold, determine whether the defect area area ratio exceeds the threshold. If it exceeds the threshold, it is determined to be an unqualified area. Use the image repair algorithm to repair the unqualified area and generate a repaired surface image of the steel coil 30. According to the repaired image, recalculate the defect area area ratio to determine the final surface quality inspection result of the steel coil 30.

[0037] Specifically, when analyzing the texture features of the surface image of the steel coil 30, the gray level co-occurrence matrix GLCM is first used as the core algorithm, and the gray level is set to 32, the distance parameter is 1 pixel, and the angles are 0°, 45°, 90° and 135° in four directions to fully capture the texture distribution characteristics of the surface of the steel coil 30. By calculating the texture feature parameters such as contrast, correlation, energy and homogeneity, for example, when the contrast value is greater than 300 and the homogeneity value is less than 2, it can be preliminarily determined that the area has surface defects. Taking a steel coil 30 image with a resolution of 2000x2000 pixels as an example, the GLCM algorithm can effectively extract texture features and generate the corresponding feature matrix. To further verify the defective area, the adaptive threshold segmentation technology is used to set the threshold range of the grayscale image to 50 to 200, and the morphological filtering such as opening and closing operations are combined to remove noise interference, and the binary image of the defective area is extracted. By calculating the ratio of the defective area to the total surface area of ​​the steel coil 30, for example, when the defective area accounts for more than 5%, it is determined that the steel coil 30 has surface quality problems. Finally, the test results are compared with the preset quality standards to generate a surface quality test report for steel coil 30, providing data support for subsequent grading and process optimization. This process can not only efficiently identify surface defects, but also provide a scientific basis for the comprehensive evaluation of steel coil 30 quality.

[0038] Step S103, according to the position of the axis of the steel coil 30, the angle between the axis of the steel coil 30 and the horizontal plane is calculated to obtain a multi-dimensional inclination component.

[0039] The spatial position coordinates of the axis of the steel coil 30 are determined by a preset coordinate system. According to the spatial position coordinates, the angle between the axis of the steel coil 30 and the horizontal plane is calculated by trigonometric functions. If the angle exceeds the preset threshold, multi-dimensional inclination component data is obtained from the image of the camera assembly 11. According to the multi-dimensional inclination component data, its distribution law is analyzed by linear regression. The inclination trend of the steel coil 30 is determined by the distribution law. According to the inclination trend, the position parameters of the steel coil 30 are adjusted by the gradient descent method.

[0040] Specifically, after determining the position of the axis centerline of the steel coil 30, the slope of the axis centerline is calculated to further analyze its angle with the horizontal plane. First, the axis centerline equation y=kx+b is fitted using the least squares method, where the slope k=25 and the intercept b=120. Based on the slope formula θ=arctan(k), the angle θ=104 degrees between the axis centerline and the horizontal plane is calculated. In order to analyze the inclination components of the steel coil 30 in multiple dimensions, a three-dimensional coordinate system is introduced. It is assumed that the axis centerline of the steel coil 30 is in the xy plane. By calculating the angles between the axis centerline and the x-axis and y-axis, the horizontal inclination component α=104 degrees and the vertical inclination component β=796 degrees are obtained respectively. To further verify the accuracy of the inclination components, the principal component analysis method is used to decompose the axis centerline eigenvector to obtain the eigenvalue λ 1 =03 and λ 2 =97, indicating that the inclination component of the axis in the horizontal direction is dominant. Combined with the calculation results of the inclination component, the spatial posture of the steel coil 30 is simulated through reverse engineering to verify that it is consistent with the actual placement state of the steel coil 30 in the production environment, providing accurate data support for subsequent automated adjustment and process optimization. Through the multi-dimensional analysis of the inclination component, the spatial position deviation of the steel coil 30 can be comprehensively evaluated to ensure the geometric accuracy and quality control in the production process.

[0041] Step S104 , using a three-dimensional reconstruction algorithm, maps the axis position and inclination angle components of the steel coil 30 to a three-dimensional space coordinate system to obtain the actual spatial position of the steel coil 30 .

[0042] A three-dimensional reconstruction algorithm is used to obtain the axis position and inclination component data of the steel coil 30, which are mapped to a preset three-dimensional space coordinate system to obtain the actual spatial position of the steel coil 30. According to the actual spatial position of the steel coil 30, the coordinate value of the steel coil 30 in the three-dimensional coordinate system is determined, and the parameters in the coordinate value are extracted. According to the parameters in the coordinate value of the steel coil 30, a preset mathematical model is used to calculate the posture information of the steel coil 30 in the three-dimensional space. According to the posture information of the steel coil 30, a clustering algorithm is used to analyze the distribution law of the steel coil 30 in the three-dimensional space to determine the spatial state of the steel coil 30. If the spatial state of the steel coil 30 is abnormal, the inclination component value in the posture information of the steel coil 30 is extracted, and the deviation range of the inclination component is calculated. According to the deviation range of the inclination component, a regression algorithm is used to predict the spatial change trend of the steel coil 30 to obtain the future spatial position of the steel coil 30. According to the future spatial position of the steel coil 30, the parameter setting of the steel coil 30 in the three-dimensional coordinate system is adjusted to complete the optimization of the spatial position of the steel coil 30.

[0043] Specifically, a three-dimensional reconstruction algorithm is used to map the axis position and inclination components of the steel coil 30 to a three-dimensional space coordinate system. First, the three-dimensional point cloud data of the surface of the steel coil 30 is obtained through a point cloud data acquisition device, and the point cloud data is aligned using the ICP algorithm to iterate the nearest point algorithm to obtain the three-dimensional coordinates of the axis of the steel coil 30. Assuming that the starting coordinates of the axis of the steel coil 30 in space are 100, 200, 300, and the end coordinates are 400, 500, 600, based on the spatial straight line equation, the direction vector of the axis in three-dimensional space is calculated to be 300, 300, 300, and the vector angle formula is further used to calculate the angles between the axis and the x-axis, y-axis, and z-axis, respectively, α=45 degrees, β=45 degrees, and γ=45 degrees, to obtain the actual inclination components of the steel coil 30 in three-dimensional space. In order to accurately describe the spatial position of the steel coil 30, a homogeneous coordinate transformation matrix is ​​used to transform the local coordinate system of the axis into a global coordinate system. Assume that the transformation matrix is:

[0044] [1,0,0,100;

[0045] 0,1,0,200;

[0046] 0,0,1,300;

[0047] 0,0,0,1];

[0048] The actual position of the axis in the global coordinate system is obtained through matrix operations. Combined with the geometric parameters of the steel coil 30, such as radius R=500mm and length L=2000mm, the geometric model of the steel coil 30 is constructed using a 3D modeling algorithm, and the spatial posture of the steel coil 30 is displayed in real time through a rendering engine, providing accurate spatial data support for subsequent automated adjustments and process optimization. Through the application of a 3D reconstruction algorithm, the spatial position deviation of the steel coil 30 can be fully evaluated to ensure geometric accuracy and quality control during the production process.

[0049] Step S105, according to the actual spatial position of the steel coil 30, combined with the preset forklift path, the adjustment amount of the forklift's travel direction and height is calculated to generate a result of dynamic path planning.

[0050] Get the actual spatial position of the steel coil 30 and extract the coordinate value of the steel coil 30 in the three-dimensional coordinate system. According to the coordinate value of the steel coil 30, combined with the preset forklift path, calculate the adjustment amount of the forklift's travel direction and height. Use the preset dynamic path planning algorithm to generate the preliminary result of the forklift's travel path. For the preliminary path results, extract the key points in the path, and calculate the smoothness and continuity of the path. If the path smoothness does not meet the preset threshold, readjust the calculation amount of the forklift's travel direction and height. Based on the adjusted calculation results, regenerate the final result of the forklift's travel path. Compare the final path result with the preset forklift path to determine the accuracy of the path planning.

[0051] Specifically, based on the actual spatial position of the steel coil 30, the dynamic path planning of the forklift is realized by real-time matching of the preset path with the position information of the steel coil 30. Assume that the starting point of the preset forklift path is 0, 0, 0, the end point is 1000, 1000, 0, the path direction vector is 1000, 1000, 0, and the current coordinates of the forklift are 200, 200, 0. By calculating the distance vector -100, 0, 300 between the current position of the forklift and the starting point 100, 200, 300 of the axis of the steel coil 30, the distance is 3123mm using the Euclidean distance formula. In order to adjust the direction of travel of the forklift, a vector projection algorithm is used to project the distance vector onto the preset path direction vector, and the projection length is -100mm, indicating that the forklift needs to move 100mm along the path direction to align with the center of the steel coil 30. At the same time, by calculating the height difference of 300mm between the forklift and the axis of the steel coil 30, the forklift lifting mechanism is adjusted to the target height. During the path planning process, the dynamic time warping algorithm DTW is used to match the forklift position and the spatial posture of the steel coil 30 in real time to ensure the smoothness and accuracy of the path adjustment. Combined with the forklift kinematic model, control instructions including speed, acceleration and steering angle are generated to achieve efficient and accurate operation of the forklift, providing reliable guarantee for the handling of the steel coil 30.

[0052] Step S106, based on the result of dynamic path planning, calculate the angle adjustment amount of the steel coil fork cylinder 20 to obtain the optimal alignment angle between the steel coil fork cylinder 20 and the axis of the steel coil 30.

[0053] The current angle information of the steel coil fork barrel 20 is obtained, and the axis position data of the steel coil 30 is extracted from the preset three-dimensional coordinate system. The alignment deviation value between the steel coil fork barrel 20 and the axis is calculated through the angle information and axis position data of the steel coil fork barrel 20. The calculation of the alignment deviation value is derived through the geometric relationship between the angle of the steel coil fork barrel 20 and the axis position. A preset angle adjustment algorithm is used to generate a preliminary adjustment amount for the alignment deviation value. The angle adjustment algorithm calculates the angle amount that needs to be adjusted for the steel coil fork barrel 20 based on the size and direction of the alignment deviation value. The generation of the preliminary adjustment amount is achieved by inputting the alignment deviation value into the angle adjustment algorithm. According to the preset threshold range, it is judged whether the preliminary adjustment amount meets the alignment accuracy requirements. The preset threshold range is a preset alignment accuracy standard. The judgment process is performed by comparing the size of the preliminary adjustment amount with the preset threshold. If the preliminary adjustment amount exceeds the threshold, the adjustment amount of the angle of the steel coil fork barrel 20 is recalculated. The recalculation process adjusts the parameters of the angle adjustment algorithm based on the difference between the preliminary adjustment amount and the preset threshold to generate a new adjustment amount. An iterative calculation method is used to perform deviation analysis on the recalculated adjustment amount to determine the deviation range from the preset threshold. The iterative calculation method gradually reduces the deviation between the adjustment amount and the preset threshold by adjusting the parameters of the angle adjustment algorithm multiple times. The deviation analysis process is performed by comparing the adjustment amount after each iteration with the preset threshold. According to the deviation analysis results, the final adjustment amount of the steel coil fork barrel 20 angle is generated, and the optimal alignment angle between the steel coil fork barrel 20 and the axis of the steel coil 30 is determined. The generation of the final adjustment amount is based on the deviation analysis results, and the angle amount that minimizes the deviation between the adjustment amount and the preset threshold is selected. The determination of the optimal alignment angle is achieved by applying the final adjustment amount to the steel coil fork barrel 20 angle.

[0054] Specifically, based on the results of dynamic path planning, the angle adjustment of the steel coil fork barrel 20 is calculated to achieve the optimal alignment angle between the steel coil fork barrel 20 and the axis of the steel coil 30. First, the current position and posture information of the steel coil 30 need to be obtained. Assume that the current position coordinates of the steel coil 30 are 1500, 2000, 300, and the posture angles are 10°, 15°, and 5°. Through the current positions of the steel coil fork barrel 20 of 1400, 1900, 280 and the posture angles of 8°, 12°, and 4°, the relative position and angle deviation between the two can be calculated using a spatial geometry algorithm.

[0055] The specific algorithm is as follows: first, calculate the offsets of the steel coil 30 and the steel coil fork barrel 20 in the three directions of X, Y, and Z, which are ΔX=100, ΔY=100, and ΔZ=20, respectively. Then, the posture angle is converted into a rotation matrix through the Euler angle conversion formula, and the rotation angle that the steel coil fork barrel 20 needs to be adjusted is calculated. Assuming that the posture angle deviation is Δθx=2°, Δθy=3°, and Δθz=1°, the steel coil fork barrel 20 needs to rotate 2°, 3°, and 1° around the X, Y, and Z axes, respectively, to achieve optimal alignment. Further, through the inverse kinematics model, the rotation angle is converted into a control signal for driving the gantry frame 10. Assuming that the gantry frame 10 is driven by a motor, the motor can drive the gantry frame to move the X, Y, and Z axes. At the same time, assuming that the number of pulses corresponding to each degree of the motor is 100, the X-axis motor needs to send 200 pulses, the Y-axis motor needs to send 300 pulses, and the Z-axis motor needs to send 100 pulses. Finally, the position and angle of the steel coil fork cylinder 20 are monitored in real time through a closed-loop control system to ensure the accuracy and stability of the adjustment process, thereby achieving precise alignment of the steel coil fork cylinder 20 with the axis of the steel coil 30.

[0056] Step S107, generating a forklift control instruction based on the dynamic path planning and the angle adjustment amount of the steel reel fork barrel 20, and sending it to the forklift control system for execution.

[0057] The path planning is obtained through the dynamic path planning algorithm, and the adjustment amount is calculated in combination with the angle value collected by the steel coil fork barrel 20° angle sensor. According to the adjustment amount and path planning, a control command is generated, and the control command is sent using the forklift control system interface. The control command is received in the forklift control system, the control command content is parsed, and the steel coil fork barrel 20° angle adjustment parameters and motion path are determined. The steel coil fork barrel 20° angle adjustment parameters are input into the steel coil fork barrel 20° angle adjustment module, the angle adjustment is performed, and the steel coil fork barrel 20° angle correction is completed. Through the path planning and forklift motion control module, the forklift is driven to move along the path of the path planning result, and the execution status of the forklift control system is monitored in real time. If the execution status is abnormal, the path planning and angle value are re-obtained, the new adjustment amount is calculated, and a new control command is generated. The new control command is sent to the forklift control system to complete the execution process of the forklift dynamic path planning and steel coil fork barrel 20° angle adjustment.

[0058] Specifically, in dynamic path planning, the position information of obstacles is first obtained around the camera assembly 11 to generate a two-dimensional map. The A* algorithm is used for path planning, and the starting point is set to 0,0 and the end point is set to 10,10 to calculate the optimal path. According to the path results, the forklift motion control instructions are generated, including a forward speed of 5 meters per second and a turning radius of 2 meters. At the same time, the position of the goods is detected by the visual system, and the angle adjustment amount of the steel coil fork barrel 20 is calculated. The PID control algorithm is used to set the target angle to 45 degrees, the proportional coefficient to 8, the integral coefficient to 1, and the differential coefficient to 2, and the steel coil fork barrel 20 angle is adjusted in real time. The results of path planning and the steel coil fork barrel 20 angle adjustment amount are integrated to generate the final control instruction, which is sent to the forklift control system through the CAN bus. After receiving the instruction, the control system drives the forklift to move forward at the specified speed, and the hydraulic system adjusts the steel coil fork barrel 20 angle to ensure that the forklift reaches the target position accurately and completes the cargo forking. The whole process is fed back in real time through the positioning system data to ensure the safety and accuracy of the forklift operation.

[0059] Step S108, obtaining the real-time position information of the forklift and comparing it with the result of the dynamic path planning. If the deviation exceeds a preset threshold, the adjustment amount is recalculated and the control instruction is updated.

[0060] The real-time position information of the forklift is obtained by the positioning system, and the results of dynamic path planning are extracted from the path planning module. According to the coordinate data of the real-time position and the planned path, the deviation between the two is calculated. If the deviation exceeds the preset threshold, the adjustment amount calculation module is triggered to recalculate the forklift movement adjustment amount. The calculated adjustment amount is input into the control instruction generation module to generate a new control instruction. The updated control instruction is sent to the forklift control system through the communication module. The forklift movement trajectory is corrected according to the control instruction so that the actual path is consistent with the planned path. The above process is executed repeatedly to realize the dynamic adjustment and real-time control of the forklift path.

[0061] Specifically, the GPS module installed on the forklift is used to obtain the latitude and longitude coordinates of the forklift in real time, with a sampling frequency of 1 time per second and an accuracy of up to 0.5 meters. The acquired coordinate data is transmitted to the central control system through a wireless network. The system uses the Kalman filter algorithm to smooth the position information and reduce random errors. The processed real-time position is compared with the preset path, and the deviation value is calculated using the Euclidean distance formula, with a threshold set to 1.2 meters. When the deviation is detected to exceed the threshold, the A* algorithm is used to replan the path, taking into account factors such as the location of obstacles and the steering radius limit of the forklift to generate a new optimal path. The system calculates the steering angle and speed adjustment amount based on the new path, and generates control instructions through the PID controller. The control cycle is 200 milliseconds, the proportional coefficient Kp is set to 0.5, the integral coefficient Ki is set to 0.2, and the differential coefficient Kd is set to 0.1. The updated control instructions are sent to the forklift controller through the Wi-Fi network, and the actuator adjusts the door frame 10 according to the instructions to achieve path correction. At the same time, the system records the time, position and adjustment amount of each adjustment for subsequent path optimization analysis to improve the efficiency of dynamic planning.

[0062] Step S109, the above steps are executed in a loop until the forklift completes the task of transporting the steel coil 30, ensuring that the path planning and the angle of the steel coil fork barrel 20 match the axis of the steel coil 30 in real time.

[0063] Use the positioning system to obtain the current position of the forklift and use the image of the camera component 11 to obtain the axis coordinates of the steel coil 30. Based on the acquired coordinate information, the Euclidean distance formula is used to calculate the relative position between the forklift and the axis of the steel coil 30. Determine whether the relative position exceeds the preset threshold. If it exceeds the threshold, control the door frame 10 to adjust the angle of the steel coil fork cylinder 20 so that the steel coil fork cylinder 20 is aligned with the axis of the steel coil 30. According to the adjusted angle of the steel coil fork cylinder 20, use the A algorithm to re-plan the movement path of the forklift. Execute the path movement, and use the camera component 11 to monitor the changes in the axis position of the steel coil 30 in real time. If the axis position of the steel coil 30 changes, update the relative position calculation. Repeat the above steps until the forklift reaches the target position. After completing the handling task, use the database to record the task status and execution time.

[0064] Specifically, when the forklift performs the task of transporting the steel coil 30, the camera assembly 11 is first used to capture the surrounding environment and generate a high-precision three-dimensional map with a map accuracy of millimeter level. The path planning algorithm of the forklift adopts the A* algorithm, combined with the dynamic window method DWA for real-time obstacle avoidance, to ensure the safe driving of the forklift in a complex environment. The control system of the forklift automatically adjusts the angle and height of the steel coil fork cylinder 20 according to the diameter and weight of the steel coil 30 to ensure that the steel coil fork cylinder 20 accurately matches the axis of the steel coil 30. For example, when the diameter of the steel coil 30 is 1500 mm, the angle of the steel coil fork cylinder 20 is automatically adjusted to 45 degrees, and the height of the steel coil fork cylinder 20 is adjusted to 750 mm. The camera assembly 11 of the forklift monitors the position and posture of the steel coil 30 in real time, and performs data fusion through the Kalman filter algorithm to improve the positioning accuracy. During the handling process, the control system of the forklift continuously optimizes the path planning to ensure the maximum handling efficiency. For example, when the forklift detects an obstacle in front, the system will immediately re-plan the path to avoid the obstacle and ensure the smooth completion of the task.

[0065] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside" and "outside" etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0066] It should be understood by those skilled in the art that the above embodiments are only for the purpose of clearly illustrating the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present invention.

Claims

1. An automated steel coil forklift control method based on image recognition, characterized in that: The method relies on a steel coil transport assembly equipped with a camera; the transport assembly comprises a camera assembly (11) mounted on a gantry frame (10); the camera assembly (11) is used to obtain a surface image of a steel coil (30), and a steel coil fork cylinder (20) is also mounted on the gantry frame (10); The forklift also has a positioning system and processor installed; Control methods include: The processor obtains a surface image of the steel coil (30), and uses an image enhancement algorithm to remove interference from rust and oil pollution, thereby obtaining a clear surface image of the steel coil (30); Extract edge feature points from a clear surface image of the steel coil (30), use a Hough transform algorithm to detect the contour of the steel coil (30), and determine the axis centerline position of the steel coil (30); According to the position of the axis centerline of the steel coil (30), the angle between the axis centerline of the steel coil (30) and the horizontal plane is calculated to obtain multi-dimensional inclination angle components; Using a three-dimensional reconstruction algorithm, the axis position and the inclination component of the steel coil (30) are mapped to a three-dimensional space coordinate system to obtain the actual spatial position of the steel coil (30); According to the actual spatial position of the steel coil (30), combined with the preset forklift path, the adjustment amount of the forklift's travel direction and height is calculated to generate a dynamic path planning result; Based on the result of dynamic path planning, the angle adjustment amount of the steel coil fork cylinder (20) is calculated to obtain the optimal alignment angle between the steel coil fork cylinder (20) and the axis of the steel coil (30); Generating a forklift control command based on the dynamic path planning and the angle adjustment amount of the steel coil fork barrel (20), and sending it to the forklift control system for execution; Obtain the real-time position information of the forklift and compare it with the result of dynamic path planning. If the deviation exceeds the preset threshold, recalculate the adjustment amount and update the control instructions; The above steps are executed repeatedly until the forklift completes the task of transporting the steel coil (30).

2. The method for controlling an automated steel coil forklift based on image recognition according to claim 1, characterized in that: The surface image of the steel coil (30) is obtained, and the interference of rust and oil pollution is removed by using an image enhancement algorithm to obtain a clear surface image of the steel coil (30), including: Acquire a surface image of the steel coil (30), use an adaptive median filter algorithm to remove noise points in the image, determine the filter window size to be 3x3 pixels, and obtain a denoised image; According to the pre-established texture feature threshold, the gray level co-occurrence matrix is ​​used to extract the texture feature area in the image and separate the background and the target area; If the image contrast is lower than the preset threshold of 50, the histogram equalization algorithm is used to enhance the image contrast; Using a Canny edge detection algorithm to identify edge features in the image, and determine the surface defect area of ​​the steel coil (30); Combining the surface features and texture analysis results, a sample-based image restoration algorithm is used to repair the defective area. Finally, a clear surface image of the steel coil (30) is obtained, and the surface quality inspection is completed. The image clarity and quality inspection standard are judged as follows: the defect area is less than 1% of the total area.

3. The method for controlling an automated steel coil forklift based on image recognition according to claim 1, characterized in that: For a clear surface image of the steel coil (30), edge feature points are extracted, and the contour of the steel coil (30) is detected using a Hough transform algorithm to determine the axis centerline position of the steel coil (30), including: Obtaining a clear surface image of the steel coil (30), extracting contour feature points in the image using a Hough transform algorithm, and determining the axis centerline position of the steel coil (30); According to the pre-established texture feature threshold, the gray level co-occurrence matrix is ​​used to analyze the texture distribution in the image and separate the target area from the background area; If the image contrast is lower than the preset threshold of 50, the histogram equalization algorithm is used to adjust the image grayscale distribution to improve the image contrast; For the target area after texture separation, the Canny edge detection algorithm is used to identify edge features in the image and mark the surface defect area of ​​the steel coil (30); Combining the contour feature points and edge detection results, a sample-based image restoration algorithm is used to repair the marked defect area; Through the repaired image, the ratio of the defect area to the total area is calculated to determine whether the defect area is smaller than the preset threshold of 1%; Outputting a surface quality inspection report of the steel coil (30) according to the defect area determination result; The method further comprises: analyzing the texture distribution using a grayscale co-occurrence matrix according to a threshold value of the surface texture feature of the steel coil (30), determining whether there are surface defects that affect the quality of the steel coil (30), and obtaining a surface quality detection result of the steel coil (30) by detecting an area ratio of a defective region.

4. The method for controlling an automated steel coil forklift based on image recognition according to claim 3, characterized in that: According to the position of the axis centerline of the steel coil (30), the angle between the axis centerline of the steel coil (30) and the horizontal plane is calculated to obtain multi-dimensional inclination components, including: Using a preset coordinate system to determine the spatial position coordinates of the axis of the steel coil (30); According to the spatial position coordinates, the angle between the axis of the steel coil (30) and the horizontal plane is calculated by trigonometric function; If the angle exceeds a preset threshold, multi-dimensional inclination component data is obtained from the sensor; According to the multi-dimensional inclination component data, linear regression is used to analyze its distribution law; Determining the inclination trend of the steel coil (30) through the distribution law; According to the tilt trend, the position parameters of the steel coil (30) are adjusted using a gradient descent method.

5. The method for controlling an automated steel coil forklift based on image recognition according to claim 4, characterized in that: A three-dimensional reconstruction algorithm is used to map the axis position and inclination angle components of the steel coil (30) to a three-dimensional space coordinate system to obtain the actual spatial position of the steel coil (30), including: Using a three-dimensional reconstruction algorithm, the axis position and inclination component data of the steel coil (30) are obtained, and mapped into a preset three-dimensional space coordinate system to obtain the actual spatial position of the steel coil (30); Determining the coordinate value of the steel coil (30) in the three-dimensional coordinate system according to the actual spatial position of the steel coil (30), and extracting parameters from the coordinate value; For the parameters in the coordinate values ​​of the steel coil (30), a preset mathematical model is used to calculate the posture information of the steel coil (30) in three-dimensional space; According to the posture information of the steel coil (30), a clustering algorithm is used to analyze the distribution pattern of the steel coil (30) in the three-dimensional space to determine the spatial state of the steel coil (30); If the spatial state of the steel coil (30) is abnormal, extracting the inclination component value in the posture information of the steel coil (30), and calculating the deviation range of the inclination component; According to the deviation range of the inclination component, a regression algorithm is used to predict the spatial variation trend of the steel coil (30) to obtain the future spatial position of the steel coil (30); According to the future spatial position of the steel coil (30), the parameter setting of the steel coil (30) in the three-dimensional coordinate system is adjusted to complete the optimization of the spatial position of the steel coil (30).

6. The method for controlling an automated steel coil forklift based on image recognition according to claim 5, characterized in that: According to the actual spatial position of the steel coil (30), combined with the preset forklift path, the adjustment amount of the forklift's travel direction and height is calculated to generate a result of dynamic path planning, including: Obtaining the actual spatial position of the steel coil (30) and extracting the coordinate value of the steel coil (30) in the three-dimensional coordinate system; According to the coordinate value of the steel coil (30), combined with the preset forklift path, the adjustment amount of the forklift's travel direction and height is calculated; Use the preset dynamic path planning algorithm to generate preliminary results of the forklift's travel path; Based on the preliminary path results, key points in the path are extracted and the smoothness and continuity of the path are calculated; If the path smoothness does not meet the preset threshold, the calculation of the forklift's travel direction and height is readjusted; Regenerate the final result of the forklift's travel path based on the adjusted calculation results; Compare the final path result with the preset forklift path to determine the accuracy of the path planning.

7. The method for controlling an automated steel coil forklift based on image recognition according to claim 6, characterized in that: Based on the result of dynamic path planning, the angle adjustment amount of the steel coil fork barrel (20) is calculated to obtain the optimal alignment angle between the steel coil fork barrel (20) and the axis of the steel coil (30), including: Obtaining current angle information of the steel coil fork barrel (20), and extracting the axis position data of the steel coil (30) from a preset three-dimensional coordinate system; Calculating the alignment deviation value between the steel coil fork tube (20) and the axis center through the angle information of the steel coil fork tube (20) and the axis center position data; The calculation of the alignment deviation value is derived from the geometric relationship between the angle of the steel reel fork barrel (20) and the axis position; Using a preset angle adjustment algorithm, a preliminary adjustment amount is generated for the alignment deviation value; The angle adjustment algorithm calculates the angle amount that the steel reel fork cylinder (20) needs to be adjusted based on the magnitude and direction of the alignment deviation value; The generation of the preliminary adjustment amount is achieved by inputting the alignment deviation value into the angle adjustment algorithm; According to a preset threshold range, determining whether the initial adjustment amount meets the alignment accuracy requirement; The preset threshold range is a preset alignment accuracy standard; The judgment process is carried out by comparing the initial adjustment amount with the preset threshold value; If the initial adjustment amount exceeds the threshold, the adjustment amount of the angle of the steel reel fork cylinder (20) is recalculated; The recalculation process adjusts the parameters of the angle adjustment algorithm based on the difference between the preliminary adjustment amount and the preset threshold value to generate a new adjustment amount; An iterative calculation method is used to perform deviation analysis on the recalculated adjustment amount to determine the deviation range from the preset threshold; The iterative calculation method gradually reduces the deviation between the adjustment amount and the preset threshold by adjusting the parameters of the angle adjustment algorithm multiple times; The process of deviation analysis is performed by comparing the adjustment amount after each iteration with the preset threshold; According to the deviation analysis result, a final adjustment amount of the angle of the steel coil fork cylinder (20) is generated, and the optimal alignment angle between the steel coil fork cylinder (20) and the axis of the steel coil (30) is determined; The final adjustment amount is generated based on the deviation analysis result, and the angle amount that minimizes the deviation between the adjustment amount and the preset threshold is selected; The optimum alignment angle is determined by applying a final adjustment to the steel reel barrel (20) angle.

8. The method for controlling an automated steel coil forklift based on image recognition according to claim 7, characterized in that: According to the dynamic path planning and the angle adjustment amount of the steel coil fork barrel (20), a forklift control instruction is generated and sent to the forklift control system for execution, including: Obtaining the path planning through a dynamic path planning algorithm, and calculating the adjustment amount in combination with the angle value collected by the angle sensor of the steel reel fork barrel (20); Generate control commands based on adjustment amount and path planning, and send control commands through the forklift control system interface; The forklift control system receives the control command, analyzes the content of the control command, and determines the angle adjustment parameters and movement path of the steel coil fork barrel (20); Inputting the angle adjustment parameters of the steel coil fork barrel (20) into the steel coil fork barrel (20) angle adjustment module, performing angle adjustment, and completing the angle correction of the steel coil fork barrel (20); Through the path planning and forklift motion control module, the forklift is driven to move along the path planning result, and the execution status of the forklift control system is monitored in real time; If the execution status is abnormal, re-obtain the path planning and angle value, calculate the new adjustment amount, and generate a new control command; A new control command is sent to the forklift control system to complete the execution process of forklift dynamic path planning and steel coil fork barrel (20) angle adjustment.

9. The method for controlling an automated steel coil forklift based on image recognition according to claim 8, characterized in that: Obtain the real-time position information of the forklift and compare it with the result of dynamic path planning. If the deviation exceeds the preset threshold, recalculate the adjustment amount and update the control instructions, including: Use the positioning system to obtain the real-time location information of the forklift and extract the results of dynamic path planning from the path planning module; According to the coordinate data of the real-time position and the planned path, the deviation value between the two is calculated; If the deviation value exceeds the preset threshold, the adjustment amount calculation module is triggered to recalculate the forklift movement adjustment amount; The calculated adjustment amount is input into a control instruction generation module to generate a new control instruction; Sending updated control instructions to the forklift control system via the communication module; Correct the forklift's motion trajectory according to the control instructions to make the actual path consistent with the planned path; The above process is executed cyclically to realize dynamic adjustment and real-time control of the forklift path.

10. The method for controlling an automated steel coil forklift based on image recognition according to claim 9, characterized in that: The above steps are executed repeatedly until the forklift completes the task of transporting the steel coil (30), ensuring that the path planning and the angle of the steel coil fork barrel (20) match the axis of the steel coil (30) in real time, including: Using the positioning system to obtain the current position of the forklift and the axis coordinates of the steel coil (30); Based on the acquired coordinate information, the relative position between the forklift and the axis of the steel coil (30) is calculated using the Euclidean distance formula; Determine whether the relative position exceeds a preset threshold; If the threshold is exceeded, the angle of the steel coil fork cylinder (20) is adjusted by controlling the door frame (10) so that the steel coil fork cylinder (20) is aligned with the axis of the steel coil (30); According to the adjusted angle of the steel coil fork barrel (20), the moving path of the forklift is replanned using the A algorithm; Execute the path movement and use the camera assembly (11) to monitor the change of the axis position of the steel coil (30) in real time; If the axis position of the steel coil (30) changes, the relative position calculation is updated; The above steps are executed repeatedly until the forklift reaches the target position; After completing the transport task, use the database to record the task status and execution time.

Images