First steel plate identification method, readable storage medium and computer program product
By acquiring top-view images from incoming steel plates and using image processing technology to identify the edge contour of the topmost steel plate, and combining this with side-view images to obtain size and pose information, the problem of automated hoisting difficulties caused by mixed specifications and disordered stacking of incoming steel plates was solved, achieving efficient and safe automated hoisting.
Patent Information
- Application Number
- CN202610592758.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-30
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to accurately identify and locate the first steel plate on the top layer when incoming steel plates are of varying specifications and stacked in an disorderly manner, resulting in low efficiency and safety hazards in automated hoisting.
By acquiring top-view images of multiple stacked steel plates, image processing technology is used to extract edge contours, and the edge contour of the topmost first steel plate is identified based on preset filtering rules. Combined with side-view images, size and pose information are obtained to achieve precise positioning of the first steel plate.
It improves the accuracy and safety of automated hoisting, increases production efficiency, avoids the drawbacks of manual operation, and ensures the precision and safety of hoisting.
Smart Images

Figure CN123118433A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, and in particular to a method for recognizing the first steel plate, a readable storage medium, and a computer program product for use in industrial automation. Background Technology
[0002] Steel plates are an indispensable basic raw material in modern industry, especially in heavy industries such as pipe manufacturing, shipbuilding, construction, and machinery manufacturing. In the logistics and production of steel plates, automated material handling, such as using intelligent unmanned vehicles for unloading, transfer, and loading, is a key development direction for improving production efficiency, reducing labor costs, and ensuring production safety.
[0003] In the process of automating steel plate hoisting, a core and fundamental step is the accurate identification and positioning of the steel plates to be hoisted. However, in actual industrial scenarios, there are significant challenges in the incoming steel plate stage. Raw material steel plates are usually supplied by different steel mills in different batches, resulting in varying specifications and shapes of the incoming steel plates. Adding to the complexity, during transportation and temporary storage, these steel plates of different specifications are often stacked together in a random and disorderly manner, with overlapping, skewed, and irregular patterns.
[0004] This situation of inconsistent specifications and disorderly stacking presents significant challenges to automated material handling and subsequent logistics warehousing. Traditional automated systems typically rely on materials having uniform specifications or being neatly stacked. However, existing visual recognition methods often fail when faced with such complex conditions. For example, because the edges of lower steel plates are severely obscured by upper steel plates, the images contain numerous incomplete and broken contours. Simple image segmentation or contour extraction algorithms struggle to accurately distinguish which contour belongs to the topmost steel plate, the first one that can be lifted. This confusion or failure in recognition directly prevents automated lifting equipment from obtaining accurate steel plate size and orientation information, hindering subsequent gripping and alignment, and potentially leading to production safety accidents such as steel plate drops or equipment collisions due to positioning errors.
[0005] Currently, to address this issue, many factories have reverted to a manual operation model, where experienced workers visually inspect and remotely control the positioning and hoisting of the first steel plate. This method is not only inefficient and labor-intensive, but also heavily reliant on human experience, leading to operational errors and safety hazards, thus becoming a bottleneck and a weakness in the entire automated production process. Therefore, there is an urgent need in this field for a new method that can overcome the challenges of mixed incoming steel plate specifications and disordered stacking, and can automatically, quickly, and accurately identify the size and position information of the first steel plate on the top layer of the stack, thereby opening up key technical links in the automated steel plate unloading process. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for identifying the first steel plate. This method aims to solve the technical problem in the prior art where the mixed specifications and disordered stacking of incoming steel plates make it difficult for automated systems to accurately identify and locate the topmost first steel plate, thus affecting the efficiency and safety of automated hoisting.
[0007] To achieve the aforementioned objectives, this invention provides a method for identifying the first steel plate. This method involves acquiring a top-view image of multiple steel plates stacked together; processing the top-view image to extract edge contours; then, creatively proposing a preset filtering rule to identify the edge contour representing the topmost first steel plate from the extracted edge contours; finally, determining the size and pose information of the first steel plate based on its identified edge contour. From a top-view perspective, due to physical occlusion, the contour of the topmost first steel plate has the highest completeness and regularity among all steel plate contours. This invention utilizes this principle to set filtering rules, thereby accurately identifying the unique contour belonging to the first steel plate from numerous incomplete lower-layer steel plate contours caused by occlusion. Therefore, this invention solves the problem of existing technologies struggling to handle inconsistent incoming material specifications and disordered stacking, resulting in high identification accuracy and strong applicability to complex working conditions. By determining the precise dimensions and orientation of the first steel plate, this invention can provide comprehensive and reliable guidance data for automated hoisting equipment, realizing closed-loop automation from identification to positioning, thereby significantly improving production efficiency and operational safety, and overcoming the drawbacks of traditional manual operation.
[0008] Preferably, the method further includes the following steps: Based on the determined pose information of the first steel plate, the side-view camera is guided to acquire a side view image of the first steel plate; and The side view image is processed to identify the edge lines corresponding to the upper and lower surfaces of the first steel plate, and the thickness information of the first steel plate is determined based on the distance between the edge lines.
[0009] Preferably, it further includes: Using the depth information of the first steel plate acquired by the side-view camera, the pixel equivalent of the top-view image is corrected, and the size and pose information of the first steel plate are recalculated; and Using the corrected size and pose information, the precise working distance from the side-view camera to the first steel plate is determined, and the pixel equivalent of the side-view camera is corrected to obtain more accurate thickness information.
[0010] Preferably, the step of identifying the edge contour characterizing the first steel plate of the top layer includes: Initially locate an initial contour and construct an energy function that includes internal and external energy; Through iterative optimization, the initial contour is deformed and evolved under the action of the energy function until it converges and precisely fits the actual edge of the first steel plate, thereby determining the edge contour of the first steel plate.
[0011] Preferably, the internal energy is used to control the continuity and smoothness of the contour, and the external energy is determined by the gradient information of the image to attract the contour to the region of high gradient value in the image.
[0012] Preferably, the step of identifying the edge contour characterizing the first steel plate of the top layer includes: The extracted edge contours are linearized to obtain a set of edge segments; The edge segments are grouped based on their orientation angles; Within each directional group, candidate edges are selected based on a scoring mechanism; and The selected candidate edges are fitted to generate the final edge profile of the first steel plate.
[0013] Preferably, the linearization process employs a Hough transform to convert the edge contour into a set of edge segments parameterized by length, orientation angle, and position information; Alternatively, the step of grouping the edge segments based on the direction angle includes: grouping edge segments with similar direction angles into the same direction group, ultimately forming two direction groups corresponding to the two sets of parallel sides of the steel plate respectively; Alternatively, the scoring mechanism may include at least one of the following metrics: the length of the edge segment, the continuity of the edge segment, and the average gradient intensity of the pixels on the edge segment.
[0014] Preferably, the preset filtering rule is a combination of multiple rules or one of them, including: First, multiple candidate contours are initially selected based on the contour area, and then the final edge contour of the first steel plate is determined from the candidate contours based on the geometric regularity. Alternatively, a selection rule based on the regularity of the contour geometry can be used: calculate the minimum bounding rectangle of each contour, determine the ratio of the area of the contour to the area of the minimum bounding rectangle, and select the contour with the largest ratio that exceeds a preset threshold as the edge contour of the first steel plate. Alternatively, based on the screening rule of contour area: select the contour with the largest area among all closed contours as the edge contour of the first steel plate.
[0015] Preferably, the step of determining the size and orientation information of the first steel plate includes: Calculate the minimum bounding rectangle of the edge profile of the first identified steel plate; Based on the length and width of the minimum bounding rectangle, and in conjunction with preset camera calibration parameters, the dimensions of the first steel plate are determined; and Based on the center point coordinates and rotation angle of the minimum bounding rectangle, the pose information of the first steel plate is determined.
[0016] The present invention also provides a computer program product comprising instructions that, when executed on a computer or processor, cause the computer or processor to perform the first steel plate identification method described in any of the preceding claims.
[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the first steel plate identification method described in any of the preceding claims. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the overall process of Embodiment 1 of this application; Figure 2 This is a flowchart illustrating step S300 of Embodiment 2 of this application; Figure 3 This is a flowchart illustrating step S300 of Embodiment 3 of this application; Figure 4 This is a flowchart illustrating Embodiment 4 of this application. Detailed Implementation
[0019] The technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. In this specification, the dimensions of the drawings do not represent the actual size ratio. The drawings are only used to illustrate the relative positional relationship and connection relationship between the components. Components with the same name or the same reference numeral represent similar or the same structure, and are limited to illustrative purposes.
[0020] The present invention aims to solve the technical problem in the prior art where the specifications, batches, and stacking states of incoming steel plates are complex and varied, making it difficult for automated unloading systems to accurately identify and locate the first (i.e., the top layer) steel plate, thus affecting hoisting efficiency and safety.
[0021] Reference Figure 1The diagram illustrates a flowchart of a first-sheet steel plate identification method according to Embodiment 1 of the present invention. This method can be applied to industrial automation control systems, particularly in vision recognition systems integrated with lifting equipment such as automated cranes or gantry cranes. The following details each step of the process. Before a vehicle loaded with steel plates enters the unloading area, the vision system needs to provide vehicle parking guidance, directing the driver to park precisely according to voice prompts and navigation areas, stopping the vehicle within the recognition system's identifiable area. During the automatic identification of the first steel plate, the vision system can identify foreign objects on the steel plate surface, such as wooden blocks, steel strips, etc., that may affect the unloading of the steel plate, and inform the host system, triggering the robot to perform pre-emptive cleaning.
[0022] To ensure the consistency and integrity of the top-view image acquired in step S100, and to avoid the randomness of vehicle parking positions causing the steel plate stack to exceed or partially fall outside the camera's field of view, thus affecting recognition accuracy, the vision system in this embodiment of the invention can integrate a vehicle parking guidance subsystem. The function of this subsystem is to precisely guide the vehicle loaded with steel plates to a preset, optimized recognition area. Specifically, when a vehicle enters the unloading area, it can be detected by a ground loop coil or image sensor deployed at the entrance, triggering the guidance process. Optionally, vehicle information can be automatically identified through a license plate recognition unit or RFID reader and associated with the production plan of the upper-level system. The guidance system uses vision sensors positioned in front of or on both sides of the unloading area to acquire the vehicle's current position and attitude in real time. Image processing algorithms calculate the lateral, longitudinal, and angular deviations between the vehicle's outline or specific markers (such as wheels) and the preset target parking position. The system converts the calculated deviation information into intuitive guidance instructions in real time and presents them to the driver through one or more human-machine interfaces. For example, a large LED display screen positioned directly in front of the driver's line of sight can graphically show the vehicle's current position, the target parking frame, and dynamically adjusting arrow indicators; alternatively, industrial projection equipment can be used to project guide lines, stop lines, and other markings directly onto the ground; or a voice broadcast system can issue clear instructions such as "Please proceed," "Correct to the left," or "In position, please stop." When the vision system determines that the vehicle's position and posture are within the preset tolerance range, the system will issue a parking completion signal and automatically proceed to the subsequent steel plate recognition process. This precise guidance step lays the foundation for the standardization and reliability of subsequent image acquisition and processing.
[0023] While performing step S300 to identify the outline of the first steel plate, the vision system of this embodiment can run a foreign object detection function in parallel. This function aims to identify and locate obstacles on the surface of the steel plate that may affect the safe adsorption of the electromagnetic chuck or lifting device, such as wooden blocks used for interlayer isolation, broken binding steel straps, or other debris. This function is an important guarantee for achieving full-process automation and improving operational safety.
[0024] This foreign object detection function utilizes the same top-view image acquired in step S100. Its technical implementation preferably employs a deep learning-based object detection algorithm. By pre-constructing an image dataset containing various common foreign object samples, a neural network model (such as YOLO, Faster R-CNN, etc.) is trained to accurately identify the type of foreign object in the image (such as "wooden pad") and output its precise location, size, and bounding box in the image coordinate system. Alternatively, traditional image processing algorithms based on differences in features such as color, texture, or shape can be used for foreign object segmentation and identification.
[0025] Once the vision system detects a foreign object, it generates data including the object's type, quantity, and precise location coordinates (which can be converted to 3D coordinates in the world or robot coordinate system using camera calibration parameters). This information is immediately sent to the upper control system. Upon receiving this information, the upper control system prioritizes the foreign object removal procedure, pausing the steel plate hoisting operation. Based on the object's location coordinates, it instructs industrial robots or dedicated cleaning devices deployed within the work area to automatically and precisely grasp and remove the foreign object from the steel plate surface. After the robot completes the cleaning and returns a confirmation signal, the upper control system resumes the hoisting process for the first steel plate. This design effectively avoids safety accidents such as hoisting failures or steel plates slipping in mid-air due to the presence of foreign objects, achieving unmanned and intelligent unloading preparation.
[0026] Step S100: Acquire a top-view image of multiple steel plates in a stacked state. In this embodiment, this step is the data input stage of the entire recognition process. The stacked state refers to the disordered or semi-ordered stacking form naturally formed during transportation or storage of the steel plates. These steel plates may vary in size, intersect, partially overlap, and have poor edge alignment. This is a typical and complex application scenario that this invention needs to address. A "top-view image" refers to an image captured when the optical axis of the image acquisition device is approximately perpendicular to the main plane of the steel plate. To achieve this, one or more high-resolution industrial cameras can be fixedly mounted directly above a hoisting device (such as the beam of a gantry crane), ensuring that its field of view completely covers the stacked steel plate area. The industrial camera can be, for example, a CCD or CMOS camera, and is typically mounted on the beam of a gantry crane.
[0027] To ensure image quality and avoid interference from shadows and reflections in subsequent processing, auxiliary light sources, such as ring LED lights or strip LED light arrays, can be deployed around the camera to provide a uniform, shadowless lighting environment. Camera selection should consider resolution, frame rate, and adaptability to industrial environments to ensure clear capture of steel plate edge details. Image acquisition can be a single-trigger shot or periodic shooting to monitor the stockpile status in real time. The acquired raw images are typically RGB color or grayscale images.
[0028] Step S200: Perform image processing on the top-view image to extract edge contours from it. The purpose of this step is to accurately separate the edge information of all objects from the original image, which may have a complex background and contain various interfering factors, laying the foundation for subsequent identification of the contour of the first steel plate. This typically involves a series of sub-steps: Step S201: Image Preprocessing. If the acquired image is color, it is first converted to grayscale. This is because the edge information of the steel plate is mainly reflected in the step changes of brightness, i.e., grayscale values. Converting to a single-channel grayscale image can significantly reduce computational complexity without losing crucial edge information. Due to the presence of dust, lighting fluctuations, or noise from the camera sensor itself in industrial environments, the original image will contain noise. To avoid these noise points being misidentified as edges, filtering is required. Algorithms such as Gaussian filtering or median filtering can be used. Gaussian filtering can effectively suppress noise that conforms to a normal distribution, while median filtering is more effective at handling impulse noise such as salt-and-pepper noise. The choice of filtering method depends on the actual noise type at the site.
[0029] Step S202: Edge Extraction. After preprocessing, the core edge extraction operation can be performed. Several mature edge detection operators exist in this field, such as the Sobel operator, Laplacian operator, or Canny operator. In this embodiment, the Canny edge detection algorithm is preferred because it has low error rate and high localization accuracy. It can identify as many real edges as possible and produce a unique response to a single edge, avoiding the generation of edges that are many pixels wide. The Canny algorithm typically includes steps such as Gaussian blurring, calculating gradient strength and direction, non-maximum suppression, double threshold detection, and edge connection. Through these steps, a binary edge image can be obtained, where white pixels represent detected edges and black pixels represent the background. At this point, the image contains the outline of the top steel plate, the exposed outline of the lower steel plate, and the outlines of other possible background objects. These outlines may be discontinuous or intertwined. After obtaining the edge image, the discrete edge points need to be connected into outlines with hierarchical relationships. Contour-finding algorithms such as the Suzuki algorithm can be used to traverse the edge image and find all independent external and internal outlines. The result is a collection containing all the contours in the image.
[0030] Step S300: Identify the edge contour of the top layer steel plate. Based on preset filtering rules, identify the edge contour representing the first steel plate at the top layer from the extracted edge contours. This is the core and innovation of the present invention. Its basic working principle is that, from a top-down view, due to occlusion, only the contour of the first steel plate at the top layer is relatively complete, closed, and unobstructed by other steel plates. The contours of lower steel plates will appear incomplete, broken, or irregular due to occlusion by upper steel plates. This step utilizes this physical and geometric principle to design filtering rules to find this unique first steel plate contour.
[0031] The preset filtering rules can be a single rule or a combination of multiple rules. Firstly, a filtering rule based on contour area can be used. Since the first steel plate is fully visible, the area enclosed by its closed contour is theoretically the largest among all contours. Therefore, all closed contours found in step S200 can be traversed, the pixel area of each contour can be calculated, and the contour with the largest area can be selected as the candidate contour for the first steel plate. This is a simple and efficient preliminary filtering method. Alternatively, a filtering rule based on the geometric regularity of the contour can be used. Steel plates are usually standard or nearly standard rectangles. Therefore, the contour of the first steel plate should be geometrically closest to a rectangle. A rectangularity score can be calculated for each contour. Specifically, for each contour, its minimum bounding rectangle can be calculated. Then, the ratio of the actual area of the contour to the area of the minimum bounding rectangle can be calculated. The closer this ratio is to 1, the closer the contour shape is to a rectangle. The contour with the largest ratio exceeding a certain preset threshold, such as 0.95, is selected as the contour of the first steel plate. This method can effectively eliminate interference from non-rectangular contours caused by reflections or stains. Finally, a selection rule based on the perimeter and closure of the outline can be used. The outline of the lower steel plate that is obscured is often broken and not closed. A rule can be set to prioritize the closed outline with the longest perimeter, indicating that the outline is more complete and straight, without large depressions or breaks.
[0032] In practical applications, the above rules can be combined to form a more robust filtering strategy. For example, the top three contours by area can be selected first, and then the one with the highest rectangularity score can be chosen from these three contours. This multi-dimensional, phased filtering can greatly improve the accuracy and anti-interference ability of the identification. By executing these filtering rules, the system can ultimately uniquely identify the edge contour representing the topmost first steel plate from among many messy edge contours.
[0033] Step S400: Calculate steel plate information. Based on the identified edge contour of the first steel plate, determine the size and pose information of the first steel plate. After successfully identifying the contour of the first steel plate, the purpose of this step is to convert the pixel-level contour information into size and pose data with actual physical meaning that can be used by automated hoisting equipment. Here is an optional technical solution. For the size information, the pixel dimensions of the length and width of the minimum bounding rectangle calculated in the previous step can be directly obtained in the image coordinate system. In order to obtain the true physical dimensions, camera calibration needs to be performed at the beginning of system deployment. By taking a picture of a standard-sized calibration board, a mapping relationship between the pixel coordinate system and the world physical coordinate system is established, and the actual physical length represented by each pixel is calculated. Multiplying the pixel length and width of the minimum bounding rectangle by this scaling factor, the actual length and width dimensions of the first steel plate can be obtained.
[0034] For determining the pose information, the pose information mainly includes the position and orientation of the steel plate. Position can be represented by the center point of the steel plate, and orientation can be represented by the rotation angle of the steel plate relative to a preset reference. Therefore, position can also be represented by the coordinates of the geometric center point of the smallest bounding rectangle. After coordinate system transformation, the pixel coordinates of this center point can be used to obtain the planar physical coordinates of the steel plate in the working space of the lifting equipment. Orientation can be represented by the rotation angle of the smallest bounding rectangle. This angle usually refers to the angle between the long side of the rectangle and a reference axis of the image coordinate system. This angle information is crucial for guiding the lifting device to rotate accordingly and align it with the long side of the steel plate. Finally, the method outputs a set of structured data, such as {length: 2500mm, width: 1200mm, center point coordinates: (X, Y), rotation angle: 0.83°}. This data is sent to the PLC or robot controller of the lifting equipment as precise instructions to adjust the position and orientation of the lifting device, thereby achieving precise, stable, and fully automated lifting operations for the first steel plate.
[0035] Example 2, based on Example 1, provides a more refined implementation of step S300. The main difference between this example and Example 1 lies in the specific method for identifying the edge contour of the first steel plate. The remaining steps, such as steps S100, S200, and S400, can utilize the technical solutions and implementation details described in Example 1, and will not be repeated here. In this example, step S300 can be specifically broken down into the following sub-steps.
[0036] Step S301: Linearize and parameterize the edge contours. In step S201, we obtained an edge image containing a large number of discrete edge points. To analyze their direction and orientation, it is first necessary to parameterize the edge segments formed by these pixels. Preferably, the Hough transform can be used. The Hough transform can map a straight line (or line segment) in the image coordinate system to a point in the parameter space, such as the ρ-θ space of the polar coordinate system. By performing peak detection in the parameter space, the straight line segments in the image composed of a large number of collinear pixels can be effectively identified. After processing by the Hough transform or a similar linear fitting algorithm, the original discrete set of edge contours is transformed into a set of structured edge segments. Each edge segment in the set has clear geometric parameters, including at least: its length, its orientation angle in the image coordinate system, and its position information.
[0037] Step S302: Group edge segments based on orientation angles. Since steel plates are typically rectangular, their four sides must belong to two mutually orthogonal directions. Utilizing this prior knowledge, this step clusters all edge segments obtained in S300-1 according to their orientation angles. Specifically, two center angles can be set, such as 0° and 90°, with a tolerance range of ±10° for each center angle. Then, all edge segments are traversed: if a segment's orientation angle is between -10° and +10°, it is assigned to the first orientation group. If a segment's orientation angle is between 80° and 100°, it is assigned to the second orientation group. Edge segments with orientation angles outside this range are considered noise or non-steel plate edges and discarded. In this way, all valid edge segments are divided into two large orientation groups, corresponding to the two sets of parallel edges of the steel plate.
[0038] Step S303: Within each direction group, candidate edges are filtered based on a multi-dimensional scoring mechanism. This is the core step of this embodiment. Its purpose is to find the best line segment that best represents the edge of the first steel plate within each direction group. Since the edge of the first steel plate is the longest, most continuous, and clearest, identification can be achieved through one of the following methods.
[0039] For all edge segments within each direction group, scores are given based on one or more of the following metrics: Length metric: The longer the pixel length of a segment, the greater the likelihood that it is the main edge of the first steel plate.
[0040] Continuity index: This detects whether there are large breaks or discontinuous areas within a line segment. Better continuity results in a higher score. If a line segment is incomplete or has numerous truncations, it is unlikely to be the edge of the first steel plate. Line segments truncated by the edge of the lower steel plate usually have edges in other directions near their endpoints. Whether a segment is truncated can be determined by analyzing the edge complexity of the endpoint's neighborhood; untruncated line segments score higher.
[0041] Gradient variation: Calculate the average gradient intensity of all pixels on the line segment along its normal direction. A larger and more stable gradient intensity indicates a clearer and more realistic edge, rather than a shadow or blurry boundary.
[0042] In the first and second directional groups, select the two edge segments with the highest overall scores. This gives us four candidate edge segments that are most likely to form the four edges of the first steel plate.
[0043] Step S304: Fitting and Validation, Generating the Final Contour. The four candidate edge segments selected in the previous step are geometrically fitted to form a closed quadrilateral, preferably a rectangle. This can be done by calculating the intersection points of these four segments or by using the least squares method to fit a rectangle that best matches these four segments. This fitted rectangle is ultimately determined as the "edge contour representing the first steel plate of the top layer." Thus, through the above-mentioned refinement process, this embodiment has also completed the accurate identification of the edge contour of the first steel plate, and the identification result can be directly used for the size and pose calculation in the subsequent step S400.
[0044] In practical applications, steel plates may undulate due to the presence of padding and other factors during placement and transportation, resulting in edges that are not straight in the image. To address this practical situation, we also provide a more adaptable implementation method for step S300. The core technology of this embodiment lies in using a deformable model to accurately fit the edges. Steps S101, S201, and S400 of this embodiment can follow the technical content described in Embodiment 1, and will not be repeated here.
[0045] In Example 3, step S300, which involves identifying the edge contour representing the top layer of the first steel plate from the extracted edge contours based on preset filtering rules, can be implemented through the following sub-steps: Step S311: Coarse localization of the initial contour. First, using the screening method based on contour area or geometric regularity described in Example 1, the most likely candidate contour is selected from the numerous edge contours extracted in Step S201 as the initial contour for iterative optimization. For example, the closed contour with the largest area is selected. The purpose of this step is to provide a good initial position for subsequent accurate fitting and to avoid the model getting trapped in erroneous local optima. This initial contour does not need to be very precise, but should be roughly located near the edge of the first steel plate.
[0046] Step S312: Construction of the Energy Function. This step introduces an active contour model, which guides the initial contour deformation and ultimately fits precisely to the target edge by defining an energy function. This energy function typically includes two parts: internal energy and external energy. Internal energy controls the properties of the contour itself. It consists of two terms: elasticity, which maintains the continuity of the contour and makes it tend to contract; and rigidity, which controls the smoothness of the contour and suppresses sharp corners or excessive bending. The presence of internal energy ensures that even if there are small breaks or noise at the edge of the steel plate in the image, the final fitted contour will still be smooth and continuous. External energy, also known as image energy, links the contour to the features of the image. In this embodiment, external energy is mainly determined by the gradient information of the image. In the edge image calculated in step S201, the larger the gradient value, the higher the probability that it is the edge of the object, and the lower the corresponding image energy value. Therefore, the role of external energy is to generate a driving force to attract the contour curve to the region with a high gradient value in the image, i.e., the real edge of the steel plate.
[0047] Step S313: Iterative Evolution and Convergence of the Contour. The initial contour is placed in an energy field, where it begins to move and deform under the combined influence of internal and external energy. This process is an iterative optimization; in each iteration, each point on the contour moves in the direction that reduces the total energy. The internal energy keeps it smooth and continuous, while the external energy pulls it towards the true edge of the image. After multiple iterations, when the total energy of the entire contour reaches a local minimum, the contour no longer changes significantly, i.e., it has reached convergence. At this point, the stabilized contour curve is the accurate fit to the actual edge of the first steel plate, including its slightly curved portions.
[0048] Step S314: Determination of the final profile. After the profile converges, its final shape and position are determined as the edge profile representing the top layer of the first steel plate. This profile accurately reflects the shape of the steel plate edge, even if it is not a standard straight line. This profile can be directly used in subsequent step S400. When calculating the size and pose information, the minimum bounding rectangle of the converged profile can be calculated, and the geometric parameters of this rectangle can be used as the equivalent size and pose information of the first steel plate.
[0049] Through the above steps, this embodiment can effectively overcome the interference caused by the non-linear deformation of the steel plate edge, and achieve more accurate and robust recognition of the outline of the first steel plate.
[0050] Finally, this solution provides Embodiment Four, which, based on Embodiments One, Two, and Three, further provides an enhanced technical solution integrating thickness recognition and depth information correction functions. This solution adds a side-view vision system to the original top-view recognition system, thereby achieving comprehensive acquisition of the three-dimensional information of the first steel plate.
[0051] The overall method flow of this embodiment executes steps S100 to S400 of any of the preceding embodiments to determine the two-dimensional pose information of the first steel plate in the top view plane, namely its center coordinates X and Y values and rotation angle. Subsequently, the following steps added in this embodiment are executed. The improvement in hardware structure of this embodiment lies in the additional installation of one or more side-view cameras on the lifting device's lifting gear or its nearby structure. The optical axis of the side-view camera is approximately parallel to the horizontal plane of the steel plate stack, and its lens faces the side of the steel plate stack. The newly added technical steps in this embodiment are described in detail below: Step S500: Acquire a side view image of the first steel plate. After completing the top-view recognition, the control system drives the hoisting equipment to move based on the obtained two-dimensional pose information of the first steel plate. The equipment will move to a predetermined position so that the side-view camera can clearly capture the edge and end face of the first steel plate. At this time, the side-view camera is triggered to take pictures, acquiring one or more side view images containing the side outline of the stacked steel plates.
[0052] Step S600: Identify the top edge and determine thickness information from the side view image. The acquired side view image is processed. Due to the clear horizontal gaps between the steel plates, two strong horizontal edge lines are formed on the upper and lower surfaces of each steel plate in the side view image. These horizontal edges can be extracted using edge detection algorithms, such as the Canny operator. The system identifies the topmost horizontal edge line in the image and identifies it as the upper surface of the first steel plate. Subsequently, it searches for the nearest next horizontal edge line below this edge line and identifies it as the lower surface of the first steel plate. Using the pixel equivalent of the side view camera obtained from camera calibration, i.e., the actual physical size represented by each pixel, the pixel distance in the vertical direction between the two horizontal edge lines is calculated and converted into an actual physical distance. This physical distance is the thickness information of the first steel plate. This thickness information can be transmitted to the control system of the hoisting equipment to accurately estimate the weight of the steel plate in conjunction with the known steel plate density, thereby setting an appropriate hoisting power and effectively avoiding the phenomenon of lower-layer adhered steel plates being hoisted together due to excessive hoisting force.
[0053] Example 5 provides a collaborative measurement S700 step for achieving high-precision 3D information acquisition, which can serve as a further optimization or alternative to the 3D information acquisition and correction steps in Example 4. The core of this example lies in utilizing information between the top-view vision system and the side-view vision system for bidirectional mutual correction to eliminate measurement errors introduced by the uncertainty of their respective working distances, thereby obtaining complete and accurate 3D dimension and pose information of the first steel plate. The prerequisite for executing this example is that the initial 2D information of the first steel plate in the top-view image has been obtained through the aforementioned examples, and the side-view image has been obtained through the hardware configuration of Example 4. The specific steps are as follows: Step S701: Preliminary guidance and depth information acquisition based on top-view information. The planar center coordinates (X and Y coordinates) of the first steel plate, initially identified by the top-view vision system, are used to control the movement of the hoisting equipment, ensuring the side-view camera is roughly aligned with the center area of the steel plate's side. After the side-view camera acquires images, image processing identifies the upper surface edge line of the first steel plate. Since the vertical installation position of the side-view camera on the equipment is known, the system calculates the initial height of the upper surface of the first steel plate, i.e., the depth information (Z coordinate), based on the position of this upper surface edge line in the image.
[0054] Step S702: Perform the first correction on the top-view measurement results using depth information. The initial depth information Z-coordinate obtained in step S701a is fed back into the measurement model of the top-view vision system. Based on this actual working distance, the top-view vision system updates its image pixel equivalent, i.e., the actual physical size represented by a unit pixel. Subsequently, the system uses the updated pixel equivalent to recalculate the top-view profile of the first steel plate, thereby obtaining a set of highly accurate length, width, and plane center coordinates X and Y after the first correction.
[0055] Step S703: Correct the side-view measurement results using the corrected top-view information. After the first correction, the top-view vision system outputs steel plate contour and position information with high planar accuracy. Based on this precise planar contour, the system can accurately calculate the precise position of the steel plate's side edge in the world coordinate system. Combining this with the known precise spatial position of the side-view camera on the hoisting equipment, the system can calculate the precise horizontal distance from the optical center of the side-view camera to the side of the steel plate. This precise horizontal distance is used as the true working distance of the side-view vision system to correct its image pixel equivalent. Previously, the pixel equivalent used when measuring thickness with the side-view camera might have been based on an estimated or inaccurate horizontal distance. Through this correction step, the measurement model of the side-view vision system is optimized.
[0056] Step S704: Output high-precision 3D information. The vision system uses the corrected pixel equivalent to recalculate the acquired side view image to obtain a high-precision first steel plate thickness value.
[0057] Thus, by using top-view information to guide side-view, side-view information to correct top-view, and the corrected top-view information to correct side-view—a complete closed-loop process of mutual correction—the system ultimately obtained high-precision information on the length, width, thickness, and three-dimensional spatial pose of the first steel plate. The entire measurement process is free from dependence on preset working distances, achieving adaptive optimization of measurement accuracy.
[0058] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0059] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0060] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0061] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0062] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Any modifications and improvements made by those skilled in the art to the technical solutions of this application without departing from the spirit of this application shall fall within the protection scope defined by the claims of this application.
Claims
1. A method for identifying the first steel plate, characterized in that, Includes the following steps: Obtain a top view image of multiple steel plates stacked together; Image processing is performed on the top view image to extract the edge contours in the top view image; Based on preset filtering rules, the edge contour representing the topmost first steel plate is identified from the extracted edge contours. Based on the identified edge contour of the first steel plate, the size and orientation information of the first steel plate are determined.
2. The method according to claim 1, characterized in that, It also includes the following steps: Based on the determined pose information of the first steel plate, guide the side-view camera to acquire a side view image of the first steel plate; as well as The side view image is processed to identify the edge lines corresponding to the upper and lower surfaces of the first steel plate, and the thickness information of the first steel plate is determined based on the distance between the edge lines.
3. The method according to claim 2, characterized in that, Further includes: Using the depth information of the first steel plate obtained by the side-view camera, the pixel equivalent of the top-view image is corrected, and the size and pose information of the first steel plate are recalculated. as well as Using the corrected size and pose information, the precise working distance from the side-view camera to the first steel plate is determined, and the pixel equivalent of the side-view camera is corrected to obtain more accurate thickness information.
4. The method according to claim 1, characterized in that, The step of identifying the edge profile that characterizes the topmost steel plate includes: Initially locate an initial contour and construct an energy function that includes internal and external energy; Through iterative optimization, the initial contour is deformed and evolved under the action of the energy function until it converges and precisely fits the actual edge of the first steel plate, thereby determining the edge contour of the first steel plate.
5. The method according to claim 4, characterized in that, The internal energy is used to control the continuity and smoothness of the contour, while the external energy is determined by the gradient information of the image to attract the contour to regions with high gradient values in the image.
6. The method according to claim 1, characterized in that, The step of identifying the edge profile that characterizes the topmost steel plate includes: The extracted edge contours are linearized to obtain a set of edge segments; The edge segments are grouped based on their orientation angles; Within each directional group, candidate edges are selected based on a scoring mechanism; and The selected candidate edges are fitted to generate the final edge profile of the first steel plate.
7. The method according to claim 6, characterized in that, The linearization process employs the Hough transform to convert the edge contour into a set of edge segments parameterized by length, orientation angle, and position information; Alternatively, the step of grouping the edge segments based on the direction angle includes: grouping edge segments with similar direction angles into the same direction group, ultimately forming two direction groups corresponding to the two sets of parallel sides of the steel plate respectively; Alternatively, the scoring mechanism may include at least one of the following metrics: the length of the edge segment, the continuity of the edge segment, and the average gradient intensity of the pixels on the edge segment.
8. The method according to claim 1, characterized in that, The preset filtering rules are a combination of multiple rules or one of them, including: First, multiple candidate contours are initially selected based on the contour area, and then the final edge contour of the first steel plate is determined from the candidate contours based on the geometric regularity. Alternatively, a selection rule based on the regularity of the contour geometry can be used: calculate the minimum bounding rectangle of each contour, determine the ratio of the area of the contour to the area of the minimum bounding rectangle, and select the contour with the largest ratio that exceeds a preset threshold as the edge contour of the first steel plate. Alternatively, based on the screening rule of contour area: select the contour with the largest area among all closed contours as the edge contour of the first steel plate.
9. The method according to claim 1, characterized in that, The steps for determining the size and orientation information of the first steel plate include: Calculate the minimum bounding rectangle of the edge profile of the first identified steel plate; Based on the length and width of the minimum bounding rectangle, and in conjunction with preset camera calibration parameters, the dimensions of the first steel plate are determined; and Based on the center point coordinates and rotation angle of the minimum bounding rectangle, the pose information of the first steel plate is determined.
10. A computer program product comprising instructions that, when executed on a computer or processor, cause the computer or processor to perform the method of any one of claims 1-17.
11. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1-17.
