Intelligent grading equipment for plate tailings based on visual guidance

Abstract

The application provides a plate tail intelligent grading equipment and method based on visual guidance, and the core is that through integration of multi-view visual perception, adaptive grabbing, digital twin and cloud-edge collaborative control technology, full-process intelligent management of plate tail from warehousing, grading storage to on-demand use is realized; multi-view images of the tail are collected by a camera array, three-dimensional point clouds are reconstructed through a stereo matching algorithm, and morphological characteristic parameters of the tail are extracted; with the help of digital twin technology, a virtual model of each tail is generated and synchronized, a stacking strategy is optimized based on a genetic algorithm, and space utilization is maximized; when a use request is received, the system can perform virtual cutting pre-performance in the digital twin environment, accurately match the demand, and use the stacked tail through the temporary storage-backfilling mechanism; finally, through the cloud-edge collaborative platform, visual monitoring and global scheduling optimization of the inventory and equipment state are realized, and the tail management efficiency and material utilization are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of waste material grading technology, specifically to an intelligent grading device for waste board materials based on vision guidance. Background Technology

[0002] In the field of sheet metal processing, the rational utilization of sheet metal is a key aspect of cost control. However, the sheet metal cutting process generates a large amount of waste material of varying sizes, thicknesses, and shapes, posing significant challenges to the subsequent management and reuse of this waste material.

[0003] Currently, traditional processing methods mainly rely on manual sorting and stacking, which is inefficient, has a high error rate, and the tailings are irregular in thickness, size and shape, resulting in messy stacking and inconvenient access, causing resource waste and increased production costs. With the development of industrial automation technology, existing technologies attempt to improve efficiency through mechanical sorting systems or visual inspection methods, but there are still many limitations.

[0004] Chinese Patent Application No. CN202311498071.4 discloses a visual inspection and conveying device and method for grading sheet materials. The device includes a grading and dropping device and a detection device. The grading and dropping device includes at least one grading stage, each stage including a first support and a grading conveyor, with a grading lifting mechanism fixed to the upper support. The detection device includes a main control device and a first position detection device. If the sheet material belongs to the grade of the grading conveyor it is about to enter, the first position detection device transmits a signal to the main control device. The main control device controls the front end of the grading conveyor, driven upwards by a fifth rotating shaft, to drop the sheet material to the bottom of the grading conveyor. If the sheet material does not belong to the grade of the grading conveyor it is about to enter, the sheet material is conveyed backwards on the upper surface of the grading conveyor. This device, through the cooperation of the grading and dropping device and the detection device, achieves rapid grading and conveying of sheet materials, improving operational efficiency and reducing costs.

[0005] While the aforementioned solution achieves rapid grading and conveying of boards through the coordinated operation of the grading and dropping device and the detection device, in practical applications, this grading visual inspection and conveying equipment is only suitable for processing standardized boards. For grading and processing irregularly shaped boards with large dimensional differences, this equipment has significant shortcomings.

[0006] While some existing automated sorting technologies have indeed improved sorting efficiency to some extent, most of these technologies focus on defect detection or physical property grading of regular boards. Their sorting logic is based on relatively uniform specifications and clear flow paths. However, the shape of leftover board materials is extremely irregular and the stacking state is very complex, which makes it difficult to directly apply existing automated sorting technologies. They cannot effectively obtain and record the specific spatial coordinates and three-dimensional shape information of each piece of leftover material. As a result, after the leftover materials are put into storage, an information-deficient "black hole" is formed.

[0007] Looking further ahead, the ultimate goal of waste material management is to achieve secondary utilization. However, the current management model lacks digital management of waste material information and the ability to match the value of waste materials with downstream demand. Users find it difficult to quickly find materials that meet specific cutting size and thickness requirements from a large amount of waste material, and they cannot conduct virtual cutting and layout simulations before using waste materials to assess their applicability and yield. As a result, the reuse process of waste materials still relies on manual experience and repeated searching, which is very inefficient. This not only hinders the full exploitation of the value of waste materials, but also affects the overall utilization efficiency of the boards. Summary of the Invention

[0008] The purpose of this invention is to provide a vision-guided intelligent grading device for leftover sheet materials, which aims to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A vision-guided intelligent grading method for leftover board materials includes the following steps:

[0011] S100. Multi-source image data of disordered tailings located in the resettlement area are collected by a multi-view visual sensing unit deployed on the crossbeam of the gantry frame. Based on the multi-source image data, the morphological feature parameters of the tailings are extracted by a three-dimensional point cloud reconstruction and contour recognition algorithm, and its effective usable area is calculated.

[0012] S200: Based on the extracted tailings feature data, construct a digital twin model that is synchronized with the physical stacking site in real time, and generate a corresponding three-dimensional digital model for each piece of tailings, recording its spatial coordinates, attribute parameters and stacking status.

[0013] S300: Based on preset grading rules and stacking strategies, target stacking grids are assigned to tailings, and the motion path of the hoisting equipment is generated; the hoisting equipment grabs the tailings according to the motion path and transfers them to the target stacking grids.

[0014] S400: When a request to retrieve tailings is received, tailings matching and virtual cutting simulation are performed based on the digital twin model to generate a retrieval plan; if the target tailings are stacked, a temporary storage-backfilling mechanism is activated, and the hoisting equipment removes the tailings above in sequence, and the stacking order is restored after the target tailings are retrieved.

[0015] Preferably, in step S100, the multi-view vision sensing unit uses multiple spatially distributed industrial cameras to synchronously acquire images; the three-dimensional point cloud reconstruction uses a stereo matching algorithm and uses contour recognition technology to filter the acute angle region of the tail material edge in order to calculate its effective usable area.

[0016] Preferably, in step S200, the digital twin model dynamically simulates the three-dimensional layout of tailings in the stacking yard through real-time data driving; the stacking strategy optimizes the spatial arrangement of tailings in the stacking grid based on a genetic algorithm to maximize the stacking density.

[0017] Preferably, in step S400, the virtual cutting simulation is based on the three-dimensional model of the tail material and the processing dimensions input by the user, to simulate the layout scheme and calculate the material utilization rate; the temporary storage-backfilling mechanism includes temporarily transferring the stacked tail material to the placement area, and backfilling it into the corresponding stacking grid in the original order after the target tail material is taken out.

[0018] Preferably, it also includes S500, which integrates multi-source data through a cloud-edge collaborative control platform to achieve visualized monitoring of tailings inventory, equipment status and usage records, and provides tailings information query and scheduling interfaces for downstream production systems.

[0019] The cloud-edge collaborative control platform integrates edge computing nodes and cloud data centers. The edge nodes are responsible for real-time control of gantry movement and visual data processing, while the cloud platform aggregates data from multiple storage yards and optimizes global inventory scheduling strategies through machine learning algorithms.

[0020] The present invention also provides a vision-guided intelligent grading device for leftover sheet materials, comprising:

[0021] The multi-source visual perception unit includes a multi-view visual sensing unit installed on the crossbeam of the gantry frame, which is used to acquire images of tailings and extract feature parameters.

[0022] The digital twin collaborative control unit includes a virtual model building module, a dynamic scheduling decision module, and a virtual pre-simulation module, which are used to build digital twin models, optimize stacking strategies, and support cutting pre-simulation.

[0023] The gantry execution unit includes a gantry that can move along a slide rail, lifting equipment, and an adaptive end effector, used to perform grabbing, handling, and stacking operations of tail material;

[0024] The storage yard management unit includes a physical storage yard divided into a placement area, a storage area, and a conveying area. The storage area is further subdivided into several storage compartments according to the attributes of the waste materials for the classified storage of waste materials.

[0025] The cloud-edge collaborative control platform is used to integrate data from multiple units to achieve collaborative control and visual monitoring of devices.

[0026] Preferably, the multi-view vision sensing unit in the multi-source vision perception unit adopts a spatially distributed camera array, and generates a three-dimensional point cloud model of the tailings through a stereo matching algorithm.

[0027] Preferably, the dynamic scheduling decision module of the digital twin collaborative control unit integrates priority management logic, which automatically generates a temporary storage-backfill path when the target tail material is stacked; the virtual pre-simulation module supports users to perform interactive cutting and layout simulation on the three-dimensional model of the tail material.

[0028] Preferably, the cloud-edge collaborative control platform further includes a predictive maintenance module, which trains a fault prediction model based on equipment operation data to achieve early warning of anomalies in key components.

[0029] Preferably, the equipment is also integrated with the upstream production management system to realize the automatic matching of waste material inventory information with production orders; when a downstream order is triggered, the system automatically recommends waste materials that meet the size requirements and generates a virtual cutting report.

[0030] The technical effects and advantages of this invention are as follows:

[0031] This invention constructs a digital twin model that is synchronized in real time with the physical storage area. It generates a corresponding three-dimensional digital model for each piece of tailings and records its coordinates, attributes, and status. It dynamically assigns target stacking grids and plans movement paths for the tailings, thereby significantly improving the space utilization and stacking order of the storage area and realizing the digital and refined management of tailings inventory. When a retrieval request is received, the system can match tailings according to the demand in the digital twin model and allow users to perform virtual cutting simulations on the three-dimensional model and calculate material utilization in real time. For the target tailings that are stacked, the system will activate a temporary storage-backfilling mechanism. The gantry will automatically remove the tailings above, retrieve the target, and then backfill in the original order. This not only improves the accuracy and efficiency of material retrieval but also minimizes material waste and manual intervention. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the system architecture of the present invention;

[0033] Figure 2 This is a schematic diagram of the main operating logic of the present invention;

[0034] Figure 3 This is a schematic diagram of the main structure of the grading device of the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of the storage yard management unit of the present invention.

[0036] In the picture:

[0037] 1. Gantry frame; 2. Lifting equipment; 3. Adaptive end effector; 4. Slide rail. Detailed Implementation

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0039] Example 1

[0040] Reference Figures 1 to 4 As shown, this embodiment provides a vision-guided intelligent grading device for sheet metal waste, including a placement area, a stacking area, and a conveying area. The stacking area is further subdivided into several regular stacking grids based on the thickness, size, and shape characteristics of the waste material, allowing the hoisting equipment to move the sheet metal waste back and forth on the crossbeam. Within the stacking grids...

[0041] The gantry execution unit includes slide rails 4 on both sides of the stacking yard, a gantry 1 on the slide rails 4, and a hoisting device 2 and an adaptive end effector 3 on the crossbeam of the gantry 1 for performing the grabbing, handling and stacking operations of the tail material.

[0042] The hoisting equipment 2 is driven by a servo motor, which allows it to move back and forth on the crossbeam so that the tail material can be placed in the corresponding stacking grid by the adaptive end effector 3.

[0043] Among them, the crossbeam of the gantry frame 1 is also equipped with a multi-source visual perception unit, which includes a multi-view visual sensing unit installed on the crossbeam of the gantry frame 1, used to collect images of tail material and extract feature parameters.

[0044] The multi-view vision sensing unit uses a spatially distributed camera array, which means that multiple cameras are arranged in space in a non-coplanar or non-collinear manner to simultaneously capture image data of the target tailings from different perspectives.

[0045] In actual use, the messy tailings are placed in the placement area. When grading the tailings of the boards piled in the placement area, the gantry frame 1 first moves above the placement area. Through the multi-view vision sensing unit in the multi-source vision perception unit, the spatially distributed camera array can capture the image information of the disordered tailings from multiple angles. These multi-view images are then input into the stereo matching algorithm. The stereo matching algorithm calculates the disparity of corresponding points in the image and reconstructs a high-precision three-dimensional point cloud model of the tailings based on this. The three-dimensional point cloud model contains the geometric shape, size and thickness information of the tailings, as well as their posture in space.

[0046] It should be noted that in this embodiment, the multi-view vision sensing unit in the multi-source vision perception unit consists of three industrial cameras installed under the crossbeam of the gantry frame 1. The three cameras are arranged in a triangular shape, with one camera located in the center and the other two located on either side and slightly behind it, to form an observation area with a good overlapping field of view. When the tailings enter the placement area, these three cameras simultaneously acquire multi-view images of the tailings. These image data are then sent to the edge computing node and processed by a stereo matching algorithm to generate a three-dimensional point cloud model of the tailings.

[0047] The constructed 3D information is transmitted to the adaptive end effector 3 in the gantry execution unit. In this embodiment, the adaptive end effector 3 adopts a magnetic or pneumatic suction structure. After receiving the 3D point cloud model of the tail material, it analyzes the center of gravity, edge contour, and surface characteristics of the tail material, and dynamically adjusts the adsorption points of its internal magnetic suction unit accordingly. For example, for irregularly shaped tail materials, the system calculates the optimal adsorption area, drives the magnetic suction unit or pneumatic suction unit to move to these areas and activates them, thereby ensuring the stability and balance of the gripping process and avoiding tail material tilting, slipping, or damage due to improper gripping.

[0048] It should be noted that the adaptive end effector 3 in this embodiment consists of an electromagnetic chuck array or a pneumatic chuck array. Each electromagnetic chuck or pneumatic chuck is mounted on an independent XY guide rail and controlled by an independent driver. After receiving the three-dimensional point cloud model of the tail material, the control system will perform gripping path planning and adsorption point optimization based on the model. For example, for a tail material identified as L-shaped, the system will calculate the optimal adsorption area of ​​its two arms, and then drive some of the chucks in the electromagnetic chuck array to move to the two arms of the L-shape, selectively activating these chucks while closing the chucks located outside the L-shape to ensure stable gripping and no waste of suction power.

[0049] After the hoisting equipment 2 and the adaptive end effector 3 lift the tail material, the tail material is placed in the pre-defined stacking grid by moving the gantry frame 1.

[0050] It is worth noting that this embodiment also includes a digital twin collaborative control unit, which comprises a virtual model construction module, a dynamic scheduling decision module, and a virtual pre-simulation module. These modules are used to construct digital twin models, optimize stacking strategies, and support cutting pre-simulation. The virtual model construction module generates a 3D digital model for each piece of tailings based on extracted feature parameters and synchronizes it in real-time with the physical stacking area, fully recording spatial coordinates, attribute parameters, and stacking status. The dynamic scheduling decision module allocates target stacking grids to tailings according to preset grading rules (such as classifying grades by thickness and effective usable area) and stacking strategies (such as prioritizing filling gaps in regularly shaped tailings). The virtual pre-simulation module, upon receiving a retrieval request, simulates the cutting and layout process in the digital twin environment to evaluate material utilization, thereby overcoming the inefficient traditional management model that relies on repeated manual searches and achieving precise matching of tailings value with downstream demand.

[0051] During use, the thickness, size, and shape of the board scrap detected by the vision inspection device are transmitted into the established stacking model. The stacking model adopts digital twin model technology. The vision inspection device establishes a virtual stacking field in the stacking model and simulates the stacking state of the board scrap in the stacking area. The stacking model includes the establishment of the virtual stacking field, the coordinate distribution of different scraps (i.e., in which stacking grid the board scrap is distributed and on which layer of the stacking grid), and the three-dimensional model corresponding to different scraps.

[0052] When it is necessary to re-cut and reuse the leftover sheet material, the required dimensions and thickness are input into the stacking model. The model then matches the data input by the vision inspection device with these dimensions and filters out multiple corresponding leftover sheet material data for the user to choose from. The selected leftover sheet material data is a visual 3D model detected by the vision inspection device. The customer can pre-set the material shape and data to be cut on this 3D model to determine whether the leftover sheet material meets the requirements. If it does, the coordinate position of the leftover sheet material in the stacking yard is retrieved, and the leftover sheet material is taken out by the gantry crane 1. If it does not meet the requirements, the user can perform secondary processing on another 3D model of the leftover sheet material to determine whether the leftover sheet material meets the requirements.

[0053] When the user drives the retrieval of the sheet material, the gantry 1 moves to retrieve the material according to the obtained coordinate position. During this process, if there are other tail materials stacked on top of the target tail material, the hoisting equipment 2 first moves the tail materials stacked on top out in sequence and temporarily stores them in the placement area. Then, it takes away the target tail material and transports it to the conveying area. After that, the tail material temporarily stored in the placement area is put back into its original stacking grid.

[0054] Example 2

[0055] While Example 1 achieves automated grading, warehousing, and digital management of leftover board materials through integrated hardware, it still suffers from insufficient process flexibility and inadequate response to different production rhythms and customized retrieval needs in practical applications. Therefore, based on Example 1, a vision-guided intelligent grading method for leftover board materials is proposed, as follows:

[0056] Reference Figures 1 to 4 As shown in the figure, this embodiment proposes a vision-guided intelligent grading method for leftover board materials, including the following steps:

[0057] S100. Multi-source image data of disordered tailings located in the resettlement area are collected by a multi-view visual sensing unit deployed on the crossbeam of the gantry frame 1. Based on the multi-source image data, the morphological feature parameters of the tailings are extracted by a three-dimensional point cloud reconstruction and contour recognition algorithm, and its effective usable area is calculated.

[0058] It should be noted that in this step, the multi-view vision sensing unit uses multiple spatially distributed industrial cameras to synchronously acquire images; the three-dimensional point cloud reconstruction uses a stereo matching algorithm and uses contour recognition technology to filter the sharp-angled areas of the tail material edge in order to calculate its effective usable area.

[0059] In this embodiment, the multi-view vision sensing unit is typically composed of multiple industrial cameras. These cameras are installed on the crossbeam of the gantry 1 in a spatially distributed manner. In this embodiment, the multiple industrial cameras are arranged in a triangular pattern on the crossbeam of the gantry 1, so that they can simultaneously acquire images of disordered tailings in the placement area from different angles such as directly above, left front side, and right front side, to ensure that visual information of each surface of the tailings can be captured and reduce visual blind spots caused by irregular shapes or stacking of objects.

[0060] In this embodiment, image acquisition can be synchronized through hardware trigger signals to ensure that all cameras are exposed within an extremely short time window or even at the same moment. Specifically, the system sends a TTL level pulse signal to the camera and connects to the external trigger port of all industrial cameras through trigger cables to ensure that all cameras are exposed simultaneously within nanosecond-level time accuracy, thereby acquiring multi-source image data of the tailings at a certain instant, providing a basis for the establishment of a three-dimensional model of the tailings.

[0061] After acquiring multi-source image data, the system calls a stereo matching algorithm (such as the semi-global matching SGM algorithm) to process images from different perspectives. By finding the corresponding pixels of the same physical point in different images and combining the camera's intrinsic and extrinsic parameters, the system uses the principle of triangulation to calculate the coordinates of these points in three-dimensional space, thereby generating a dense three-dimensional point cloud model of the tailings.

[0062] After 3D point cloud reconstruction, the edges of the scrap material may contain sharp protrusions or depressions formed by cutting processes or wear. These areas are often difficult to utilize effectively in actual processing and may even affect subsequent cutting and layout. Therefore, after 3D point cloud reconstruction, the system processes the point cloud model using a contour recognition algorithm. That is, it uses contour recognition technology to analyze the 3D point cloud or its 2D projected contour of the scrap material to identify these irregular, excessively small, or excessively sharp edge parts. Specifically, a 2D projected contour is generated based on the scrap material point cloud. Sharp angles are identified by calculating the contour curvature, analyzing the included angle between adjacent edge segments, or checking the vertex angle after polygon approximation. For example, if an interior angle on the scrap material is less than 30 degrees and the lengths of its two adjacent sides are both less than 5 millimeters, the area is determined to be an acute angle area and marked as invalid.

[0063] Once these acute-angled areas are identified, the system will remove them from the effective geometric model of the tailings to ensure that the effective usable area calculated later is more in line with actual processing requirements.

[0064] After filtering out sharp-angled regions, the system projects the simplified 3D point cloud data onto the best-fit plane and then calculates the area of ​​the projected contour. Calculation methods may include, but are not limited to: finding the smallest bounding rectangle or the smallest bounding convex polygon and calculating its area; or, considering actual processing constraints, finding the largest inscribed rectangle or inscribed polygon and calculating its area. This calculation method can more accurately reflect the actual utilization value of waste materials, providing reliable data support for subsequent digital twin modeling, grading, and scheduling.

[0065] S200: Based on the extracted tailings feature data, construct a digital twin model that is synchronized with the physical stacking site in real time, and generate a corresponding three-dimensional digital model for each piece of tailings, recording its spatial coordinates, attribute parameters and stacking status.

[0066] Specifically, in this embodiment, when constructing a digital twin model that is synchronized with the physical stacking site in real time, the system first receives tailings feature data extracted from the multi-source visual perception unit. This data includes key attribute parameters such as the precise geometric dimensions, thickness, contour shape, and effective usable area of ​​each tailings piece calculated through 3D point cloud reconstruction and contour recognition algorithms. This tailings feature data is transmitted to the digital twin collaborative control unit, which uses this data to generate a high-fidelity 3D digital model for each independent tailings piece in virtual space. It should be noted that this 3D digital model is not a simple geometric shape replacement, but strives to accurately reflect the irregular shape and volume of the tailings. Its generation process may involve surface reconstruction and meshing of point cloud data to ensure that the model and the physical entity are highly consistent in geometry.

[0067] While generating a 3D digital model of each piece of tailings, the system assigns and associates a series of attribute parameters with it. These attribute parameters include the physical dimensions, thickness, surface area, and effective usable area directly measured by the vision system. At the same time, the spatial coordinates of the tailings in the physical stacking area are also recorded in real time and bound to its corresponding 3D digital model. In addition, the current stacking status of the tailings, such as whether it is in a usable state, whether it is stacked with other tailings, and whether it is the tailings at the top of the stack, are also marked and updated as dynamic attributes in the digital twin model.

[0068] It is important to note that the spatial coordinates of the tailings are calculated by fusing the results of the gantry positioning system and the visual positioning system. Specifically, the gantry positioning system provides the absolute position coordinates of the hoisting equipment 2 in the global coordinate system of the stacking yard through feedback from its track encoder and servo motor. The visual positioning system, on the other hand, processes the images acquired by the multi-view visual sensing unit installed on the gantry 1, identifies the outline of the tailings, and calculates its precise pose relative to the hoisting equipment 2. The system fuses and registers the relative coordinates obtained by the visual system with the absolute reference coordinates provided by the gantry 1 through coordinate transformation, thereby determining the specific three-dimensional coordinates (X, Y, Z) of each tailing in the coordinate system of the stacking yard, its layer position in the stacking grid, and its horizontal rotation attitude (such as the placement angle).

[0069] To achieve real-time synchronization with the physical storage area, the digital twin model maintains a connection with the physical world through a continuously running data interaction layer. The weight sensor network deployed in the storage area to verify the storage status, along with motion feedback data from the gantry execution unit, is input into the digital twin collaborative control unit along with periodically updated data from the vision system. When the lifting equipment 2 on gantry 1 performs tailings stacking, relocation, or retrieval operations, the execution results of its action commands are fed back to the digital twin model. The model dynamically updates the spatial coordinates and stacking status of the relevant tailings' 3D digital model based on this feedback data. For example, when a piece of tailings is moved to a new stacking grid or covered by another piece of tailings, its position and status in virtual space change accordingly, ensuring consistency between the virtual and physical environments.

[0070] Ultimately, all these spatial coordinates, attribute parameters, and dynamically changing stacking status data are integrated and recorded in the database or data model associated with the digital twin model, so that the historical trajectory, current status, and availability information of each piece of tailings from its entry into the warehouse for grading to its final use can be queried.

[0071] It should be noted that the digital twin model described in this step is driven by real-time data to dynamically simulate the three-dimensional layout of tailings in the stockpile; the stockpile strategy is based on a genetic algorithm to optimize the spatial arrangement of tailings in the stockpile grid in order to maximize the stockpile density.

[0072] Specifically, the system first relies on multi-source image data of the tailings collected by multi-view visual sensing units deployed on the crossbeam of gantry 1, as well as real-time information from various sensors (such as pressure sensors and RFID) that may be installed within the storage yard. This data is transmitted via an IoT platform or the Industrial Internet, forming a real-time data stream that continuously drives the updates of the virtual model. This allows the digital twin model to reproduce the static geometry of the tailings and the storage yard. Simultaneously, by integrating physical attributes (such as tailings material and weight), behavioral rules (such as stacking stability constraints), and real-time status (such as the current spatial coordinates of the tailings and whether they are stacked), it achieves dynamic simulation of the operating logic of the physical world. For example, when the lifting equipment 2 on gantry 1 places a piece of tailings into a certain stacking grid, the digital twin model immediately updates the three-dimensional coordinates of the tailings in virtual space and the stacking layer number of its grid based on the execution feedback, thereby accurately reproducing the real-time three-dimensional layout of the physical storage yard in the virtual environment.

[0073] Furthermore, this embodiment optimizes the spatial arrangement of tailings in the stacking grid using a genetic algorithm to maximize the stacking density. Specifically, the sequence of tailings to be stacked, the position of each tailing in the stacking grid, and the placement orientation are first encoded into chromosomes, with each chromosome representing a possible stacking scheme. Subsequently, the algorithm initializes a population containing multiple such chromosomes, with each chromosome being a candidate stacking scheme.

[0074] It is important to note that the fitness function is the core of the genetic algorithm optimization, used to evaluate the merits of each stacking scheme. In this scenario, the design of the fitness function will focus on multiple objectives such as space utilization, i.e., the ratio of the total volume of stacked tail material to the effective volume of the stacking grid, and stacking stability. By iteratively performing operations such as selection, crossover, and mutation, the natural evolution process is simulated.

[0075] It is important to note that the selection operation tends to retain individuals with high fitness, i.e., stacking schemes with high space utilization and good stability. The crossover operation exchanges some genes (representing the stacking methods of some waste materials) of different parent chromosomes to generate new schemes. The mutation operation randomly changes the values ​​of some genes in the chromosome with a small probability, such as fine-tuning the position or angle of a piece of waste material, to explore new possible solutions. After multiple generations of evolution, the algorithm will converge to one or a few optimal stacking schemes.

[0076] S300: Based on preset grading rules and stacking strategies, target stacking grids are assigned to tail materials, and the motion path of the hoisting equipment 2 is generated; the hoisting equipment 2 on the gantry 1 grabs the tail materials according to the motion path and transfers them to the target stacking grids.

[0077] Based on the extracted tailings feature data, target stacking grids are assigned to the tailings according to preset grading rules. The grading rules are usually formulated based on the physical properties of the tailings, such as automatically classifying tailings by thickness range, size range, or shape characteristics. After classification, the remaining capacity of the stacking grids, stacking status, and center of gravity stability of the tailings are analyzed in real time through a digital twin model, and the optimal stacking position is dynamically calculated. For example, tailings with regular shapes are preferentially placed at the edge of the stacking grid to maximize the filling of internal space, while irregular tailings are reduced by adjusting the placement angle or combining with tailings of complementary shapes, thereby improving the overall space utilization and accessibility.

[0078] In this embodiment, the process of assigning target stacking grids is completed by the digital twin collaborative control unit. Based on the real-time updated virtual stacking yard model, a target stacking grid matching the attributes of each piece of tailing material is matched. The assignment results are synchronized to the management unit of the physical stacking yard, and a unique stacking grid number, layer coordinates and attitude information are recorded for each piece of tailing material.

[0079] After the tailings are classified and the target stacking grid is determined, the system plans a movement path from the current location of the tailings to the target stacking grid. When generating the motion path of gantry 1, the system uses the coordinates of the current location of the tailings and the coordinates of the target stacking grid as the start and end points of the path, and ensures the collision-free and efficient nature of the path through spatial trajectory planning. The generated path data will be directly sent to the controller of the gantry execution unit. It should be noted that the generated path data includes a sequence of three-dimensional coordinate points, a velocity curve, and the grasping posture.

[0080] After the generated path data is sent to the controller of the gantry execution unit, the lifting equipment 2 on gantry 1 performs gripping and transfer operations according to the motion path. The control system of gantry 1 first parses the path instructions and drives the servo motors of the trolley and carriage to move the lifting mechanism equipped with the adaptive end effector 3 along the slide rail 4 to directly above the target tail material in the placement area. Then, the adaptive end effector 3 dynamically adjusts the adsorption point or clamping position according to the three-dimensional model data of the tail material to ensure that the gripping force is evenly distributed and the center of gravity is stable before performing the gripping action. After the gripping is completed, the lifting equipment 2 on gantry 1 will smoothly lift and transfer the tail material to the top of the target stacking grid according to the preset motion path. After the tail material is transported to the designated position above the target stacking grid, the lifting device slowly descends and accurately places the tail material at the coordinate position pre-planned by the digital twin model. After the placement action is completed, the end effector releases the tail material, the hoisting equipment 2 on the gantry 1 raises the lifting device and prepares to execute the next task instruction, and at the same time sends an operation completion signal to the digital twin collaborative control unit to drive the virtual model to update synchronously.

[0081] S400: When a tailings retrieval request is received, tailings matching and virtual cutting simulation are performed based on the digital twin model to generate a retrieval plan; if the target tailings are stacked, the temporary storage-backfilling mechanism is activated, and the hoisting equipment 2 on the gantry 1 removes the tailings above in sequence, and the stacking order is restored after the target tailings are retrieved.

[0082] When the system receives a request to retrieve leftover materials, the digital twin collaborative control unit initiates a leftover material matching and virtual cutting simulation process to generate the optimal material retrieval plan. Specifically, the system extracts the processing requirements input by the user, such as key parameters like the specific dimensions, thickness, and material of the required sheet material. Subsequently, the digital twin model quickly scans and matches the 3D digital models and attribute parameters of all registered leftover materials in its virtual inventory. By comparing dimensions and comprehensively considering factors such as the effective usable area of ​​the leftover materials, their shape, and compatibility with the target parts, multiple candidate lists of leftover materials that meet the criteria are selected, providing a basis for subsequent decision-making.

[0083] After selecting candidate waste materials, an interactive cutting and layout simulation is performed on the selected waste material's 3D model using a virtual pre-simulation module. Specifically, users can drag, rotate, and arrange the 2D outline of the part to be processed on the virtual model of the waste material. The system performs collision detection in real time to ensure that the part outline does not exceed the waste material boundary and dynamically calculates key indicators such as material utilization rate under the current layout scheme. This visual pre-simulation allows users to intuitively evaluate the applicability and yield of each candidate waste material before physical cutting, thereby making the optimal choice and effectively avoiding material waste caused by improper material selection. Once the user confirms the use of a waste material and completes the virtual layout, the system generates the final material handling plan based on the spatial location of that waste material in the stockpile. This plan not only specifies the target waste material but also includes information such as the suggested cutting path.

[0084] It is important to note that the collision detection method mentioned in this embodiment involves inputting the two-dimensional contour of the part to be processed into the system. When the user performs interactive layout operations in the digital twin model, the system calculates the boundary range of each part contour in real time and compares it with the effective usable area of ​​the tail material model. The basic principle is to perform geometric boundary intersection judgment to ensure that every edge and every vertex of the part contour is within the effective boundary of the tail material. If any boundary violation is detected, the system will immediately prompt the user through visual feedback, that is, by highlighting the boundary violation part and prohibiting the current illegal operation, thereby geometrically ensuring the physical feasibility of the layout scheme.

[0085] After ensuring that all part outlines are within the boundary of the waste material, the system will accumulate the total area of ​​all laid-out part outlines and calculate the ratio between this total area and the current effective usable area of ​​the waste material. The result will be displayed in real time as a percentage. This indicator directly reflects the effective utilization of materials under the current layout scheme; a higher value indicates less waste.

[0086] After identifying the target tailings, the digital twin model is used to determine the state of the target tailings in the stockpile. If the target tailings are in a state where they can be directly grabbed in the stockpile, the gantry crane 1 can directly grab them according to the plan.

[0087] However, when the target tailings are piled up on the lower layer by other tailings, the system will automatically activate the temporary storage-backfilling mechanism to retrieve them in an orderly manner. Specifically, the digital twin model first identifies all tailings piled above the target tailings, as well as their spatial positions and stacking order. Then, the gantry crane 1, according to the optimized path generated by the system, sequentially grabs the tailings above and moves them to the placement area for temporary storage. After the target tailings are successfully retrieved and transported to the conveying area, the system will activate the backfilling procedure. The hoisting equipment 2 on the gantry crane 1 will grab the tailings temporarily stored in the placement area again and put them back into their original stacking grids according to the original order recorded by the digital twin model.

[0088] It is important to note that in this step, the virtual cutting simulation is based on the 3D model of the waste material and the processing dimensions input by the user. It simulates the layout scheme and calculates the material utilization rate. Specifically, the virtual simulation module loads the selected 3D model of the waste material from the digital twin model. This model is a high-precision mesh model generated by 3D point cloud reconstruction technology based on data collected by multi-source visual perception units. Then, using the real-time rendering capabilities of visualization toolkits or game engines, the waste material model is presented in a 3D interactive form in the user interface. Users can rotate, scale, and translate the model using input devices such as a mouse, touch screen, or VR controller to observe the shape of the waste material from any angle, providing an intuitive visual basis for subsequent layout operations.

[0089] Users input the two-dimensional outline of the part to be cut through the interface or select a preset shape from the standard parts library. These part outlines are displayed as three-dimensional surfaces or sketches overlaid on the tailings model. Users can flexibly arrange the part outlines on the surface of the tailings model by dragging, rotating, mirroring and other operations.

[0090] As the layout process progresses, the background continuously calculates the material utilization rate of the current layout scheme. Specifically, the surface of the tail material model is discretized into grid cells, and the effective usable area is accurately calculated by detecting the proportion of cells covered by the part outline. Users can compare the utilization rate, cutting path length and other indicators of different schemes and select the optimal solution.

[0091] After the user confirms the layout plan, the module starts a virtual cutting simulation, which is achieved through geometric Boolean operations: the system treats the part outline as a cutting tool and performs real-time Boolean difference operations with the tail material model to dynamically generate a model of the remaining material after cutting. After the simulation is completed, a detailed cutting report is generated, including data such as material utilization rate, number of parts, and distribution map of remaining scrap. If the simulation results are not ideal, the user can go back to the previous step to adjust the layout until satisfied.

[0092] The S500 integrates multi-source data through a cloud-edge collaborative control platform to achieve visualized monitoring of waste material inventory, equipment status, and usage records, and provides waste material information query and scheduling interfaces for downstream production systems.

[0093] The cloud-edge collaborative control platform integrates edge computing nodes and cloud data centers. The edge nodes are responsible for real-time control of the movement of the gantry 1 and visual data processing. The cloud platform aggregates data from multiple storage yards and optimizes the global inventory scheduling strategy through machine learning algorithms.

[0094] Specifically, the cloud-edge collaborative control platform achieves deep integration and collaborative management of multi-source data through its distributed architecture. In this embodiment, the platform adopts an architecture combining edge computing nodes and a cloud data center. The edge nodes are deployed locally in the storage yard and directly interact with the gantry execution unit, vision sensing unit, and storage yard management unit. They are responsible for collecting real-time changes in tailings inventory, operating parameters of the motors on gantry 1, the grabbing status of hoisting equipment 2, and real-time readings from the stacking grid sensors. Meanwhile, the cloud data center receives pre-processed data uploaded from multiple edge nodes through a message queue service and integrates it into the database to form a unified data view covering the entire storage yard.

[0095] In terms of visual monitoring, the platform constructs a digital twin model that is synchronized with the physical world and uses this as the basis to drive the visualization interface. Specifically, the platform uses a 3D rendering engine to map the integrated multi-source data onto the 3D scene of the virtual storage yard, so that users can intuitively view the real-time dynamics of the entire storage yard through terminal devices, such as the spatial distribution of tailings, the occupancy rate of storage grids, and the current task execution progress of gantry 1.

[0096] The multi-source data includes a 3D model of each piece of tailings, real-time inventory location, real-time movement trajectory of gantry 1, and operating parameters such as equipment current and temperature.

[0097] The platform provides standardized interfaces for querying and scheduling surplus material information for downstream production systems. These interfaces typically use protocols such as WebSocket, allowing downstream systems to query surplus material inventory information on demand, such as filtering available surplus materials based on criteria like material, thickness, and minimum usable area.

[0098] It should be noted that the core principles and basic implementations of the various algorithms involved in this application are all existing technologies in this field. Specifically, the stereo matching algorithm used in 3D reconstruction, with its basic process and mathematical model for calculating disparity and generating point clouds through pixel matching, is already well-known and widely used in the field of computer vision. The curvature calculation and polygon approximation methods used in contour recognition and geometric feature analysis are also conventional techniques in digital image processing and computational geometry. In the digital twin synchronization and data processing stages, the sensor data fusion, Kalman filtering, or interpolation methods mentioned, as well as the search algorithms that may be involved in path planning, all fall within the classic algorithm category of control engineering and computer science. Furthermore, the genetic algorithm used to optimize stacking strategies, as a mature global optimization algorithm, has its basic operation processes such as selection, crossover, and mutation disclosed in numerous documents and engineering practices.

[0099] The inventive point of this application lies in the creative combination and application of these known algorithms to solve the specific technical problem of intelligent grading of sheet metal waste, rather than an improvement on the basic principles of the algorithms themselves. The specific formulas and basic implementation code of these known algorithms can be obtained by those skilled in the art from publicly available materials, and therefore will not be elaborated upon in this solution.

[0100] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0101] Although embodiments of the invention have been shown and described, those skilled in the art will recognize that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vision-guided intelligent grading method for leftover board materials, characterized in that, Includes the following steps: S100. Multi-source image data of disordered tailings located in the resettlement area are collected by a multi-view visual sensing unit deployed on the crossbeam of the gantry frame. Based on the multi-source image data, the morphological feature parameters of the tailings are extracted by a three-dimensional point cloud reconstruction and contour recognition algorithm, and its effective usable area is calculated. S200: Based on the extracted tailings feature data, construct a digital twin model that is synchronized with the physical stacking site in real time, and generate a corresponding three-dimensional digital model for each piece of tailings, recording its spatial coordinates, attribute parameters and stacking status. S300: Based on preset grading rules and stacking strategies, target stacking grids are assigned to tailings, and the motion path of the hoisting equipment is generated; the hoisting equipment grabs the tailings according to the motion path and transfers them to the target stacking grids. S400: When a request for retrieving tail material is received, a tail material matching and virtual cutting simulation are performed based on the digital twin model to generate a material retrieval plan. If the target tailings are piled up, the temporary storage-backfilling mechanism is activated, and the hoisting equipment removes the tailings on top in sequence. After the target tailings are retrieved, the stacking order is restored.

2. The intelligent grading method for sheet metal waste based on vision guidance according to claim 1, characterized in that, In step S100, the multi-view vision sensing unit uses multiple spatially distributed industrial cameras to synchronously acquire images; the three-dimensional point cloud reconstruction uses a stereo matching algorithm and uses contour recognition technology to filter the sharp-angled areas of the tail material edge in order to calculate its effective usable area.

3. The intelligent grading method for sheet metal waste based on vision guidance according to claim 1, characterized in that, In step S200, the digital twin model dynamically simulates the three-dimensional layout of tailings in the stacking yard through real-time data driving; the stacking strategy optimizes the spatial arrangement of tailings in the stacking grid based on a genetic algorithm to maximize the stacking density.

4. The intelligent grading method for sheet metal waste based on vision guidance according to claim 1, characterized in that, In step S400, the virtual cutting simulation is based on the three-dimensional model of the tail material and the processing dimensions input by the user to simulate the layout scheme and calculate the material utilization rate; the temporary storage-backfilling mechanism includes temporarily transferring the stacked tail material to the placement area and backfilling it into the corresponding stacking grid in the original order after the target tail material is taken out.

5. The intelligent grading method for sheet metal waste based on vision guidance according to claim 1, characterized in that, Also includes: The S500 integrates multi-source data through a cloud-edge collaborative control platform to achieve visualized monitoring of tail material inventory, equipment status, and usage records, and provides tail material information query and scheduling interfaces for downstream production systems. The cloud-edge collaborative control platform integrates edge computing nodes and cloud data centers. The edge nodes are responsible for real-time control of gantry movement and visual data processing, while the cloud platform aggregates data from multiple storage yards and optimizes global inventory scheduling strategies through machine learning algorithms.

6. A vision-guided intelligent grading device for sheet metal waste, used to implement the method according to any one of claims 1-5, characterized in that, include: The multi-source visual perception unit includes a multi-view visual sensing unit installed on the crossbeam of the gantry frame, which is used to acquire images of tailings and extract feature parameters. The digital twin collaborative control unit includes a virtual model building module, a dynamic scheduling decision module, and a virtual pre-simulation module, which are used to build digital twin models, optimize stacking strategies, and support cutting pre-simulation. The gantry execution unit includes a gantry that can move along a slide rail, lifting equipment, and an adaptive end effector, used to perform grabbing, handling, and stacking operations of tail material; The storage yard management unit includes a physical storage yard divided into a placement area, a storage area, and a conveying area. The storage area is further subdivided into several storage compartments according to the attributes of the waste materials for the classified storage of waste materials. The cloud-edge collaborative control platform is used to integrate data from multiple units to achieve collaborative control and visual monitoring of devices.

7. The vision-guided intelligent grading equipment for sheet metal waste according to claim 6, characterized in that, The multi-source visual perception unit uses a spatially distributed camera array to generate a three-dimensional point cloud model of the tailings through a stereo matching algorithm.

8. The vision-guided intelligent grading equipment for sheet metal waste according to claim 6, characterized in that, The dynamic scheduling decision module of the digital twin collaborative control unit integrates priority management logic, which automatically generates a temporary storage-backfill path when the target tail material is stacked; the virtual pre-show module supports users to perform interactive cutting and layout simulation on the three-dimensional model of the tail material.

9. The vision-guided intelligent grading equipment for sheet metal waste according to claim 6, characterized in that, The cloud-edge collaborative control platform also includes a predictive maintenance module, which trains a fault prediction model based on equipment operation data to achieve early warning of anomalies in key components.

10. The vision-guided intelligent grading equipment for sheet metal waste according to claim 6, characterized in that, The equipment is also integrated with the upstream production management system to automatically match the inventory information of leftover materials with production orders; when a downstream order is triggered, the system automatically recommends leftover materials that meet the size requirements and generates a virtual cutting report.

Images