A multimodal collaborative perception and space-time fusion device coil steel cargo handling system

Abstract

The application relates to the technical field of industrial automation, and discloses a multi-modal collaborative perception and space-time fusion device coiled steel cargo handling system, which comprises a multi-source acquisition module, a space-time fusion module, a collaborative calculation module and a collaborative verification module. Through multi-modal collaborative perception and space-time fusion of coiled steel image, three-dimensional contour data and lifting appliance state data, and based on a coiled steel instance vector, a coiled number candidate vector, a geometric positioning vector and an execution constraint vector, a collaborative verification vector is constructed. The identity recognition result, the geometric positioning result and the execution state result can be jointly verified at the same target coiled steel and the same candidate synchronization time, so that the cargo handling error risk caused by the coiled number hanging error object, the geometric center drift and the lifting appliance misclamping is effectively reduced, and the accuracy, stability and reliability of the target coiled steel identity confirmation and the lifting coordinate output are improved.

Description

Technical Field

[0001] This application relates to the field of industrial automation technology, and in particular to a multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system. Background Technology

[0002] In the production, warehousing, and shipping of coiled steel, sorting operations typically require the identification, location determination, and hoisting of the target coiled steel to ensure the continuity and accuracy of warehousing, transfer, and outbound processes. Existing technologies commonly employ methods such as using industrial cameras to capture images of the coil numbers for identification, or using laser ranging and contour extraction to obtain the geometric contour information of the coiled steel to determine its approximate location and shape. In some automated hoisting scenarios, operational parameters such as the opening and closing status of the lifting equipment, clamping pressure, and height feedback are also considered to monitor the hoisting process. For monitoring and control, the above technologies can meet certain cargo handling needs under single working conditions and single data sources. Especially when the coils are placed relatively neatly, there is little obstruction on site, and the lighting conditions are relatively stable, basic identification and positioning of the coils can be achieved. However, actual application scenarios such as coil storage areas, shipping bays, and finishing buffer areas usually have characteristics such as strong reflectivity of the coil surface, easy obstruction or contamination of the coil number area, dense stacking and limited spacing between coils, and swaying and offset during hoisting. These characteristics make it easy for inconsistencies to occur between the coil number image, geometric contour information, and the feedback information from the hoisting equipment.

[0003] When the coil number image is unstable due to reflection, blurring, dirt, or partial occlusion, the system may only obtain incomplete identification information. When the edge of the coil is obstructed by adjacent coils, stacked skewed, or the viewing angle is limited, the geometric contour data may have a center positioning deviation. Furthermore, the opening and closing amount, pressure value, and height information fed back by the lifting device during the actual grabbing process are often only used for equipment motion control and are not correlated with the aforementioned identification and positioning results. As a result, the existing technology generally has the following problems: the coil number identification result and the geometric positioning result are disconnected, making it difficult to determine whether the current identification object is the target coil. The lack of cross-verification between visual recognition results and the actual grasping state of the lifting equipment makes it difficult to detect misidentification, mispositioning, or deviation of the lifting object in a timely manner. When anomalies are identified, there is also a lack of reliable judgment mechanisms for factors such as image quality, degree of occlusion, and positioning error, often relying solely on manual verification. These defects can easily lead to errors in the identification of coiled steel, inaccurate output of lifting coordinates, frequent repeated data collection, and decreased inventory management efficiency. In severe cases, it may also cause problems such as mis-lifting, collisions, or inconsistencies between warehouse location records and on-site conditions. Therefore, it is necessary to propose a technical solution that can achieve collaborative verification of multi-source perception information for coiled steel inventory management scenarios. Summary of the Invention

[0004] This application proposes a multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, this application adopts the following technical solution: a multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system, comprising:

[0006] The multi-source acquisition module is used to acquire the coil number image, 3D contour data and lifting device status data corresponding to the target coil steel.

[0007] The spatiotemporal fusion module is used to perform time synchronization processing and spatial calibration processing on the coil number image, 3D contour data and spreader status data to establish the spatiotemporal correspondence of the coil number image, 3D contour data and spreader status data for the same target coil, and generate spatiotemporal fusion data.

[0008] The collaborative solution module is used to perform coil instance detection, coil number detection, coil number recognition and instance association processing based on the coil number image in the spatiotemporal fusion data to construct coil instance vectors and coil number candidate vectors; to perform end face local plane estimation and local plane projection circle fitting processing based on the 3D contour data in the spatiotemporal fusion data to construct geometric positioning vectors; and to perform contact determination and opening / closing amount mapping processing based on the spreader status data in the spatiotemporal fusion data to construct execution constraint vectors.

[0009] The collaborative verification module is used to construct a collaborative verification vector based on the coil instance vector, coil number candidate vector, geometric positioning vector, and execution constraint vector. Based on the collaborative verification vector, it performs credibility assessment and consistency verification processing to output the target coil identity data and hoisting coordinate data when the verification passes, and output review control data when the verification fails.

[0010] Further time synchronization processing includes:

[0011] Add acquisition time markers to the roll number image, 3D contour data, and lifting device status data respectively;

[0012] Establish a time-series correspondence relationship for the same target coil based on the acquisition time marker;

[0013] Remove incomplete acquisition fragments from any data item in missing roll number images, 3D contour data, or lifting device status data;

[0014] Timestamp alignment is performed on the retained acquisition segments to generate synchronized data sets.

[0015] Further spatial calibration processing includes:

[0016] Based on preset camera extrinsic parameters and preset contour sensor extrinsic parameters, the image coordinates corresponding to the roll number image and the contour coordinates corresponding to the 3D contour data in the synchronized data group are transformed to the same reference coordinate system;

[0017] The spatial orientation of the lifting device is determined based on the preset installation posture parameters and the height of the lifting device.

[0018] Establish a spatial correspondence between the roll number image, 3D contour data, and the spatial pose of the lifting device to generate spatiotemporal fusion data.

[0019] Furthermore, the process of performing coil instance detection and coil number detection to construct coil instance vectors includes:

[0020] Perform coil instance detection on the coil number image to obtain at least one coil instance region;

[0021] Perform volume number detection on the volume number image to obtain at least one volume number region;

[0022] Remove volume number regions that do not meet the preset region validity conditions;

[0023] Based on the inclusion relationship, center distance relationship, and relative position relationship between the coil number area and the coil instance area, the coil instance area corresponding to the coil number area is determined.

[0024] The center of the coil instance is determined based on the coil instance area, and an instance identifier is assigned to the coil instance area.

[0025] Construct a coil instance vector based on the coil instance region, coil instance center, and instance identifier.

[0026] Furthermore, the volume number identification and instance association processing also includes:

[0027] Perform character recognition processing based on the volume number region to obtain at least one volume number candidate result and corresponding recognition confidence information;

[0028] Remove candidate volume numbers that do not meet the preset character integrity conditions;

[0029] Associate the retained coil number candidate results with the corresponding coil instance vector;

[0030] Construct a roll number candidate vector based on the roll number candidate results, identification confidence information, roll number region, and the roll steel instance vector corresponding to the roll number region;

[0031] Write the volume number candidate results, identification confidence information, and instance identifier into the volume number candidate vector.

[0032] Furthermore, the end-face local plane estimation and local plane projection circle fitting processes include:

[0033] Extract the candidate contour point set of the end face corresponding to the target coil from the 3D contour data;

[0034] Perform outlier removal and smoothing on the candidate contour point set of the end face;

[0035] Estimate the local plane of the end face based on the processed candidate contour point set of the end face;

[0036] The processed 3D contour data is projected onto a local plane of the end face to form projected contour data.

[0037] Perform circle fitting based on the projected profile data to determine the geometric center, the geometric center projection point, and the outer diameter of the coil.

[0038] Further determine the contour fitting residuals and contour integrity markers based on the projected contour data;

[0039] A geometric positioning vector is constructed based on the geometric center, the geometric center projection point, the outer diameter of the coil, the contour fitting residual, and the contour integrity marker.

[0040] Furthermore, the contact determination and opening / closing quantity mapping processing includes:

[0041] Threshold determination is performed on the clamping pressure, the rate of change of the opening and closing amount of the spreader, and the rate of change of the spreader height in the spreader status data;

[0042] Select stable contact segments that meet preset stability conditions from the spreader status data;

[0043] When the clamping pressure meets the preset pressure threshold, and the rate of change of the opening and closing amount of the spreader and the rate of change of the spreader height are less than the corresponding preset rate of change thresholds, the contact state is determined to be an effective contact state.

[0044] When the contact state is an effective contact state, the equivalent clamping diameter is determined based on the preset mapping relationship between the opening and closing amount and the clamping diameter.

[0045] Extract the current spreader opening / closing amount and current spreader height corresponding to the effective contact state;

[0046] An execution constraint vector is constructed based on the contact state, equivalent clamping diameter, current spreader opening / closing amount, and current spreader height.

[0047] Furthermore, the collaborative verification vector construction process includes:

[0048] Extract the coil instance center and instance identifier from the coil instance vector;

[0049] Extract candidate volume numbers and identify confidence information from the candidate volume number vector;

[0050] Extract the geometric center projection point, outer diameter of the coil, contour fitting residual, and contour integrity marker from the geometric positioning vector;

[0051] Extract the contact state, equivalent clamping diameter, current spreader opening / closing amount, and current spreader height from the execution constraint vector;

[0052] Determine the projection deviation based on the projection points of the coil example center and the geometric center;

[0053] The clamping deviation is determined based on the outer diameter of the coil and the equivalent clamping diameter.

[0054] A collaborative verification vector is constructed based on candidate volume numbers, identification confidence information, projection bias, clamping bias, contour fitting residuals, contour integrity markers, contact states, and instance identifiers.

[0055] Further, the credibility assessment and consistency verification processes include:

[0056] Extracting image quality information from the volume number image;

[0057] Determine whether the contact state in the collaborative verification vector meets the preset valid contact conditions;

[0058] When the preset effective contact conditions are met, a consistency check is performed based on image quality information, recognition confidence information, projection deviation, clamping deviation, contour fitting residual and contour integrity mark to obtain the check status, which includes pass status and fail status.

[0059] If the image quality information does not meet the preset quality conditions, the contact status does not meet the preset valid contact conditions, or the consistency verification fails, the process will proceed to the review stage.

[0060] Furthermore, the review of control data includes:

[0061] When the verification status is "failed", output the image segment to be verified from the volume number image, the volume number candidate result, the identification confidence information, the instance identifier, and at least one of the projection deviation alarm and clamping deviation alarm.

[0062] And when the verification status is "passed", output the result as a pass mark.

[0063] The beneficial effects of this invention are as follows:

[0064] 1. This invention acquires coil number images, 3D contour data, and spreader status data around the same target coil, and performs time synchronization and spatial calibration processing on the coil number images, 3D contour data, and spreader status data. This enables data from different sources to correspond to the same target coil and the same candidate synchronization time, solving the problems in the prior art where multi-source acquired data are independent of each other, the spatiotemporal correspondence is unclear, cross-coil mismatch is likely to occur, and incomplete acquisition segments lead to distortion of tallying basis. In this way, it realizes unified alignment and reliable fusion of multi-source sensing data, and improves the consistency and usability of tallying input data.

[0065] 2. This invention performs coil instance detection, coil number detection, coil number identification, and instance association processing, combined with end face local plane estimation, local plane projection circle fitting, contact determination, and opening / closing amount mapping processing, to form a collaborative constraint relationship between coil identity information, geometric positioning information, and lifting device execution status information. This solves the problems in the prior art where coil number identification results are disconnected from the target coil object, geometric positioning results are separated from the actual clamping state, and it is difficult to confirm the consistency of the lifting target. In this way, it realizes the collaborative solution between target coil identity confirmation, spatial positioning, and execution constraints, improves the accuracy of lifting positioning, and reduces the risk of misidentification, mispositioning, and misclamping.

[0066] 3. By constructing a collaborative verification vector and performing credibility assessment and consistency verification processing, this invention can output target coil steel identity data and hoisting coordinate data when the verification passes, and output review control data when the verification fails. This solves the problems of lack of a unified verification mechanism for tallying results and difficulty in timely identification and diversion of abnormal results in the prior art. It also realizes hierarchical output and abnormal review control of tallying results, improves the accuracy, stability and reliability of tallying result output, and reduces the dependence on manual review and repeated data collection under abnormal working conditions. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort:

[0068] Figure 1 This is a diagram of the collaborative solution module of the present invention;

[0069] Figure 2 This is a diagram of the collaborative verification module of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example

[0072] like Figure 1 and Figure 2 As shown, the present invention discloses a multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system, including: a multi-source acquisition module, a spatiotemporal fusion module, a collaborative calculation module, and a collaborative verification module.

[0073] In this embodiment, the multi-source acquisition module is used to acquire the coil number image, three-dimensional contour data, and lifting device status data corresponding to the target coil. The lifting device status data includes the lifting device opening / closing amount, clamping pressure, and lifting device height. The target coil refers to the coil object to be identified, located, or lifted in the current tallying task. The coil number image is used to characterize the appearance information of the coil number area of ​​the target coil for subsequent coil number detection, coil number recognition, and instance association processing. The three-dimensional contour data is used to characterize the spatial contour information of the end face of the target coil and its neighborhood for subsequent end face local plane estimation and local plane projection circle fitting processing. The lifting device status data is used to characterize the execution state of the lifting device relative to the target coil for subsequent contact determination and opening / closing amount mapping processing.

[0074] Since the coil number image, 3D contour data, and lifting device status data correspond to identity information, geometric information, and execution information, respectively, and their physical attributes are different, the multi-source acquisition module only acquires and records the acquisition time of the same target coil steel organization data. It does not perform time synchronization processing and spatial calibration processing on the three types of data. The time synchronization processing and spatial calibration processing described later are completed by the spatiotemporal fusion module.

[0075] In existing technologies, the acquisition of roll number images by industrial cameras, the acquisition of 3D contour data by contour sensors, and the output of lifting device status data by lifting device controllers are all conventional support methods. In this embodiment, the roll number image can be acquired by an industrial camera located in the storage area, lifting channel, or near the lifting device; the 3D contour data can be acquired by a laser contour sensor, a line laser scanning device, or a depth camera, as long as it can output the 3D point set corresponding to the end face area of ​​the target coil steel; the lifting device status data can be output by the lifting device controller, the gantry controller, or a data acquisition unit connected to the lifting device. The opening and closing amount of the lifting device can be acquired by a displacement sensor or an encoder, the clamping pressure can be acquired by a pressure sensor, and the lifting device height can be acquired by a height detection unit corresponding to the lifting mechanism. The above hardware configuration is only used to provide data sources for this invention and is not a core innovative means of this invention in the multi-source acquisition module, therefore it is not limited in detail.

[0076] In response to the problem in the background technology that the coil number image, three-dimensional contour data and lifting device status data are easily obtained from different time slices and are difficult to correspond to the same target steel coil, in this embodiment, the multi-source acquisition module does not perform independent discrete acquisition of the three types of data, but organizes data acquisition around the same lifting operation cycle of the same target steel coil.

[0077] Specifically, the lifting operation cycle refers to the time interval from when the lifting device enters the preset operating height range corresponding to the target coil's operating area, to when the lifting device leaves the preset operating height range or the current lifting task ends, provided that the current cargo handling task has been loaded. By introducing the dual constraints of the target coil and the lifting operation cycle, the coil number image, 3D contour data, and lifting device status data can be converged into the same physical object and the same operation process during the acquisition phase, thereby reducing the risk of errors in the correspondence of different source data.

[0078] To ensure a clear triggering basis for the hoisting operation cycle, in this embodiment, the preset operating height range can be determined based on the preset stacking reference height of the warehouse where the target coil is located. Preferably, the preset operating height range can be set to 0.5 meters to 2.5 meters above the preset stacking reference height, more preferably 0.8 meters to 1.5 meters. When the hoisting device enters the preset operating height range, the multi-source acquisition module begins to acquire coil number images, 3D contour data, and hoisting device status data. When the hoisting device leaves the preset operating height range, or when the current hoisting task ends, the multi-source acquisition module ends the current round of data acquisition. The reason for adopting the above range is that if the preset operating height range is set too large, it will introduce a large amount of redundant data unrelated to the target coil, increasing the burden of subsequent screening. If the preset operating height range is set too small, it may not be able to cover the stage when the hoisting device approaches the target coil and the stage when the coil number area is visible, resulting in incomplete coil number images or 3D contour data. Therefore, adopting the above range can balance data integrity and acquisition efficiency.

[0079] To enable the subsequent spatiotemporal fusion module to establish a temporal correspondence, the multi-source acquisition module records the acquisition time for each acquired roll number image, 3D contour data, and lifting device status data. Specifically, for each roll number image, the corresponding image acquisition time is recorded; for each set of 3D contour data, the corresponding contour acquisition time is recorded; and for each piece of lifting device status data, the corresponding status acquisition time is recorded.

[0080] Based on this, the multi-source acquisition module aggregates the roll number images, 3D contour data, and lifting device status data acquired within the same lifting operation cycle to form a multi-source raw data set for the same lifting operation cycle. The multi-source raw data set includes at least: a roll number image set with the image acquisition time, a 3D contour data set with the contour acquisition time, and a lifting device status data set with the status acquisition time. In this way, a clear original time basis can be provided for subsequent time synchronization processing, thereby avoiding the problem of independent acquisition by each system and difficulty in confirming the correspondence in the existing technology.

[0081] In this embodiment, the acquisition frequency of the coil number image, 3D contour data, and spreader status data can be set according to the on-site lifting speed and the target coil size. Preferably, the acquisition frequency of the coil number image is 10 Hz to 30 Hz, the acquisition frequency of the 3D contour data is 10 Hz to 50 Hz, and the acquisition frequency of the spreader status data is 20 Hz to 200 Hz. The reason for adopting the above configuration is that if the acquisition frequency of the coil number image is too low, it will be difficult to obtain an effective image containing the coil number area in a timely manner when the spreader swings or is partially obstructed; if the acquisition frequency of the coil number image is too high, it will significantly increase image redundancy and storage burden. If the acquisition frequency of 3D contour data is too low, it will reduce the continuity of the end face contour, which is not conducive to the subsequent local plane estimation and local plane projection circle fitting processing of the end face. If the acquisition frequency of 3D contour data is too high, the contour point set of adjacent time moments will be too repetitive. The lifting device status data belongs to the execution status feedback quantity, and its change rate is usually higher than that of the image and contour data. Therefore, its acquisition frequency is preferably higher than that of the roll number image and 3D contour data to ensure that the subsequent contact determination and opening and closing amount mapping processing have sufficient time resolution. The above acquisition frequency is a preferred parameter configuration for implementing the present invention and does not constitute a limitation on the scope of protection of the present invention.

[0082] In a preferred embodiment, to further reduce the interference of adjacent coils on the coil number image and 3D contour data, the multi-source acquisition module can perform regional constraint acquisition within the area where the target coil is located in the current tallying task. Specifically, based on the storage location range and the current position of the lifting device corresponding to the current tallying task, a preset working area corresponding to the target coil can be determined, and the coil number image and 3D contour data falling within the preset working area can be preferably acquired. The preset working area can be set as an area extending horizontally from the estimated center of the target coil by 0.3 meters to 1.5 meters. If the area is set too small, the coil number area or end face contour may not be fully captured when the lifting device swings or the target coil shifts position. If the area is set too large, the coil number area or contour boundary of adjacent coils may be introduced into the subsequent processing, weakening the stability of subsequent coil instance detection, coil number detection, and instance association processing. Therefore, by constraining the acquisition area, the phenomenon of multiple coils mixing can be suppressed at the acquisition stage, which is a preferred enhancement method for addressing the problem of easy confusion between different coils in the background art.

[0083] In another preferred embodiment, to avoid data failure at a single moment due to rapid movement of the lifting device, momentary occlusion of the roll number area, or partial loss of contour data, the multi-source acquisition module adopts a continuous acquisition method, rather than acquiring only a single frame of roll number image, a single set of 3D contour data, or a single piece of lifting device status data. Specifically, a pre-acquisition time of 0.5 to 2.0 seconds can be reserved before the lifting device enters the preset working height range, and a post-acquisition time of 0.5 to 2.0 seconds can be reserved after the lifting device leaves the preset working height range. Preferably, the pre-acquisition time and the post-acquisition time are each set to 0.8 to 1.5 seconds. The reason for setting a pre-acquisition duration is that the coil number area is usually clearer and the three-dimensional contour data is more complete when the target coil has not yet formed a significant obstruction by the lifting device. The reason for setting a post-acquisition duration is that the clamping pressure, lifting device opening and closing amount, and lifting device height after the lifting device establishes contact with the target coil are more conducive to subsequent contact determination and determination of the equivalent clamping diameter. By setting the pre-acquisition duration and post-acquisition duration, the coverage of effective samples in the multi-source raw data set can be improved, thereby reducing the impact of single-moment acquisition failure on the subsequent processing link. This continuous acquisition method is also a preferred enhancement method in this embodiment.

[0084] Therefore, in this embodiment, the multi-source acquisition module does not output isolated single-type sampled values, but rather a set of multi-source raw data organized for the same target steel coil and the same hoisting operation cycle. The multi-source raw data set includes at least the coil number image, three-dimensional contour data, hoisting tool status data and their corresponding acquisition time information. The subsequent spatiotemporal fusion module can continue to perform time synchronization processing and spatial calibration processing on the basis of the multi-source raw data set to output spatiotemporal fusion data.

[0085] Compared to the existing technology that uses a decentralized approach of image system, contour system, and lifting control system for data acquisition and passive correlation in the later stages, this implementation method organizes multi-source acquisition around the lifting operation cycle of the same target steel coil and explicitly records the acquisition time for various types of data. This ensures that all subsequent data processing is based on the original data of the same physical object and the same operation process, thereby reducing the risk of misjudgment caused by mismatch of different source data, confusion across steel coils, and incomplete acquisition segments. It also provides stable data support for the subsequent construction of steel coil instance vectors, coil number candidate vectors, geometric positioning vectors, and execution constraint vectors.

[0086] In this embodiment, the spatiotemporal fusion module performs time synchronization and spatial calibration processing on the coil number image, 3D contour data, and lifting device status data to establish the spatiotemporal correspondence of the coil number image, 3D contour data, and lifting device status data for the same target coil, and generates spatiotemporal fusion data. Its input consists of the coil number image, 3D contour data, and lifting device status data acquired by the multi-source acquisition module within the same target coil and the same lifting operation cycle, as well as the corresponding image acquisition time, contour acquisition time, and status acquisition time. Its output is a combination of data that corresponds to each other in time and belongs to the same reference coordinate system in space, which is used by the subsequent collaborative solution module to perform coil instance detection, coil number detection, coil number recognition and instance association processing, end face local plane estimation and local plane projection circle fitting processing, as well as contact determination and opening / closing amount mapping processing.

[0087] In existing technologies, timestamp alignment from different acquisition sources and extrinsic parameter calibration from different sensors are conventional supporting technical means. This implementation does not attempt to change these basic calibration and communication mechanisms themselves. Instead, it addresses the problem in the background technology that although roll number images, 3D contour data, and lifting device status data exist separately, they are difficult to confirm whether they correspond to the same target steel roll and the same moment of operation due to different sampling frequencies, acquisition times, and coordinate systems. The organization of time synchronization processing and spatial calibration processing is adapted to this problem, so that it can directly serve the construction of subsequent steel roll instance vectors, roll number candidate vectors, geometric positioning vectors, and execution constraint vectors.

[0088] Specifically, a time-series correspondence is established based on the acquisition times of the roll number image, 3D contour data, and spreader status data, and timestamp alignment processing is performed on the data for which the time-series correspondence is established.

[0089] In this embodiment, considering that the acquisition frequency of spreader status data is usually higher than that of roll number images and 3D contour data, and that spreader status data can continuously reflect the physical process of spreader approaching the target coil, establishing contact, and leaving the target coil, it is preferable to use the status acquisition time sequence corresponding to the spreader status data as the reference time sequence, and use the status acquisition time in the reference time sequence as the candidate synchronization time. Here, the candidate synchronization time refers to the reference time used to organize a set of roll number images, 3D contour data, and spreader status data. For each candidate synchronization time, the roll number image corresponding to the image acquisition time with the smallest time difference with the candidate synchronization time is selected from the roll number image set, the 3D contour data corresponding to the contour acquisition time with the smallest time difference with the candidate synchronization time is selected from the 3D contour data set, and the spreader status data corresponding to the candidate synchronization time is extracted, thereby forming a set of candidate time sequence corresponding data.

[0090] The reason for using the spreader status data as the reference time series is that: if only the roll number image is used as the synchronization reference, the state transition before and after contact may not be accurately reflected due to the large image acquisition interval during the rapid approach or contact establishment of the spreader; if only the three-dimensional contour data is used as the synchronization reference, the spatial contour changes at key moments may be easily missed when the contour sampling interval is large or there is short-term occlusion; while the spreader status data, due to its higher sampling frequency and more continuous changes, is more suitable as the basis for generating candidate synchronization moments. In this way, the subsequent spatiotemporal fusion data can be organized around the same moment of operation, which is different from the passive matching method based only on coarse temporal proximity in the existing technology.

[0091] To avoid introducing incorrect correspondences under different sampling frequency conditions by simple nearest neighbor matching, this embodiment preferably sets a preset time alignment tolerance for the volume number image and the three-dimensional contour data respectively.

[0092] Specifically, the preset time alignment tolerance for roll number images can be set to 30 to 150 milliseconds, preferably 50 to 100 milliseconds, and the preset time alignment tolerance for 3D contour data can be set to 20 to 100 milliseconds, preferably 30 to 80 milliseconds. The reason for adopting the above range is that if the preset time alignment tolerance is set too large, roll number images or 3D contour data that are far from the current candidate synchronization time may be mismatched to the same data group, resulting in obvious misalignment between the subsequent coil instance center, geometric center projection point and the spatial pose of the lifting device; if the preset time alignment tolerance is set too small, in the case of on-site communication delay, sensor internal buffering or short-time frame loss, a large number of candidate synchronization times may not be able to match valid roll number images or valid 3D contour data, reducing data availability. Therefore, setting preset time alignment tolerances for different data sources helps to achieve a balance between synchronization accuracy and matching robustness.

[0093] In this embodiment, when a candidate synchronization moment fails to match a valid roll number image or valid 3D contour data within the corresponding preset time alignment tolerance range, the data group corresponding to the candidate synchronization moment can be determined as an incomplete data group. For incomplete data groups, one of two processing methods can be preferably executed: one is to directly remove it to avoid the subsequent collaborative solution module from performing instance association or geometric fitting in the absence of key input; the other is to retain the matched spreader status data and add missing markers for subsequent modules to filter as needed. The latter method is mainly used when it is necessary to retain the spreader execution continuity information, such as when the subsequent contact determination process needs to identify the spreader pressure change trend near a certain candidate synchronization moment.

[0094] In a preferred embodiment, to further improve the physical consistency of time synchronization processing, the candidate synchronization moments can be divided into operation stages based on the trends of spreader height and clamping pressure in the spreader status data. Specifically, the state of the candidate synchronization moments can be divided into the approach stage, the contact establishment stage, and the departure stage. Then, only the candidate time-series corresponding data where both the roll number image and the 3D contour data fall into the same operation stage are retained as candidate outputs of the spatiotemporal fusion data. The reason for introducing this operation stage constraint is that the roll number image usually has better roll number area visibility in the approach stage, and the 3D contour data is also more complete when the spreader has not yet significantly obscured it. The effective change in clamping pressure mainly occurs in the contact establishment stage. If the data of different physical stages are mechanically aligned without distinction, even if the time difference meets the requirements, it may cause the roll number image used for subsequent roll steel instance detection, roll number detection, roll number recognition, and instance association processing to not correspond to the same physical state as the 3D contour data used for end face local plane estimation and local plane projection circle fitting processing. It should be noted that the operation stage constraint is a preferred enhancement method in this embodiment.

[0095] After completing the time synchronization process, the spatiotemporal fusion module further performs spatial calibration processing. The spatial calibration processing includes: based on the preset camera extrinsic parameters and the preset contour sensor extrinsic parameters, uniformly transforming the image coordinates corresponding to the roll number image and the contour coordinates corresponding to the three-dimensional contour data to the same reference coordinate system, and determining the spatial pose of the lifting device based on the preset lifting device installation pose parameters and the lifting device height.

[0096] In this embodiment, the preset camera extrinsic parameters refer to the pose parameters of the industrial camera relative to the same reference coordinate system, including at least rotation and translation parameters; the preset contour sensor extrinsic parameters refer to the pose parameters of the contour sensor relative to the same reference coordinate system, including at least rotation and translation parameters; the preset lifting device installation pose parameters refer to the pose parameters of the lifting device relative to the same reference coordinate system at a preset installation reference position. The same reference coordinate system is preferably the global reference coordinate system of the warehouse area, or it can be a local reference coordinate system of a single working channel, as long as the roll number image, three-dimensional contour data, and the spatial pose of the lifting device determined by the lifting device height can ultimately belong to the same reference coordinate system.

[0097] In the specific implementation, the image coordinates corresponding to the roll number image are located in the camera imaging plane, while the contour coordinates corresponding to the 3D contour data are located in the contour sensor coordinate system. The two cannot be directly used for the subsequent comparison between the projection points of the coil instance center and the geometric center. Therefore, this implementation first associates the roll number image with the spatial position of the camera in the same reference coordinate system based on preset camera extrinsic parameters. Then, based on preset contour sensor extrinsic parameters, the 3D contour data is transformed from the contour sensor coordinate system to the same reference coordinate system. It should be emphasized here that the roll number image does not directly generate 3D spatial points, but establishes a geometric mapping relationship between it and the same reference coordinate system so that when the subsequent collaborative calculation module calculates the geometric center projection point based on the geometric center, it can return to the correct image plane position. In contrast, after the coordinate transformation is completed, the 3D contour data can directly enter the subsequent end face local plane estimation and local plane projection circle fitting processing.

[0098] For the spreader status data, this embodiment does not treat it as coordinate information as a whole. Specifically, the clamping pressure and spreader opening / closing amount are scalar execution state quantities and do not participate in spatial coordinate transformation. Only the spreader height is combined with the preset spreader installation posture parameters to determine the spreader spatial posture. The reason for this approach is that if the clamping pressure or spreader opening / closing amount is mistakenly treated as spatial coordinate quantities for transformation, it will cause confusion between physical and geometric quantities, which will not only fail to meet the requirement of dimensional consistency but also destroy the physical meaning of the subsequent execution constraint vector. Therefore, in this embodiment, the spreader spatial posture is jointly determined by the preset spreader installation posture parameters and the spreader height, so that the spreader-related spatial information, the roll number image, and the three-dimensional contour data are in the same reference coordinate system.

[0099] In a preferred embodiment, if the lifting device mainly moves up and down in the vertical direction, the spatial pose of the lifting device can be obtained by superimposing the vertical displacement represented by the lifting device height on the initial spatial pose corresponding to the preset lifting device installation pose parameters. If the lifting device also has a translation compensation structure or a swing compensation structure, the spatial pose of the lifting device can be further updated by combining the corresponding compensation parameters. In this way, the spatial pose of the lifting device is no longer an isolated device state, but an intermediate spatial quantity that can establish a unified spatial correspondence with the subsequent geometric center projection point and the coil steel instance center. Preferably, the rotation angle error of the preset lifting device installation pose parameters is controlled within the range of 0.1 degrees to 2.0 degrees, and the translation error is controlled within the range of 1 mm to 20 mm. The basis for adopting the above range is that if the rotation angle error or translation error exceeds the range, the spatial pose of the lifting device determined by the lifting device height will deviate significantly from the actual spatial position, thereby affecting the accuracy of the subsequent collaborative verification module when comparing the coil steel instance center with the geometric center projection point and the coil steel outer diameter with the equivalent clamping diameter.

[0100] In this embodiment, to ensure the long-term stability of spatial calibration processing, a static calibration can be performed during the system initialization phase to determine the preset camera extrinsic parameters, preset contour sensor extrinsic parameters, and preset lifting device installation pose parameters. Periodic recalibration is performed during the equipment maintenance cycle to correct pose drift caused by equipment vibration, loose installation, or long-term operation. It should be noted that static calibration and periodic recalibration are conventional supporting engineering methods and are not the core innovation of this invention in the spatiotemporal fusion module. The focus of this invention is not on proposing a new extrinsic parameter calibration algorithm, but rather on: based on the multi-source acquisition module already ensuring the acquisition of multi-source raw data for the same target coil and within the same lifting operation cycle, through time synchronization processing and spatial calibration processing, ensuring that the coil number image, 3D contour data, and lifting device-related spatial information correspond to the same candidate synchronization time and belong to the same reference coordinate system, thereby forming spatiotemporal fusion data that can directly participate in subsequent collaborative calculation and collaborative verification.

[0101] Therefore, in this embodiment, the spatiotemporal fusion data output by the spatiotemporal fusion module includes at least: a roll number image after timestamp alignment, three-dimensional contour data after timestamp alignment, corresponding spreader status data, and the spatial correspondence between the roll number image, three-dimensional contour data, and spreader spatial pose relative to the same reference coordinate system. This embodiment introduces a candidate synchronization time construction method based on spreader status data, sets a preset time alignment tolerance for different sources, and establishes a spatial correspondence under the same reference coordinate system based on preset extrinsic parameters and spreader height. This ensures that the roll number image, three-dimensional contour data, and spreader status data not only correspond to the same operational instant in time but also belong to the same reference coordinate system in space. Thus, the subsequent... When performing coil instance detection, coil number detection, coil number recognition, and instance association processing, the collaborative solution module can use coil number images that are consistent with the current 3D contour data and the spatial pose of the spreader. When performing end-face local plane estimation and local plane projection circle fitting processing, it can use 3D contour data that are consistent with the current coil number image and the spatial pose of the spreader. When performing contact determination and opening / closing amount mapping processing, it can also use data sets that are at the same candidate synchronization time as the current coil number image and 3D contour data. Ultimately, this can significantly reduce the adverse effects of cross-source mismatch, operational stage misalignment, and coordinate system inconsistency on subsequent tallying results data, providing a stable and reliable input foundation for the collaborative verification module to construct projection deviation, clamping deviation, and collaborative verification vectors.

[0102] In this embodiment, the collaborative solution module is divided into three cooperative solution links: volume number detection, volume number identification and instance association processing, end face local plane estimation and local plane projection circle fitting processing, and contact determination and opening / closing amount mapping processing. The three links share the spatiotemporal fusion data under the same candidate synchronization moment and finally output comparable vector results.

[0103] The roll number detection, roll number recognition, and instance association processing include: performing roll steel instance detection on the roll number image to obtain the roll steel instance region; performing roll number detection on the roll number image to obtain the roll number region; determining the roll steel instance vector corresponding to the roll number candidate vector based on the inclusion relationship, center distance relationship, and relative position relationship between the roll number region and the roll steel instance region; and performing character recognition processing based on the roll number region to obtain at least one roll number candidate result and corresponding recognition confidence information.

[0104] In this embodiment, coil instance detection is first performed on the coil number image to obtain a set of coil instance regions in the coil number image at the current candidate synchronization time. Coil instance detection can be implemented using existing target detection networks, instance segmentation networks, or boundary segmentation methods. This part belongs to conventional supporting technical means, and this invention does not limit it. This embodiment focuses more on the fact that the coil instance regions obtained by coil instance detection are not directly used to output the inventory results, but serve as the object basis for subsequent coil number detection, coil number recognition, and instance association processing. For each coil instance region, its coil instance center is further determined. Preferably, the coil instance center is determined based on the regional centroid of the coil instance region. The reason for using the regional centroid instead of the circumscribed rectangle center or the local edge center is that the regional centroid is less sensitive to local notches on the end face, edge reflections, and local occlusions, and is more suitable as the image plane reference position when constructing the projection deviation with the geometric center projection point.

[0105] After obtaining the coil instance region, coil number detection is performed on the coil number image to obtain the coil number region. The coil number detection can be implemented using existing text detection networks or character region extraction methods. This part is also a conventional supporting technical means. Based on this, this embodiment does not directly regard the character recognition results in the coil number region as the identity of the target coil, but further performs instance association processing. The purpose of instance association processing is to solve the problem in the background technology that although the coil number is identified, it is impossible to reliably confirm which coil object it belongs to. To this end, this embodiment uses the inclusion relationship, center distance relationship and relative position relationship between the coil number region and the coil instance region for joint determination.

[0106] Specifically, the inclusion relationship is used to determine whether the coil number area is entirely or mostly located within a certain coil steel instance area; the center distance relationship is used to determine the distance between the center of the coil number area and the center of each coil steel instance; and the relative position relationship is used to determine whether the coil number area is located within a preset coil number distribution zone within the coil steel instance area.

[0107] Here, the preset roll number distribution zone is set based on the actual situation that the roll number on the end face of the coil is usually not located near the center hole, nor does it usually appear outside the outer edge of the end face. Preferably, the preset roll number distribution zone can be set as a ring area formed with the center of the coil instance as the center, with an inner radius of 0.2 times the equivalent radius of the coil instance area and an outer radius of 0.95 times the equivalent radius of the coil instance area. The equivalent radius of the coil instance area refers to the radius corresponding to the area of ​​the coil instance area after it is equivalent to the area of ​​a circle. The reason for using this equivalent radius definition is that the coil instance area in the image may not be a standard circle due to viewing angle, occlusion, and edge defects. Using the radius after area equivalence to represent its overall scale is more conducive to defining the reasonable distribution range of the roll number area in a uniform way. If the roll number area is too close to the center of the coil instance, it may correspond to noise, stains, or reflections near the center hole; if the roll number area exceeds 0.95 times the equivalent radius of the coil instance area, it is more likely to belong to adjacent coils or background markings. Therefore, by introducing the preset roll number distribution zone, the reasonable belonging range of the roll number area can be effectively narrowed.

[0108] In a preferred embodiment, a correlation score can be further constructed between the roll number area and the coil instance area. Specifically, the correlation score can be constructed by weighting the following three types of quantities: the area ratio of the roll number area falling into the coil instance area, the reciprocal of the normalized distance between the center of the roll number area and the center of the coil instance, and the determination result of whether the roll number area is within a preset roll number distribution zone. Among them, the area ratio is used to characterize the spatial inclusion degree between the roll number area and the coil instance area; the reciprocal of the normalized distance is used to characterize the proximity between the roll number area and the center of the coil instance; and the ring zone determination result is used to characterize the physical location rationality of the roll number area.

[0109] Preferably, the association score ranges from 0 to 1. When the association score corresponding to a certain coil steel instance area is the maximum value among all candidate coil steel instance areas and is greater than the preset association threshold, it is determined that the coil number area belongs to the coil steel instance area. The preset association threshold is preferably set to 0.45 to 0.80, more preferably 0.55 to 0.70. If the threshold is set too low, it is easy to mistakenly associate coil number areas that do not belong to the target coil steel with a certain coil steel instance area. If the threshold is set too high, it is easy to cause association failure in the case of partial missing coil number areas or incomplete edges of coil steel instance areas. In this way, instance association processing no longer relies on a single overlapping relationship or nearest distance relationship, but is based on the joint constraints of multiple spatial relationships. Compared with the coarse attribution method in the prior art, it can more robustly solve the problem of incorrect coil number attribution in dense multi-coil steel scenarios.

[0110] After determining the association between the roll number region and the coil instance region, character recognition processing is performed on the roll number region to obtain at least one roll number candidate result and corresponding recognition confidence information to construct a roll number candidate vector. Character recognition processing can be implemented using convolutional neural networks, recurrent neural networks, attention recognition networks, or character template matching methods. This part belongs to conventional supporting technical means. In this embodiment, it is preferable to output at least two roll number candidate results and corresponding recognition confidence information, including the best roll number candidate result and the second best roll number candidate result. The reason for this setting is that when the roll number region has local dirt, missing characters, or strong reflection, outputting only a single roll number result cannot reflect the distinguishability of the recognition result. The confidence difference between the best roll number candidate result and the second best roll number candidate result can provide a more reliable basis for recognition stability for the subsequent collaborative verification module. The recognition confidence information is preferably represented by a normalized probability value between 0 and 1. Thus, the roll number candidate vector is no longer a simple character recognition result, but a combination of the roll number candidate result, recognition confidence information, and instance affiliation relationship.

[0111] The end face local plane estimation and local plane projection circle fitting process includes: performing outlier removal and smoothing on the three-dimensional contour data; estimating the end face local plane based on the processed three-dimensional contour data; projecting the three-dimensional contour data onto the end face local plane to form projected contour data; and performing circle fitting based on the projected contour data to determine the geometric center, the geometric center projection point, and the outer diameter of the coiled steel.

[0112] In this embodiment, the role of the geometric positioning vector is to convert the spatiotemporally fused 3D contour data into geometric parameter results that can be compared with the coil instance vector and the constraint vector. In the prior art, if the circumcenter is directly obtained on the original 3D contour data or the center is approximated on the image edge, it is easily affected by oblique viewing, local occlusion, depth anomalies and the mixing of adjacent coil boundaries, resulting in unstable geometric positioning. The key innovation of this invention lies in first restoring the local plane where the coil end face is located, and then performing circle fitting in the local plane. By utilizing the physical property that the coil end face is approximately a circular plane, the stability of the geometric center and the outer diameter of the coil is improved.

[0113] First, outlier removal and smoothing are performed on the 3D contour data. Outlier removal can employ neighborhood statistical filtering, median filtering, or outlier removal methods based on distance thresholds. Smoothing can use local weighted averaging, moving least squares, or low-order surface smoothing methods. These are all conventional supporting techniques. Outlier removal is used to remove outliers caused by abnormal laser reflection, depth jumps, or the intrusion of adjacent coil boundaries. Smoothing is used to reduce local noise and make subsequent local plane estimation of the end face more stable. Preferably, the neighborhood radius for outlier removal is set to 10 mm to 80 mm, more preferably 20 mm to 50 mm. If the neighborhood radius is too small, outliers may not be effectively identified; if the neighborhood radius is too large, the true edge of the coil end face may be misjudged as an outlier.

[0114] After obtaining the processed 3D contour data, the end face local plane estimation process is performed. Specifically, the least squares plane fitting method can be used to solve for the plane parameters that minimize the sum of the squared distances from the processed 3D contour points to the fitting plane. Preferably, the end face local plane can be represented as a linear relationship between the plane normal parameters and the spatial coordinates, and normalization constraints are applied to the plane normal parameters to ensure that subsequent distance calculations have clear physical meaning. After the end face local plane estimation is completed, the distances from each contour point to the local plane can be further statistically analyzed, and the fitting error obtained by the local plane projection circle fitting process is defined as the contour fitting residual. Here, the contour fitting residual refers to the average or root mean square distance from each point in the projected contour data to the fitted circle. The reason for adopting this definition is that the subsequent collaborative verification module needs to use the contour fitting residual to characterize the geometric reliability of the three-dimensional contour data. Therefore, the source of this quantity is clearly defined in the collaborative solution module. Preferably, the plane fitting error threshold corresponding to the local plane estimation of the end face can be set to 2 mm to 20 mm, more preferably 5 mm to 12 mm. If the plane fitting error exceeds this range, it usually means that non-end face points, adjacent coil steel boundaries, or depth abnormal areas are mixed in the contour.

[0115] After the local plane of the end face is determined, the processed three-dimensional contour data is projected onto the local plane of the end face to form two-dimensional projected contour data. This processing method of first fitting a plane and then projecting a fitted circle, instead of directly fitting a circle on the image plane or any slice plane, is adopted because the end face of the coiled steel usually has a certain degree of oblique gaze and attitude offset in actual scenarios. If a circle is directly fitted on the image plane, the real circular end face will be affected by the projection deformation and often appear as an ellipse or an irregular curve, thereby reducing the accuracy of the geometric center and the outer diameter of the coiled steel. On the other hand, restoring the local plane of the end face first and then performing circle fitting in the local plane is more in line with the physical geometric properties of the end face of the coiled steel itself. This is the key technical means of this invention to solve the problem that geometric positioning is easily affected by oblique gaze and local occlusion.

[0116] After obtaining the projected contour data, a circle fitting process is performed. Preferably, least squares circle fitting, RANSAC circle fitting, or weighted circle fitting methods can be used. In this embodiment, least squares circle fitting with boundary point weights is preferred to reduce the influence of local gaps or sparse edge regions on the fitting results. By minimizing the sum of squared distances from each projection point to the fitting circle circumference, the center of the fitting circle and the fitting radius are obtained. Then, the center of the fitting circle is back-mapped to three-dimensional space to obtain the geometric center. Furthermore, based on the image space correspondence established by the spatiotemporal fusion module, the geometric center projection point is obtained. At the same time, the outer diameter of the coiled steel is determined according to the fitting radius. Preferably, the contour fitting residual is controlled within the range of 1 mm to 15 mm, more preferably 2 mm to 8 mm. If the contour fitting residual exceeds this range, it indicates that there is a significant missing boundary, contour mixing, or point set distortion in the current projected contour, which is not conducive to constructing a stable geometric positioning vector.

[0117] Therefore, in this embodiment, the geometric positioning vector includes at least the geometric center, the geometric center projection point, and the outer diameter of the coil. The geometric center is used to determine the lifting coordinate data, the geometric center projection point is used to construct the projection deviation with the coil instance center, and the outer diameter of the coil is used to construct the clamping deviation with the equivalent clamping diameter. Compared with the prior art, which directly obtains the geometric center and the outer diameter of the coil based on the image edge or contour data without pose recovery, this embodiment can more stably restore the actual geometric state of the coil end face through a combination of end face local plane estimation and local plane projection circle fitting.

[0118] The contact determination and opening / closing amount mapping process includes: performing threshold determination on the clamping pressure, opening / closing amount change rate, and lifting height change rate in the lifting device status data; determining the contact state as a valid contact state when the clamping pressure meets the preset pressure threshold and the opening / closing amount change rate and lifting height change rate are respectively less than the corresponding preset change rate thresholds; and determining the equivalent clamping diameter based on the preset opening / closing amount-clamping diameter mapping relationship when the contact state is a valid contact state.

[0119] In this embodiment, the role of the execution constraint vector is to transform the spreader status data from a simple equipment feedback quantity into an execution constraint result that can participate in the subsequent consistency judgment together with the geometric positioning vector. In the prior art, spreader feedback is usually only used for equipment motion control and does not participate in the validity verification of the current target coil geometric positioning result. This invention enables the spreader status data to directly enter the subsequent collaborative verification link through contact judgment and opening / closing quantity mapping processing.

[0120] Specifically, the clamping pressure, the rate of change of the opening and closing amount of the spreader, and the rate of change of the spreader height in the spreader status data are first evaluated using thresholds. The reason for using these three quantities to jointly determine the contact state is that relying solely on clamping pressure can easily misjudge short-term collisions and vibration disturbances as effective contact; relying solely on the rate of change of the opening and closing amount of the spreader or the rate of change of the spreader height cannot confirm whether the spreader has effectively clamped the target coil. Therefore, this embodiment adopts a joint determination mechanism based on pressure satisfaction, stable rate of change of the opening and closing amount, and stable rate of change of height. Preferably, the preset pressure threshold is set to 0.1 kN to 5 kN, more preferably 0.3 kN to 2 kN. The preset threshold for the change rate of the spreader opening and closing is set to 1 mm / s to 50 mm / s, more preferably 3 mm / s to 20 mm / s. The preset threshold for the change rate of the spreader height is set to 1 mm / s to 80 mm / s, more preferably 5 mm / s to 30 mm / s. If the pressure threshold is set too low, slight disturbances may be misjudged as effective contact. If the pressure threshold is set too high, the state of newly established contact may be missed. If the change rate threshold is set too high, the spreader may still be in obvious motion and may be misjudged as stable contact. If the change rate threshold is set too low, the determination of effective contact state will be too delayed.

[0121] When the clamping pressure meets the preset pressure threshold, and the rate of change of the spreader opening and closing amount and the rate of change of the spreader height are less than the corresponding preset rate of change thresholds, the contact state is determined to be a valid contact state. The technical effect of this processing method is that only when the spreader has not only applied sufficient clamping pressure, but its opening and closing action and lifting action have entered a relatively stable stage, is the current state regarded as a valid contact state that can be used for subsequent geometric constraint judgment. Thus, false contact situations when the spreader approaches rapidly, has short-term collisions, or is not stably clamped can be eliminated.

[0122] After confirming the contact state as an effective contact state, the equivalent clamping diameter is further determined based on a preset opening / closing amount-clamping diameter mapping relationship. Here, the equivalent clamping diameter refers to the coil diameter corresponding to the current clamping configuration of the spreader, which is derived from the current spreader opening / closing amount. In the prior art, the spreader opening / closing amount is usually only used as a control quantity for the equipment and is not used to deduce the geometric constraints of the target coil. This invention maps the spreader opening / closing amount to the equivalent clamping diameter, allowing it to be directly compared with the outer diameter of the coil, thereby establishing the actual clamping dimensions of the equipment to the geometric dimensions of the target coil. The posterior constraint relationship between them, the preset opening and closing amount-clamping diameter mapping relationship is preferably established in advance based on the opening and closing amount calibration samples of the lifting device under different standard clamping diameter working conditions. It can be achieved by piecewise linear mapping, polynomial fitting mapping or table lookup interpolation mapping. Preferably, the opening and closing amount-clamping diameter mapping error is controlled within the range of 2 mm to 25 mm, more preferably 5 mm to 15 mm. If the mapping error is too large, the equivalent clamping diameter obtained by back-derived from the lifting device status data will lose its practical significance in comparing with the outer diameter of the coiled steel, affecting the construction of subsequent clamping deviation.

[0123] Therefore, in this embodiment, the execution constraint vector includes at least the contact state and the equivalent clamping diameter. The contact state is used to characterize whether the current lifting device has formed a stable and effective contact with the target coil steel, and the equivalent clamping diameter is used to characterize the geometric clamping scale of the current lifting device in the contact state. In this way, the lifting device state data no longer only serves the equipment motion control, but is transformed into the execution constraint result that can be entered into the collaborative verification module for consistency judgment together with the geometric positioning vector.

[0124] In summary, in this embodiment, the collaborative solution module outputs a coil instance vector, a coil number candidate vector, a geometric positioning vector, and an execution constraint vector based on spatiotemporal fusion data. The coil instance vector includes at least the coil instance center; the coil number candidate vector includes at least the coil number candidate result and corresponding identification confidence information; the geometric positioning vector includes at least the geometric center, the geometric center projection point, and the outer diameter of the coil; and the execution constraint vector includes at least the contact state and the equivalent clamping diameter.

[0125] This implementation ensures that the candidate coil number no longer deviates from the coil instance area, the geometric center and outer diameter of the coil no longer deviate from the coil object in the coil number image, and the contact state and equivalent clamping diameter no longer deviate from the current geometric positioning result. As a result, it can provide structurally unified, physically meaningful and comparable input data for the subsequent collaborative verification module to construct projection deviation, clamping deviation and collaborative verification vectors, thereby significantly reducing the risk of tallying errors caused by incorrect coil number attachment, geometric positioning drift and mis-clamping of the lifting device.

[0126] In this embodiment, the collaborative verification module first constructs a collaborative verification vector based on the projection deviation between the center of the coil instance and the geometric center projection point, as well as the clamping deviation between the outer diameter of the coil and the equivalent clamping diameter. Then, it combines the image quality parameters corresponding to the coil number image and the contour fitting residual obtained by local plane projection circle fitting processing to perform credibility assessment and consistency verification processing, thereby forming a joint judgment result.

[0127] The projection deviation is determined based on the distance between the projection point of the coil instance center and the geometric center, while the clamping deviation is determined based on the absolute value of the difference between the outer diameter of the coil and the equivalent clamping diameter.

[0128] In this embodiment, the coil instance center is derived from the coil instance vector, the geometric center projection point and the coil outer diameter are derived from the geometric positioning vector, and the equivalent clamping diameter is derived from the execution constraint vector. The projection deviation is used to characterize whether the coil object in the coil number image and the geometric center obtained from the 3D contour data calculation correspond to the same target coil in the image plane. The clamping deviation is used to characterize whether the coil outer diameter obtained from the 3D contour data calculation is consistent with the equivalent clamping diameter corresponding to the current clamping configuration of the lifting device. Unlike the prior art that only relies on the coil number to identify confidence information or only relies on the geometric positioning result to make a judgment, this invention introduces both projection deviation and clamping deviation, incorporating identity attribution, geometric positioning and execution constraints into the same verification link.

[0129] Specifically, the projection deviation is preferably determined based on the Euclidean distance between the projection points of the coil instance center and the geometric center. To eliminate the influence of different field-of-view scales and different coil sizes, this embodiment preferably further normalizes this distance. Preferably, the equivalent diameter of the coil instance region can be used as the normalization scale. Here, the equivalent diameter of the coil instance region is the diameter of the circle obtained by converting the area of ​​the coil instance region into the area of ​​a circle. If the equivalent radius of the coil instance region has already been defined above, then the equivalent diameter of the coil instance region is twice the equivalent radius of the coil instance region. Therefore, the projection deviation can be expressed as the ratio of the distance between the projection points of the coil instance center and the geometric center to the equivalent diameter of the coil instance region. The reason for using normalized projection deviation is that if pixel distance is used directly, the same pixel difference has different physical meanings in large-size and small-size coil images, making it difficult to establish a unified threshold. After normalization, the projection deviation can be uniformly mapped to a dimensionless quantity. Preferably, the threshold for projection deviation is set to 0.02 to 0.25, more preferably 0.05 to 0.15. If the threshold is set too small, it is easy to misjudge inconsistency when the edge of the coil is slightly occluded, the image segmentation boundary fluctuates slightly, or the geometric center projection point is affected by a small number of pixel errors. If the threshold is set too large, the judgment boundary for cross-coil mismatch will be relaxed, weakening the ability of this invention to suppress the mislabeling of coil numbers.

[0130] The clamping deviation is preferably determined based on the absolute value of the difference between the outer diameter of the coil and the equivalent clamping diameter. To facilitate a unified comparison between coils of different diameters, this embodiment preferably normalizes the absolute value of the difference according to the outer diameter of the coil, thereby obtaining a dimensionless clamping deviation. The reason for using a normalized clamping deviation is that the same absolute difference represents different degrees of clamping deviation for coils with larger diameters and coils with smaller diameters. If the absolute difference is used directly for unified judgment, it is easy to cause the problem of being too lenient for small-diameter coils and too strict for large-diameter coils. Preferably, the threshold for clamping deviation is set to 0.01 to 0.20, more preferably 0.03 to 0.10. If the threshold is set too small, under the influence of the error in the mapping between the opening and closing amount of the lifting device and the clamping diameter, the change in the thickness of the coil coating layer, or slight elastic deformation, it is easy to misjudge the actual acceptable clamping state as inconsistent. If the threshold is set too large, it will weaken the ability to use the execution constraint to reverse the geometric positioning result.

[0131] In a preferred embodiment, the clamping deviation is determined only when the contact state in the execution constraint vector is an effective contact state, based on the absolute value of the difference between the outer diameter of the coil and the equivalent clamping diameter. If the contact state is not an effective contact state, it indicates that the current lifting device has not yet formed a stable and effective contact with the target coil, and the lifting device state data cannot provide reliable execution constraint information. In this case, the data group corresponding to the current candidate synchronization moment can be marked as a data group with insufficient execution constraint information, and its priority in subsequent reliability assessment and consistency verification processing can be reduced, or the verification control data can be directly output. The reason for this processing is that if the clamping deviation is still forcibly calculated in an ineffective contact state, the opening and closing amount mapping result of the unstable action stage will be mistakenly taken as the effective clamping scale, which will reduce the reliability of the collaborative verification.

[0132] The co-verification vector is constructed based on the identification confidence information, projection bias, and clamping bias in the volume number candidate vector.

[0133] In this embodiment, the role of the collaborative verification vector is to organize the identification confidence information, projection bias, and clamping bias in the roll number candidate vector into a structured input that can reflect the credibility of the identity, the consistency between the image and geometry, and the consistency between geometry and execution. This structured input is then used for subsequent credibility assessment and consistency verification processing. Compared with the existing technology that judges the identity of the target roll steel solely based on the probability of roll number recognition or judges the matching relationship solely based on the proximity of geometric positions, this invention introduces a collaborative verification vector that simultaneously covers the three types of information: identity, geometry, and execution. This allows the system to comprehensively examine whether the results from multiple sources support each other within the same vector structure.

[0134] Specifically, the identification confidence information in the roll number candidate vector preferably includes the identification confidence value corresponding to the optimal roll number candidate result. In a preferred embodiment, it may also include the confidence difference between the optimal roll number candidate result and the second-best roll number candidate result, which is used to characterize the distinguishability of the roll number identification result. When the identification confidence value of the optimal roll number candidate result is high and the confidence difference with the second-best roll number candidate result is large, it indicates that the roll number identification result is more stable; otherwise, it indicates that the roll number identification result has strong ambiguity. The projection deviation is used to reflect the consistency between the roll steel instance vector and the geometric positioning vector, and the clamping deviation is used to reflect the consistency between the geometric positioning vector and the execution constraint vector. Therefore, in a preferred embodiment, the co-verification vector includes at least the following components: the identification confidence value corresponding to the optimal roll number candidate result, the confidence difference of the roll number candidate result, the projection deviation, and the clamping deviation. If only one roll number candidate result and its corresponding identification confidence information are output in the roll number candidate vector, the confidence difference may not be included in the construction.

[0135] To facilitate subsequent credibility assessment and consistency verification, it is preferable to map each component in the co-verification vector to a normalized range of 0 to 1. The identification confidence information itself is preferably already a normalized quantity between 0 and 1. The projection bias and clamping bias can be normalized by comparing them with their respective thresholds. The purpose of normalization is that different components have different original meanings and dimensions. If the original values ​​are directly combined, it is easy to cause a single scale to dominate the overall result. Through normalization, each component can participate in subsequent processing within the same numerical scale.

[0136] In a preferred embodiment, the collaborative verification vector can be further represented as a combination of positive support terms and negative constraint terms. Here, the identification confidence information belongs to the positive support terms, and the larger the value, the more reliable the volume number identification result. The projection bias and clamping bias belong to the negative constraint terms, and the larger the value, the worse the consistency between the volume number candidate vector, the geometric positioning vector, and the execution constraint vector. The reason for adopting this grouping method is that the subsequent credibility assessment process needs to consider whether the support terms are strong enough and whether the constraint terms are small enough. If the positive support terms and negative constraint terms are not distinguished, it is difficult to form a clear collaborative evaluation relationship.

[0137] The credibility assessment and consistency verification process includes: performing credibility assessment and consistency verification on the co-verification vector based on the image quality parameters corresponding to the volume number image and the contour fitting residual obtained by local planar projection circle fitting.

[0138] In this embodiment, the image quality parameter is used to characterize whether the current volume number image has good volume number detection and recognition conditions, and the contour fitting residual is used to characterize whether the current three-dimensional contour data can stably support the construction of the geometric positioning vector. In the prior art, the volume number recognition result and the geometric positioning result often directly participate in subsequent decisions without further considering whether the input image and input contour itself are reliable. The present invention uses the image quality parameter and the contour fitting residual as auxiliary evaluation quantities of the collaborative verification vector, thereby avoiding misjudgment caused by low-quality volume number images or low-quality three-dimensional contour data.

[0139] Specifically, the image quality parameters are preferably determined by a combination of the sharpness, saturation ratio, and occlusion ratio in the roll number image. Sharpness is used to characterize whether the edges of the roll number area are clear, saturation ratio is used to characterize whether there are too many areas of strong reflection, and occlusion ratio is used to characterize whether the roll number area is occluded by the hoist, adjacent roll steel, or background objects. Preferably, the image quality parameters are normalized to between 0 and 1, with larger values ​​indicating better roll number image quality. Preferably, the preset image quality threshold is set to 0.40 to 0.85, more preferably 0.55 to 0.75. If the threshold is set too low, low-quality roll number images may still enter the pass branch; if the threshold is set too high, roll number images that, although they have a small amount of reflection or slight occlusion, can still support identity determination, may be misjudged as unusable.

[0140] The contour fitting residual originates from the local planar projection circle fitting process in the collaborative solution module. It is preferably represented by the average distance or root mean square distance from the projection contour point to the fitting circle. In order to facilitate its participation in the credibility evaluation together with the image quality parameters, the contour fitting residual is preferably normalized to between 0 and 1 by a preset residual upper limit and then inversely transformed to form a geometric quality evaluation quantity obtained by converting the contour fitting residual. The larger the geometric quality evaluation quantity, the better the geometric fitting quality. Preferably, the preset residual threshold is set to 1 mm to 15 mm, more preferably 2 mm to 8 mm. This range is consistent with the preferred range of the contour fitting residual in the collaborative solution module, which facilitates the unification of terminology and parameters between upstream and downstream modules.

[0141] In a preferred embodiment, a comprehensive reliability evaluation value can be calculated based on the identification confidence information, projection bias, and clamping bias in the collaborative verification vector, as well as image quality parameters and geometric quality evaluation values ​​obtained from the contour fitting residual transformation. Preferably, the comprehensive reliability evaluation value can be obtained through a weighted combination method, wherein the identification confidence information, image quality parameters, and geometric quality evaluation values ​​are used as positive support terms, and the projection bias and clamping bias are used as negative constraint terms. In order to take into account the comprehensive effects of identity recognition, geometric localization, and execution constraints, the weight of the identification confidence information is preferably set to 0.20 to 0.40, the constraint weight corresponding to the projection bias is set to 0.15 to 0.30, the constraint weight corresponding to the clamping bias is set to 0.15 to 0.30, and the weight of the image quality parameters is set to... The weights of the geometric quality evaluation quantity are set to 0.10 to 0.20. The basis for these weight values ​​is as follows: the candidate coil number vector is the direct source of the target coil steel identity data, so the weight of the identification confidence information should not be too low; projection bias and clamping bias jointly determine the consistency of multi-source results, so their constraint weights should be roughly at the same level as the weights of the identification confidence information; image quality parameters and geometric quality evaluation quantity reflect the reliability of input data and are auxiliary evaluation quantities, so their weights can be appropriately lower than the main verification items. If the weights of image quality parameters and geometric quality evaluation quantity are too high, it may lead to data groups with good input data quality but incorrect object matching still being misjudged as reliable; if the weights of projection bias and clamping bias are too low, it will weaken the role of the collaborative verification module in suppressing cross-source mismatches.

[0142] In addition to the comprehensive reliability evaluation value, this embodiment also performs consistency verification processing. Specifically, the consistency verification processing includes at least the following conditions: the projection deviation is not greater than a preset projection deviation threshold; when the contact state is an effective contact state, the clamping deviation is not greater than a preset clamping deviation threshold; and the identification confidence information is not lower than a preset identification confidence threshold. Preferably, the preset identification confidence threshold is set to 0.50 to 0.95, more preferably 0.65 to 0.85. If the threshold is set too low, it is easy to directly use the results of the roll number identification with strong ambiguity for the cargo handling decision. If the threshold is set too high, it is easy to misjudge the data group that can pass the collaborative verification as failing when the roll number is partially damaged but can still be mutually verified with the geometric positioning and execution constraints.

[0143] In a preferred embodiment, a pass threshold corresponding to the comprehensive reliability evaluation value can be further set. Preferably, the pass threshold is set to 0.60 to 0.90, more preferably 0.70 to 0.85. If the comprehensive reliability evaluation value is lower than the pass threshold, even if the projection deviation or clamping deviation does not significantly exceed the limit, it can be determined that the overall reliability of the data group corresponding to the current candidate synchronization moment is insufficient. The reason for this setting is that in some cases, the distance between the center of the coil instance and the projection point of the geometric center is relatively close, and the difference between the outer diameter of the coil and the equivalent clamping diameter is also small. However, if the quality of the coil number image is too poor or the contour fitting residual is too large, it is still not appropriate to directly output the sorting result. Therefore, by adding a comprehensive reliability evaluation value threshold in addition to the consistency verification process, the robustness of the system to boundary scenarios can be further improved.

[0144] When the results of the credibility assessment and consistency verification process meet the preset conditions, the target coil steel identity data determined based on the coil number candidate vector and the hoisting coordinate data determined based on the geometric center are output; when the results of the credibility assessment and consistency verification process do not meet the preset conditions, the review control data are output.

[0145] In this embodiment, when the consistency verification process passes and the comprehensive credibility evaluation value is not lower than the preset passing threshold, it is determined that there is sufficient consistency and reliability among the current coil instance vector, coil number candidate vector, geometric positioning vector, and execution constraint vector. Then, the target coil identity data and lifting coordinate data are output. The target coil identity data is preferably determined by the optimal coil number candidate result. The lifting coordinate data is preferably determined based on the geometric center. In a preferred embodiment, the lifting coordinate data can be further determined by combining the preset lifting tool offset and safe approach distance on the basis of the geometric center. However, its essence is still the lifting spatial position result constructed based on the geometric center. The reason for adopting this processing method is that the geometric center is the most stable representation of the coil spatial position in the geometric positioning vector. Using it to determine the lifting coordinate data can reduce the risk of displacement caused by local defects or edge reflections on the outer edge of the coil.

[0146] When the consistency verification process fails, or the overall credibility evaluation value is lower than the preset pass threshold, it is preferable not to directly output the target coil identity data and hoisting coordinate data, but to output review control data. The review control data may include at least one of the following: coil number image reacquisition control mark, 3D contour data reacquisition control mark, manual review control mark, or low-speed retry control mark. The specific review control data to be used can be determined according to the reason for failure. Preferably, when the identification confidence information is lower than the preset identification confidence threshold or the image quality parameter is lower than the preset image quality threshold, the coil number image reacquisition control mark is output; when the contour fitting residual is higher than the preset residual threshold, the 3D contour data reacquisition control mark is output; when the projection deviation and clamping deviation are both abnormal and the source of the problem cannot be determined, the manual review control mark is output.

[0147] The reason for using categorized review control data is that different reasons for failure correspond to different rectification methods. If only the failure result is output uniformly, it will be difficult to guide the subsequent system to perform targeted corrections. Low-speed retry control flags are the preferred enhanced control method in this implementation.

[0148] In summary, in this embodiment, the collaborative verification module first constructs a collaborative verification vector based on the projection deviation between the coil instance center and the geometric center projection point, and the clamping deviation between the coil outer diameter and the equivalent clamping diameter. Then, it combines image quality parameters and contour fitting residuals to perform credibility assessment and consistency verification processing. Finally, it outputs the target coil identity data and lifting coordinate data when the coil passes the test, and outputs review control data when the coil fails the test. This significantly reduces the adverse effects of factors such as incorrect coil number attachment, geometric center drift, and mis-clamping of lifting equipment on the tallying result data, thereby establishing the target coil identity data and lifting coordinate data on a more reliable multi-source collaborative verification basis.

[0149] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimodal collaborative sensing and spatiotemporal fusion equipment coil steel handling system, characterized in that, include: The multi-source acquisition module is used to acquire the coil number image, 3D contour data and lifting device status data corresponding to the target coil steel. The spatiotemporal fusion module is used to perform time synchronization processing and spatial calibration processing on the coil number image, 3D contour data and spreader status data to establish the spatiotemporal correspondence of the coil number image, 3D contour data and spreader status data for the same target coil, and generate spatiotemporal fusion data. The collaborative solution module is used to perform coil instance detection, coil number detection, coil number recognition and instance association processing based on the coil number image in the spatiotemporal fusion data to construct coil instance vectors and coil number candidate vectors; to perform end face local plane estimation and local plane projection circle fitting processing based on the 3D contour data in the spatiotemporal fusion data to construct geometric positioning vectors; and to perform contact determination and opening / closing amount mapping processing based on the spreader status data in the spatiotemporal fusion data to construct execution constraint vectors. The collaborative verification module is used to construct a collaborative verification vector based on the coil instance vector, coil number candidate vector, geometric positioning vector, and execution constraint vector. Based on the collaborative verification vector, it performs credibility assessment and consistency verification processing to output the target coil identity data and hoisting coordinate data when the verification passes, and output review control data when the verification fails.

2. The equipment coil steel handling system according to claim 1, characterized in that, Time synchronization processing includes: Add acquisition time markers to the roll number image, 3D contour data, and lifting device status data respectively; Establish a time-series correspondence relationship for the same target coil based on the acquisition time marker; Remove incomplete acquisition fragments from any data item in missing roll number images, 3D contour data, or lifting device status data; Timestamp alignment is performed on the retained acquisition segments to generate synchronized data sets.

3. The equipment coil steel handling system according to claim 1, characterized in that, Spatial calibration processing includes: Based on preset camera extrinsic parameters and preset contour sensor extrinsic parameters, the image coordinates corresponding to the roll number image and the contour coordinates corresponding to the 3D contour data in the synchronized data group are transformed to the same reference coordinate system; The spatial orientation of the lifting device is determined based on the preset installation posture parameters and the height of the lifting device. Establish a spatial correspondence between the roll number image, 3D contour data, and the spatial pose of the lifting device to generate spatiotemporal fusion data.

4. The equipment coil steel handling system according to claim 1, characterized in that, The process of performing coil instance detection and coil number detection to construct coil instance vectors includes: Perform coil instance detection on the coil number image to obtain at least one coil instance region; Perform volume number detection on the volume number image to obtain at least one volume number region; Remove volume number regions that do not meet the preset region validity conditions; Based on the inclusion relationship, center distance relationship, and relative position relationship between the coil number area and the coil instance area, the coil instance area corresponding to the coil number area is determined. The center of the coil instance is determined based on the coil instance area, and an instance identifier is assigned to the coil instance area. Construct a coil instance vector based on the coil instance region, coil instance center, and instance identifier.

5. The equipment coil steel handling system according to claim 4, characterized in that, the coil... Number recognition and instance association processing also includes: Perform character recognition processing based on the volume number region to obtain at least one volume number candidate result and corresponding recognition confidence information; Remove candidate volume numbers that do not meet the preset character integrity conditions; Associate the retained coil number candidate results with the corresponding coil instance vector; Construct a roll number candidate vector based on the roll number candidate results, identification confidence information, roll number region, and the roll steel instance vector corresponding to the roll number region; Write the volume number candidate results, identification confidence information, and instance identifier into the volume number candidate vector.

6. The equipment coil steel handling system according to claim 1, characterized in that, The end-face local plane estimation and local plane projection circle fitting processes include: Extract the candidate contour point set of the end face corresponding to the target coil from the 3D contour data; Perform outlier removal and smoothing on the candidate contour point set of the end face; Estimate the local plane of the end face based on the processed candidate contour point set of the end face; The processed 3D contour data is projected onto a local plane of the end face to form projected contour data. Perform circle fitting based on the projected profile data to determine the geometric center, the geometric center projection point, and the outer diameter of the coil. Further determine the contour fitting residuals and contour integrity markers based on the projected contour data; A geometric positioning vector is constructed based on the geometric center, the geometric center projection point, the outer diameter of the coil, the contour fitting residual, and the contour integrity marker.

7. The equipment coil steel handling system according to claim 1, characterized in that, Contact determination and opening / closing amount mapping processing include: Threshold determination is performed on the clamping pressure, the rate of change of the opening and closing amount of the spreader, and the rate of change of the spreader height in the spreader status data; Select stable contact segments that meet preset stability conditions from the spreader status data; When the clamping pressure meets the preset pressure threshold, and the rate of change of the opening and closing amount of the spreader and the rate of change of the spreader height are less than the corresponding preset rate of change thresholds, the contact state is determined to be an effective contact state. When the contact state is an effective contact state, the equivalent clamping diameter is determined based on the preset mapping relationship between the opening and closing amount and the clamping diameter. Extract the current spreader opening / closing amount and current spreader height corresponding to the effective contact state; An execution constraint vector is constructed based on the contact state, equivalent clamping diameter, current spreader opening / closing amount, and current spreader height.

8. The equipment coil steel handling system according to claim 1, characterized in that, The collaborative verification vector construction process includes: Extract the coil instance center and instance identifier from the coil instance vector; Extract candidate volume numbers and identify confidence information from the candidate volume number vector; Extract the geometric center projection point, outer diameter of the coil, contour fitting residual, and contour integrity marker from the geometric positioning vector; Extract the contact state, equivalent clamping diameter, current spreader opening / closing amount, and current spreader height from the execution constraint vector; Determine the projection deviation based on the projection points of the coil example center and the geometric center; The clamping deviation is determined based on the outer diameter of the coil and the equivalent clamping diameter. A collaborative verification vector is constructed based on candidate volume numbers, identification confidence information, projection bias, clamping bias, contour fitting residuals, contour integrity markers, contact states, and instance identifiers.

9. A multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system according to claim 8, characterized in that, Credibility assessment and consistency verification processes include: Extract image quality information from the volume number image; Determine whether the contact state in the collaborative verification vector meets the preset valid contact conditions; When the preset effective contact conditions are met, a consistency check is performed based on image quality information, recognition confidence information, projection deviation, clamping deviation, contour fitting residual and contour integrity mark to obtain the check status, which includes pass status and fail status. If the image quality information does not meet the preset quality conditions, the contact status does not meet the preset valid contact conditions, or the consistency verification fails, the process will proceed to the review stage.

10. A multimodal collaborative sensing and spatiotemporal fusion equipment coil handling system according to claim 9, characterized in that, The review and control data includes: When the verification status is "failed", output the image segment to be verified from the volume number image, the volume number candidate result, the identification confidence information, the instance identifier, and at least one of the projection deviation alarm and clamping deviation alarm. And when the verification status is "passed", output the result as a pass mark.

Images