A Self-Inspection Method for Container Loading and Unloading Based on Multi-View Perception

Abstract

This invention relates to the field of image processing and container loading and unloading monitoring technology, and discloses a self-inspection method for container loading and unloading based on multi-view perception. The method collects front, rear, left, right, and top views of the loading and unloading channel, container type parameters, and operation location information; performs time alignment and frame filtering to generate a detection input set; then extracts features of the container edges, corner fittings, and door frames, completes view registration and container surface stitching to form a global view; establishes a partitioned detection model for the door area, side panel area, top panel area, corner fitting area, and bottom connection area to identify pose deviations, surface damage, door lock closure status, and corner fitting anomalies; and combines lifting points, placement status, fixing status, historical anomaly records, and loading and unloading sequence to generate a self-inspection report and handling instructions. This method can improve the consistency of anomaly location, loading and unloading correlation determination, and result verification.

Description

Technical Field

[0001] This invention relates to the fields of image processing and container loading and unloading monitoring technology, and in particular to a self-inspection method for container loading and unloading based on multi-view perception. Background Technology

[0002] In the field of image processing and container loading and unloading monitoring technology, existing solutions for container loading and unloading self-inspection typically combine image acquisition of the loading and unloading channel, recording of operation location information, and manual verification. This approach suffers from limitations such as large blind spots, unclear location of abnormal parts, and a disconnect between structural status and loading / unloading status. Existing methods often rely on separate identification of front and rear views or local area images, combined with container type parameters or operation location information for single-stage judgment. In the loading and unloading channel, this often leads to unstable correspondences between container edges, corner outlines, and door frame features, and difficulties in linking lifting points, placement status, and fixing status within the same chain, making it difficult to meet the requirement of consistently achieving abnormal operation results. Existing technologies generally suffer from common shortcomings in the joint processing of detection input sets, global views, structural self-inspection results, hoisting points, placement status, and fixing status. These shortcomings include insufficient time alignment, inconsistent viewpoint registration, fragmentation between zoned detection models and loading / unloading correlation judgment processing, and difficulty in synchronizing historical anomaly records and loading / unloading sequences. This makes it difficult to form a consistent process in the loading / unloading channel, encompassing data acquisition, time alignment, viewpoint registration, structural judgment, loading / unloading correlation judgment processing, and historical correlation processing. Consequently, self-inspection reports and handling instructions are not sufficiently integrated with the actual loading / unloading process, affecting the verification, recording, and subsequent processing during loading / unloading operations. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a container loading and unloading self-inspection method based on multi-view perception, comprising: S100. Based on multi-view image acquisition, box-type parameters and operation position information, time alignment and frame filtering processing are performed to obtain the detection input set; S200. Based on the detection input set, perform edge extraction, corner piece contour and door frame feature extraction, and then perform viewpoint registration and box surface stitching to obtain a global view; S300. Based on the global view, perform partition mapping processing to obtain a partition detection model; S400. Based on the partition detection model, perform structural judgment processing to obtain structural self-inspection results; S500. Based on the self-inspection results of the structure, perform loading and unloading association judgment processing to obtain the operation abnormality result; S600. Based on the abnormal operation results, perform historical correlation processing to generate a self-inspection report and handling instructions.

[0004] Furthermore, the process of multi-view image acquisition, time alignment, and frame selection includes: The multi-view image acquisition includes: front-view, rear-view, left-view, right-view and top-view images are acquired by front-view, rear-view, left-view, right-view and top-view acquisition units respectively, and triggering devices issue trigger signals when the container enters the loading and unloading channel, the spreader enters the working position, the container passes through the middle of the channel and leaves the channel, and time synchronization devices write timestamps for each frame of image. The time alignment and frame filtering process includes: grouping the acquisition batches according to a unified event identifier, sorting the multi-view images under the same identifier by timestamp and matching the time difference between adjacent views, selecting the corresponding frames of the multi-views in the same acquisition period; then calculating the image sharpness, box coverage and occlusion ratio of each candidate frame, removing blurry frames, half-box frames and heavily occluded frames, retaining valid frames and writing valid frame tags. Based on the above processing, a detection input set is obtained, which includes front-view, back-view, left-view, right-view, and top-view images, as well as viewpoint identifiers, frame numbers, box numbers, box lengths, box heights, work position numbers, and valid frame markers.

[0005] Furthermore, the processes of edge extraction, corner contour extraction, and door frame feature extraction include: The edge extraction unit locates the top edge, bottom edge, left edge, and right edge from the main body area of ​​the box; when a break occurs in the edge in a certain view, the corresponding boundary of other views in the same processing batch is called for correction and a break repair mark is written. The contour extraction unit locates the top corner piece contour, bottom corner piece contour, four corner points of the door frame, and door seam boundary from the corner piece area and the door area; The feature verification unit compares the extracted top corner component outline, bottom corner component outline, four corner points of the door frame and door seam boundary with the standard shape constraints corresponding to the box length and box height. When the shape ratio or boundary direction deviates from the preset range, it is marked as a feature to be verified, and adjacent view images are called to re-extract the feature.

[0006] Furthermore, the viewpoint registration process includes: A unified pose coordinate system is constructed based on the top edge line, bottom edge line, top corner component outline, bottom corner component outline, and four corner points of the door frame. The unified pose coordinate system includes the corresponding position of each viewpoint to the main body of the box, the relative direction of the key boundaries, and the corresponding order of the corner points.

[0007] Furthermore, the process of splicing the container surfaces includes: The effective regions in the multi-view images are integrated according to the six-sided unfolding relationship of the box to generate a global view containing a region index. The region index includes the image source view, the stitching segment number, the box surface position code, and the boundary connection order. Based on the above processing, a global view is obtained, which includes global image number, unified pose coordinates, region index, viewpoint mapping relationship and spliced ​​segment number.

[0008] Furthermore, the partition mapping process includes: The boundary landing point sub-units are marked sequentially on the unfolded box surface according to the unified pose coordinates and area index, marking the partition boundaries of the box door area, side panel area, top panel area, corner fitting area and bottom connection area. Each partition is written with partition number, partition start and end boundary and partition source segment number. For each component, the corresponding subunit establishes relative position constraints for each zone: the door zone includes the relative position of the lock rod centerline and the door frame edge line, the relative position of the lock seat position and the lock rod centerline, and the relative position of the door seam boundary and the left and right door body boundaries; the side panel zone includes the relative position of the main boundary of the panel surface and the section boundary; the top panel zone includes the relative position of the front edge, middle, and rear edge and the top edge line; the corner piece zone includes the relative position of the corner piece outer contour and the adjacent edge line; the bottom connection zone includes the relative position of the bottom support position, the locking piece position, and the boundaries of the front connection zone and the rear connection zone. The threshold loading subunit loads the corresponding offset threshold and anomaly category identifier according to the partition number. The offset threshold refers to the permissible range of the structural position of each partition deviating from the standard position, and the anomaly category identifier refers to the anomaly type number of each partition that is allowed to enter the subsequent structural judgment and processing. Based on the above processing, a partition detection model is obtained that includes partition boundaries, relative position constraints of components, offset thresholds, anomaly category identifiers, partition numbers, region source fragment numbers, restricted markers, and local mapping markers.

[0009] Furthermore, the structural determination process includes: The pose determination subunit reads the unified pose coordinates and box-type parameters, measures the lateral offset, longitudinal offset, tilt offset and rotation offset of the box body relative to the standard box body position, and writes them into the pose offset record. The surface determination subunit performs a surface structure inspection on the front, middle, and rear sections of the left side plate and the front, middle, and rear sections, as well as the leading edge, middle, and trailing edge sections of the right side plate, identifying dents, bulges, cracks, perforations, and scratches, and writes the classification results into the surface determination record. The door lock determination subunit extracts the center direction of the left and right lock bars in the door area, compares the relative position of the lock bar center line with the lock seat position, and checks the continuity of the door gap boundary and the opening width. It outputs the door lock closure status record of not closed, not closed in place or lock bar misalignment. The corner piece determination subunit extracts the main outline of the corner piece in each corner piece area, compares the current outline with the outline of the top corner piece, the outline of the bottom corner piece and the adjacent edge lines, classifies the corner piece abnormalities as defects, deformations or cracks, and writes them into the corner piece determination record. When the pose determination results of adjacent partitions are inconsistent or the determination results of the left and right locking bar areas are inconsistent, the conflict verification subunit retrieves the splicing record and view mapping relationship to reposition and outputs a unified determination record. Based on the above processing, a structural self-inspection result is obtained, which includes the anomaly type, anomaly location, anomaly partition number, anomaly image fragment index, and structural risk level.

[0010] Furthermore, the process of determining and handling loading and unloading associations includes: The corresponding sub-unit for hoisting reads the corner anomalies, pose offsets, and corner contact positions and lifting tool action sequences from the structural self-inspection results and hoisting point records. When the abnormal part coincides with the corner contact position, or the offset direction is consistent with the force direction in the lifting tool action sequence, it is written into the hoisting association record, and the mark generation sub-unit generates a hoisting anomaly mark. The placement corresponding sub-unit reads the relevant abnormal parts of the bottom connection area in the structure self-inspection results and the support status of the front connection area and the support status of the rear connection area in the placement status record; when the abnormal part falls in the front connection area or the rear connection area and there is a lack of support, support offset or inconsistent front and rear support, or the position offset is tilt offset or longitudinal offset and there is a height difference between the front and rear support status, it is written into the placement association record, and the marking generation sub-unit generates a placement abnormal mark; The corresponding subunit reads the door lock closure status, related abnormal parts in the bottom connection area, and the position and locking action status of the locking component in the fixed status record from the structural self-inspection results; when the door lock closure status is not closed, not closed in place, or the lock rod is misaligned and the locking action has been executed, or when the abnormal part in the bottom connection area is inconsistent with the position of the locking component, it is written into the fixed association record, and the fixed abnormality mark is generated by the mark generation subunit; When the conflict verification subunit determines that the results of the left and right lock bar areas are inconsistent, it calls the four corner points of the door frame and the door gap boundary to re-verify the door face posture and output a unified door lock closure status record.

[0011] Furthermore, abnormal operation results include: The abnormal operation results include container number, operation node, hoisting abnormality mark, placement abnormality mark, fixing abnormality mark, abnormal part number, abnormal zone number, and operation abnormality category.

[0012] Furthermore, the process of handling historical associations includes: The historical retrieval sub-unit retrieves historical abnormal records by box number, abnormal part number, and time sequence, forming a batch linking the current record with the historical record; The hierarchical review subunit classifies the current anomaly as a continuing anomaly, a recurring anomaly, a newly added anomaly, or a repeated anomaly. When there is inconsistency in cross-view judgment, anomaly confidence is lower than the threshold, target partition is occluded, or historical anomaly record conflict occurs, supplementary sampling review is triggered, view registration, partition mapping processing, and structure judgment processing are re-executed, and the new results are written back and then reclassified. The part identification subunit generates the door area number, side panel area number, top panel area number, corner piece area number, or bottom connection area number according to the abnormal partition number, and writes it into the part identification field; The report generation subunit generates a self-inspection report containing the box number, operation node, abnormal part number, abnormality type, abnormality level, historical correlation results, review conclusion, and disposal instruction number; The instruction generation subunit generates handling instructions according to the abnormality level and operation node: when the operation abnormality result contains the hoisting abnormality mark, it generates a pause hoisting instruction or a manual review instruction; when it contains the placement abnormality mark, it generates a repositioning instruction or a reset detection instruction; when it contains the fixing abnormality mark, it generates a lock review instruction or a prohibition release instruction. The structured storage sub-unit completes structured storage according to box number, operation node, location identifier, anomaly type, anomaly level, historical correlation results and handling instructions, forming a closed-loop record.

[0013] The key innovations of this invention include: (1) Collect the front view, rear view, left view, right view, top view of the loading and unloading channel, box type parameters and operation position information, perform time alignment and frame filtering, generate a detection input set, and input the detection input set into edge extraction and contour extraction processing to extract the box edge, corner contour and door frame features, perform view registration and box surface splicing to generate a global view.

[0014] (2) Input the global view into the partition mapping process to map the door area, side panel area, top panel area, corner piece area and bottom connection area, establish a partition detection model, and input the partition detection model into the structure judgment process to identify the pose offset, surface damage, door lock closure status and corner piece abnormality, and generate the structure self-inspection result.

[0015] (3) Input the self-inspection results of the structure into the loading and unloading association judgment processing, combine the hoisting point, the placement status and the fixing status to generate the operation abnormal results, input the operation abnormal results into the historical association processing, associate the historical abnormal records and loading and unloading sequence, perform hierarchical review and location identification, and generate self-inspection reports and disposal instructions.

[0016] The following are its main beneficial effects: (1) By organizing the front view image, rear view image, left view image, right view image, top view image, box type parameters and operation position information in the same link, and completing edge extraction and contour extraction processing, view registration and box surface stitching on the basis of detection input set, the box edge, corner piece contour and door frame features in the global view form a corresponding relationship, which corresponds to the relationship dispersion problem caused by separate image processing in the existing scheme, and provides continuous input for subsequent partition mapping processing under loading and unloading channel.

[0017] (2) By incorporating the door area, side panel area, top panel area, corner piece area and bottom connection area into the same partition mapping process and structural judgment process link, the positional offset, surface damage, door lock closing status and corner piece abnormality are kept in correspondence with the abnormal partition number, which addresses the problems of unclear abnormal part location and scattered structural status judgment in the existing scheme, and enables the structural self-inspection results to directly enter the subsequent loading and unloading related judgment process.

[0018] (3) By performing loading and unloading association judgment processing on the structural self-inspection results with the hoisting point, placement status and fixing status, and continuing to perform historical association processing with historical abnormal records and loading and unloading sequence, the abnormal operation results, self-inspection reports and disposal instructions are kept in correspondence under the same operation node, which addresses the problem of the disconnect between structural status and loading and unloading status and the separation of records and disposal in the existing scheme, and forms a hierarchical review and part identification link for loading and unloading channels. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a container loading and unloading self-inspection method based on multi-view perception, provided as an embodiment of this application. Detailed Implementation

[0020] Example 1: Refer to Figure 1 This is a flowchart illustrating a container loading and unloading self-inspection method based on multi-view perception provided by an embodiment of the present invention. The process may include at least steps S100-S600: S100. Based on multi-view image acquisition, box-type parameters and operation position information, time alignment and frame filtering processing are performed to obtain the detection input set; S200. Based on the detection input set, perform edge extraction, corner piece contour and door frame feature extraction, and then perform viewpoint registration and box surface stitching to obtain a global view; S300. Based on the global view, perform partition mapping processing to obtain a partition detection model; S400. Based on the partition detection model, perform structural judgment processing to obtain structural self-inspection results; S500. Based on the self-inspection results of the structure, perform loading and unloading association judgment processing to obtain the operation abnormality result; S600. Based on the abnormal operation results, perform historical correlation processing to generate a self-inspection report and handling instructions.

[0021] S100. Based on multi-view image acquisition, box-type parameters and operation position information, time alignment and frame filtering processing are performed to obtain the detection input set; In this step, the front view, rear view, left view, right view, and top view images are acquired by the front view acquisition unit, rear view acquisition unit, left view acquisition unit, right view acquisition unit, and top view acquisition unit, respectively. The front view acquisition unit is positioned directly in front of the loading and unloading channel entrance, the rear view acquisition unit is positioned directly behind the loading and unloading channel exit, the left view acquisition unit and the right view acquisition unit are positioned on both sides of the loading and unloading channel, and the top view acquisition unit is positioned above the loading and unloading channel. Each acquisition unit includes an industrial camera, a mounting bracket, a triggering device, a time synchronization device, and a field controller. The industrial camera is responsible for continuous imaging. The triggering device is responsible for issuing acquisition trigger signals when a container enters the loading and unloading channel, a spreader enters the working position, a container passes through the middle of the channel, and a container leaves the loading and unloading channel. The time synchronization device is responsible for writing a timestamp to each frame of the image. The field controller is responsible for receiving the image stream, recording the acquisition status, and writing the viewpoint identifier and frame number. The container type parameters are derived from the basic container data issued by the operation management side, including the container number, length, and height. The operation location information is derived from the location data issued by the loading / unloading equipment side or the site management side, including the operation location number and the current operation node identifier. Therefore, the input sources for this step include on-site imaging data of the loading / unloading channel, the container type parameters, and the operation location information. The aforementioned data undergoes initial aggregation within the on-site controller before proceeding to subsequent time alignment and frame filtering processes.

[0022] Specifically, when the triggering device in the loading / unloading channel detects that the container has entered the channel entrance, the forward-looking acquisition unit first starts acquisition and writes the first frame image along with the current timestamp, current container number, and current work position number into the cache. When the container continues to move forward to the middle of the channel, the left-looking acquisition unit, the right-looking acquisition unit, and the top-looking acquisition unit simultaneously start acquisition, recording the side and top images under the same work event. When the container approaches the channel exit, the rear-looking acquisition unit starts acquisition to supplement the container's appearance image from the exit direction. If there is a hoisting work position on site, the triggering device also receives the lifting gear arrival signal and the placement completion signal, and adds trigger acquisition before the lifting gear is clamped, after the lifting gear is lifted, and after the container is placed, so that a complete image sequence of forward, rear, left, right, and top views is formed for the same container number under the same loading / unloading sequence. Understandably, the front view image is used to characterize the forward appearance of the container when it enters the channel, the rear view image is used to characterize the reverse appearance of the container when it leaves the channel, the left view image and the right view image are used to characterize the appearance of the side panel areas of the container, the top view image is used to characterize the appearance of the top panel area and the top corner area, the container shape parameters are used to define the outer shape boundary for subsequent container contour recognition, and the operation position information is used to define the operation space position and operation node corresponding to the same container number, so that the aforementioned image data can be associated with the same loading and unloading event.

[0023] The time alignment is jointly performed by the field controller and the background processing node. The field controller first groups the acquired batches according to a unified event identifier, which is generated by combining the container number, trigger time, and work location number. Then, the front, rear, left, right, and top views under the same unified event identifier are timestamped to form a multi-view candidate image sequence for the same container number. Next, based on the time difference between adjacent views, the container's movement order through the loading / unloading channel, and the current work node, alignment matching is performed on images from different views, selecting corresponding frames from the candidate image sequence belonging to the same acquisition period. If a view is missing a frame in the current acquisition period, a missing frame marker is recorded, and adjacent frames are retained for subsequent replacement. If the same container number is repeatedly triggered within a short period, the repeated batches are merged according to the unified event identifier, and a repeated trigger marker is written into the merge record. Through the above processing, the front, rear, left, right, and top views, container type parameters, and work location information are organized into a synchronized data object under the same event, for subsequent frame filtering and retrieval.

[0024] The frame filtering is performed after the time alignment is completed. The filtering criteria include image sharpness, container coverage, and occlusion percentage. Image sharpness is used to determine whether the container edges and structural outlines are clear in the current frame. Container coverage is used to determine whether the main body of the container is fully within the field of view in the current frame. Occlusion percentage is used to determine the degree of occlusion of key areas of the container by lifting equipment, vehicles, personnel, or passageway components. The background processing node calculates the sharpness level, coverage level, and occlusion percentage level for each candidate frame from each viewpoint, and removes blurry frames, half-container frames, and heavily occluded frames according to preset filtering rules. Valid frames are marked as valid, and invalid frames are marked as invalid. If consecutive frames from a certain viewpoint are all deemed invalid, adjacent time-series frames under the same unified event identifier are called for replacement, and the replacement source is written in the replacement record. If no valid frames are found after replacement, a missing marker is written in the viewpoint field, allowing the data from other viewpoints to continue flowing. Through this processing path, the system forms a stable combination of valid frames from multiple viewpoints without changing the original operating rhythm of the loading and unloading passageway.

[0025] In a practical scenario, the loading and unloading channel is located at the yard's entry and exit inspection position. A front-view acquisition unit and a rear-view acquisition unit are installed on the front and rear gantries of the inspection position, respectively. A left-view acquisition unit and a right-view acquisition unit are installed inside the channel's guardrail, and a top-view acquisition unit is installed on the lower surface of the gantry beam. When a container-carrying vehicle enters the inspection position, the entrance ground sensor sends a trigger signal. The front-view acquisition unit first acquires an image of the container door direction. As the vehicle continues to the middle of the channel, images are simultaneously acquired on the left and right sides and from the top. When the vehicle leaves the inspection position, the rear-view acquisition unit acquires an image of the exit direction. The field controller reads the container number, length, height, and work location number issued by the operation system, merges images of the same vehicle and container according to a unified event identifier, and performs timestamp sorting, synchronization matching, and valid frame filtering on the images from each perspective to obtain a set of front, rear, left, right, and top views corresponding to the same container number, along with associated fields. In this embodiment, the front view image, the rear view image, the left view image, the right view image, and the top view image together cover the passage of the box, and the box type parameters and the operation position information together define the box identity and field position relationship, satisfying the input requirements for subsequent edge extraction and contour extraction processing.

[0026] After time alignment and frame filtering, the detection input set is generated. The detection input set includes a viewpoint identifier, frame number, container number, container length, container height, and work location number, and retains the timestamp, valid frame marker, missing frame marker, duplicate trigger marker, and invalid reason marker corresponding to the aforementioned fields. Specifically, the viewpoint identifier distinguishes between front, rear, left, right, and top views; the frame number distinguishes the acquisition order within the same viewpoint; the container number binds to the same container object; the container length and height define the boundaries for subsequent contour recognition; and the work location number binds to the work location in the work space. This detection input set serves as the direct input for edge extraction and contour extraction processing in step S200, entering the extraction process for container edges, corner contours, and door frame features.

[0027] The technical effects of this step can be summarized as follows: This step establishes a collaborative acquisition link around the loading and unloading channel, encompassing forward, backward, left, right, and top views. It integrates the box-type parameters and the operation location information into the same event data object, providing a unified input basis for subsequent processing. This step moves time alignment and frame filtering to the acquisition stage. The detection input set already contains viewpoint relationships, temporal relationships, and valid frame relationships before entering S200, thus providing a stable data starting point for subsequent viewpoint registration and box surface stitching. This step also incorporates trigger records, missing frame records, and replacement records into the same data stream, facilitating subsequent structure determination, loading and unloading association determination, and historical association processing to continue along the same link.

[0028] S200. Based on the detection input set, perform edge extraction, corner piece contour and door frame feature extraction, and then perform viewpoint registration and box surface stitching to obtain a global view; The input source for this step is the detection input set generated by S100. The detection input set contains the front view image, rear view image, left view image, right view image, top view image, viewpoint identifier, frame number, box number, box length, box height and operation position number, and retains the timestamp, valid frame mark, missing frame mark, repeated trigger mark and invalid reason mark. Specifically, the detection input set is sent to the edge extraction and contour extraction processing link. The edge extraction and contour extraction processing consists of an image processing unit, an edge extraction unit, a contour extraction unit, a feature verification unit, a viewpoint registration unit, and a box surface stitching unit. The image processing unit is responsible for recombining multi-view image sequences according to box number, work location number, and timestamp. The edge extraction unit is responsible for locating the top edge line, bottom edge line, left edge line, and right edge line from the main body area of ​​the box. The contour extraction unit is responsible for locating the top corner piece contour, bottom corner piece contour, four corner points of the door frame, and door seam boundary from the corner piece area and the door area. The feature verification unit is responsible for comparing the extraction results with the standard shape constraints corresponding to the box length and box height. The viewpoint registration unit is responsible for establishing unified pose coordinates between each viewpoint. The box surface stitching unit is responsible for integrating the multi-view local images into the global view. Understandably, the box edge refers to the set of continuous boundary lines of the box's outer contour in the image, the corner contour refers to the corner shape boundary formed by the top corner contour and the bottom corner contour, and the door frame feature refers to the box door structure positioning information formed by the four corner points of the door frame and the door seam boundary. The above features together constitute the core input for the view registration and the box surface stitching.

[0029] Specifically, the image processing unit first reads the viewpoint identifiers and valid frame markers within the detection input set, merging front, rear, left, right, and top view images with the same box number and consistent job position number into the same processing batch. Then, it sorts the images by timestamp to form a multi-view image group arranged in the order of job execution. For processing batches with missing frame markers, the image processing unit calls up valid replacement images with the same box number, adjacent timestamps, and valid frame markers, and writes the replacement source into the processing record. For processing batches with duplicate trigger markers, the image processing unit selects a set of main images based on time intervals and box coverage, and writes the remaining images into a supplementary record for use by the feature verification unit. Subsequently, the edge extraction unit performs grayscale unification, boundary enhancement, and background suppression processing on the merged multi-view images. When the channel components, vehicle chassis, and lifting equipment components are occluded, the search range of the main body of the container is first limited by the container length and height. Then, continuous straight line boundaries and corner boundaries are extracted from the search range to obtain the top edge line, the bottom edge line, the left edge line, and the right edge line. The start position, end position, direction, and confidence of each edge line are recorded as edge fields. If an edge line is broken in a certain view, the edge extraction unit calls the corresponding boundaries of other views in the same processing batch for correction and writes a break repair mark in the edge field.

[0030] Furthermore, after completing the edge localization of the box body, the contour extraction unit performs contour recognition separately for the corner region and the door region. For the corner region, the contour extraction unit determines the top corner component search window and the bottom corner component search window based on the top edge line, the bottom edge line, the left edge line, and the right edge line. Within the search windows, it extracts the corner closed contour, the opening boundary, and the corner turning boundary to generate the top corner component contour and the bottom corner component contour. For the door region, the contour extraction unit extracts the rectangular door frame contour based on the door surface direction in the front view image or the rear view image, locates the four corner points of the door frame from the door frame contour, and then extracts the door seam boundary along the door body seam direction to generate the door frame feature. The feature verification unit then verifies the top corner piece outline, the bottom corner piece outline, the four corner points of the door frame, and the door seam boundary against the box length and the box height. When the corner piece shape ratio, the door frame width-to-height ratio, or the door seam boundary direction deviates from the preset range, the feature verification unit marks it as a feature to be verified and calls adjacent view images under the same box number to perform local extraction again. If conflicts still exist after verification, the current feature is retained and a conflict mark is added and written to the feature record. At this point, this step completes the extraction of the box edge, the corner piece outline, and the door frame features. The feature record includes edge field, corner piece field, door frame field, and conflict mark field for the view registration unit to continue to use.

[0031] In the viewpoint registration stage, the viewpoint registration unit reads the edge field, the corner piece field, the door frame field, the viewpoint identifier, and the box-type parameters. First, it establishes a corner point correspondence between the front view image, rear view image, left view image, right view image, and top view image. Then, it constructs a unified pose coordinate system based on the top edge line, bottom edge line, top corner piece outline, bottom corner piece outline, and the four corner points of the door frame. The unified pose coordinate system refers to a shared spatial positioning relationship established for the same box number, including the corresponding positions of each viewpoint to the main body of the box, the relative directions of each key boundary, and the corresponding order between each corner point. The viewpoint registration unit first uses the top view image as the top reference viewpoint, then attaches the left and right view images to the outer surfaces of both sides of the box, and subsequently attaches the front and rear view images to the door area or the rear area of ​​the box, thereby completing the unified positioning among multiple viewpoints. If the four corner points of the door frame are missing from a certain viewpoint, the viewpoint registration unit calls the top and bottom corner component contours from other views to supplement the corner point constraints. If the edge field and corner component field directions conflict within a certain viewpoint, the boundary direction is recalculated preferentially according to the corner component field and box-shaped parameters, and the recalculation result is written into the pose record. The pose record includes unified pose coordinates, viewpoint mapping relationship, corner point corresponding number, and conflict correction flag, wherein the unified pose coordinates are the key field for subsequent partition mapping processing by S300.

[0032] During the container splicing stage, the container splicing unit reads the unified pose coordinates and the viewpoint mapping relationship, integrates the effective areas in the multi-view images according to the six-sided unfolding relationship of the container, and generates the global view containing the region index. The region index refers to the source identifier and position identifier attached when the effective areas retained in the front view image, rear view image, left view image, right view image, and top view image are written into the same view object, specifically including the image source viewpoint, splicing segment number, container position code, and boundary connection order. In specific implementation, in the scenario of the yard hoisting channel, after the container vehicle passes through the loading and unloading channel and completes S100 acquisition, the background processing node first retrieves five view images of the same processing batch according to the container number, and then completes the edge line extraction, corner piece contour extraction, door frame four corner point extraction, and door seam boundary extraction. Subsequently, a unified pose coordinate is established, the top image is positioned to the top plate area, the left and right side images are positioned to the side plate area, and the front and rear images are positioned to the container door area or the rear area, finally generating a global view that can be used for region segmentation. If an overlapping area appears at the splicing boundary, the box splicing unit selects the main segment according to the boundary clarity and corner consistency, and writes the auxiliary segment information into the splicing record; if a gap appears at the splicing boundary, the gap mark is retained and the gap position is recorded for the S300's partition mapping processing to identify the available area and the restricted area.

[0033] After edge extraction and contour extraction, viewpoint registration, and box-face stitching, the global view is generated. The global view records the global image number, unified pose coordinates, region index, viewpoint mapping relationship, stitched segment number, and stitching record. The global image number, unified pose coordinates, and region index serve as direct inputs to "input the global view into the partition mapping process" in step S300, while the viewpoint mapping relationship and stitching record serve as fields called during partition boundary verification. Thus, the detection input set generated in step S100 completes the transformation from a multi-view image set to a unified pose image object within this step and flows along the step chain to step S300.

[0034] In summary, this step integrates multi-view images, box-shaped parameters, and operational positional relationships within the detection input set into the edge extraction and contour extraction processing chain, forming stable feature objects around the box body, corner regions, and door regions. This step establishes unified pose coordinates through viewpoint registration and then generates a global view with region indexes through box surface stitching, providing a unified image and positional basis for subsequent partition mapping processing. This step incorporates edge fields, corner fields, door frame fields, pose records, and stitching records into the same data object, maintaining the continuity of cross-step call relationships.

[0035] S300. Based on the global view, perform partition mapping processing to obtain a partition detection model; The input source for this step is the global view generated by S200. The global view contains a global image number, unified pose coordinates, region index, viewpoint mapping relationship, stitched segment number, and stitching record. Specifically, the global view is sent to the partition mapping processing link, which consists of a pose reading subunit, a region unfolding subunit, a boundary landing point subunit, a component mapping subunit, a threshold loading subunit, and a model generation subunit. The pose reading subunit is responsible for reading the unified pose coordinates and region index. The region unfolding subunit is responsible for unfolding the surface region of the box according to the six-sided unfolding relationship of the box. The boundary landing point subunit is responsible for calibrating the boundaries of each partition within the unfolded region. The component mapping subunit is responsible for establishing the correspondence between each partition and the box structure components. The threshold loading subunit is responsible for loading the offset threshold and anomaly category identifier corresponding to each partition. The model generation subunit is responsible for summarizing and generating the partition detection model. Understandably, the door area refers to the area containing the door body, door frame, lock rod, lock seat, and door gap; the side panel area refers to the main panel area of ​​the left and right side panels; the top panel area refers to the top panel area of ​​the box; the corner piece area refers to the corner area containing the top and bottom corner pieces; and the bottom connection area refers to the connection area at the bottom of the box related to hoisting, positioning, and locking. The above areas together constitute the regional basis for subsequent structural determination and processing.

[0036] Specifically, the pose reading subunit first retrieves the unified pose coordinates according to the global image number, and then checks the unfolded positions of the front view image, rear view image, left view image, right view image, and top view image in the global image according to the viewpoint mapping relationship. Subsequently, the region unfolding subunit unfolds the surface of the main body of the box according to the top edge line, bottom edge line, left edge line, right edge line, top corner component outline, bottom corner component outline, four corner points of the door frame, and door seam boundary. For the junction of splicing segments, the region unfolding subunit reads the main segment mark and gap mark in the splicing record. When the main segment exists, it continues to unfold along the boundary of the main segment. When the gap exists, it retains the gap area code and prohibits direct demarcation across the gap. After unfolding, the boundary landing point subunit first executes the door area landing point, dividing the left door area, right door area, left locking bar area, right locking bar area, and door seam area within the door surface area according to the four corner points of the door frame and the door seam boundary; then it executes the side panel area landing point, dividing the left side panel into the left side panel front section area, left side panel middle section area, and left side panel rear section area according to the left and right panel length direction in the unified pose coordinate system, and dividing the right side panel into the right side panel front section area, right side panel middle section area, and right side panel rear section area; then it executes the top panel area landing point, dividing the top panel area into the front edge area, middle section area, and rear edge area according to the front and back direction of the top panel; then it executes the corner piece area landing point, dividing the corner into the upper front corner piece area, upper rear corner piece area, lower front corner piece area, and lower rear corner piece area according to the position of the top corner piece outline and the bottom corner piece outline; finally, it executes the bottom connection area landing point, dividing the front connection area and rear connection area according to the front and back connection position of the bottom of the box. After each partition landing point is completed, the boundary landing point subunit writes the partition number, the partition start and end boundaries, and the partition source segment number for subsequent component subunits to call.

[0037] Furthermore, after the partitioning of the corresponding subunits is completed, relative position constraints are established for each partition. For the door area, the relative position constraints include the relative positions of the lock rod centerline and the door frame edge, the lock seat position and the lock rod centerline, and the door seam boundary and the left and right door body boundaries; for the side panel area, the relative position constraints include the relative positions of the main edge of the panel surface and the section boundary, and the relative positions of the panel surface rib direction and the section length direction; for the top panel area, the relative position constraints include the relative positions of the front edge, middle, and rear edge of the top panel and the top edge line; for the corner piece area, the relative position constraints include the relative positions of the corner piece outer contour and the adjacent edge line, and the relative positions of the corner piece opening boundary and the corner turning boundary; for the bottom connection area, the relative position constraints include the relative positions of the bottom support position, the locking piece position, and the boundaries of the front and rear connection areas. The threshold loading subunit then loads the corresponding offset threshold and anomaly category identifier according to the partition number. The offset threshold refers to the permissible range of deviation of the structural position of each partition from the standard position, and the anomaly category identifier refers to the anomaly type number of each partition that is allowed to enter subsequent structural judgment and processing. Specifically, the door area loads anomaly category identifiers related to the door lock closure status, the side panel area and the top panel area load anomaly category identifiers related to surface damage, the corner piece area loads anomaly category identifiers related to corner piece anomalies, and the bottom connection area loads anomaly category identifiers related to the support position and locking component position. Thus, the partition boundary, component relative position constraints, offset threshold, and anomaly category identifier are bound together within the same partition object.

[0038] In one engineering implementation, the global view corresponds to a container unloading operation scenario at a port yard. After the container completes the unified pose coordinate establishment via S200, the partition mapping processing link first reads the container length and height corresponding to the container number, and then unfolds the front, left and right side panels, top panel, and bottom connection positions in the global image. If the current global image comes from a standard dry cargo container, the container door area falls directly within the front area corresponding to the rear view image, the left and right side panels fall within the panel areas corresponding to the left and right view images respectively, the top panel area falls within the top area corresponding to the top view image, the corner pieces fall within the unfolded positions of the four top corners and four bottom corners respectively, and the bottom connection area is marked according to the two bottom connection positions at the front and rear. If there is a splicing gap in the current global image, the boundary landing point subunit writes a restricted marker around the gap and incorporates the restricted marker into the partition record; if there is an obstruction in the current container door area, the component corresponding subunit only establishes local constraints on the locking bar centerline, locking seat position, and door seam boundary for the unobstructed area, and writes a local mapping marker in the corresponding partition record. In this implementation, the partition mapping process does not change the unified pose coordinates, but only superimposes partition boundaries and component constraints on the global view, thereby maintaining consistency with the previous steps and continuously transmitting structural positional relationships to subsequent steps.

[0039] After the partition mapping process, the partition detection model is generated. The partition detection model includes partition boundaries, component relative position constraints, offset thresholds, and anomaly category identifiers, and includes partition numbers, region source segment numbers, restricted markers, and local mapping markers. Specifically, the partition boundaries define the image scope of the structural determination process in S400; the component relative position constraints define the determination relationships for pose offset, door lock closure status, and corner component anomalies in S400; the offset threshold defines the determination range for pose offset and component deviation in S400; and the anomaly category identifier defines the determination entry points for surface damage, door lock closure status, and corner component anomalies in S400. The partition detection model serves as the direct input to the structural determination process in S400, and proceeds along the step chain to the recognition process for pose offset, surface damage, door lock closure status, and corner component anomalies.

[0040] The key technical effect of this step is that it transforms the unified pose coordinates, region indexes, and stitching relationships within the global view into partitioned objects oriented towards the door area, side panel area, top panel area, corner component area, and bottom connection area, allowing subsequent structural determination processing to proceed on a unified basis. This step incorporates partition boundaries, component relative position constraints, offset thresholds, and anomaly category identifiers into the same model object, maintaining a continuous link from the global image to the structural determination entry point. Furthermore, this step integrates restricted markers and local mapping markers into the partition detection model, ensuring that subsequent steps continue processing according to the same rules even in scenarios involving occlusion, gaps, and fragment boundaries.

[0041] S400. Based on the partition detection model, perform structural judgment processing to obtain structural self-inspection results; The input source for this step is the partition detection model generated by S300. This model includes partition boundaries, component relative position constraints, offset thresholds, anomaly category identifiers, partition numbers, region source fragment numbers, restricted markers, and local mapping markers. It is associated with the corresponding image fragment in the global view through the region source fragment number. Specifically, the partition detection model is input into the structure determination processing chain. This chain consists of a region retrieval subunit, a pose determination subunit, a surface determination subunit, a door lock determination subunit, a corner component determination subunit, a conflict verification subunit, and a result aggregation subunit. The region retrieval subunit retrieves local image fragments according to the partition number and the region source fragment number. The pose determination subunit identifies pose offsets. The surface determination subunit identifies surface damage. The door lock determination subunit identifies door lock closure status. The corner component determination subunit identifies corner component anomalies. The conflict verification subunit handles determination conflicts corresponding to cross-partition, cross-viewpoint, and restricted marker issues. The result aggregation subunit generates the structure self-inspection result. Understandably, the pose offset refers to the deviation of the main body of the box from the standard position under a unified pose coordinate system; the surface damage refers to the abnormal state of the panel surface in the side panel area and the top panel area; the door lock closure state refers to the closure relationship between the left lock bar area, the right lock bar area, the lock seat position and the door gap area; and the corner piece abnormality refers to the abnormal state of the corner piece shape in the upper front corner piece area, the upper rear corner piece area, the lower front corner piece area and the lower rear corner piece area. The above four types of judgment objects together constitute the core content of the structural judgment processing.

[0042] Specifically, the region invocation subunit first reads the partition boundaries and partition numbers in the partition detection model, then extracts local image fragments of the corresponding regions from the global view according to the region source fragment numbers, and establishes judgment batches for the door area, side panel area, top panel area, corner piece area, and bottom connection area respectively. For partitions with the restricted marker, the region invocation subunit only extracts the effective areas without occlusion or gaps, and writes the effective area boundaries into the invocation record; for partitions with the local mapping marker, the region invocation subunit calls the boundary orientation of adjacent partitions as auxiliary constraints to keep the current partition judgment within the same unified pose coordinate system. After completing the region invocation, the pose judgment subunit reads the offset threshold and component relative position constraints in the partition detection model, compares the unified pose coordinates with the box-type parameters, measures the lateral deviation, longitudinal deviation, angular deviation, and end height difference relationship of the box body relative to the standard box body position, and divides the pose offset into lateral offset, longitudinal offset, tilt offset, and rotation offset. The lateral offset refers to the deviation of the main body of the container along the width direction; the longitudinal offset refers to the deviation of the main body of the container along the length direction; the tilt offset refers to the deviation where the two ends or sides of the container are uneven; and the rotational offset refers to the deviation where the main body of the container changes angle relative to the standard orientation. If the pose determination results of the same container number are inconsistent in adjacent partitions, the conflict verification subunit retrieves the splicing record and the viewpoint mapping relationship, repositions the boundary intersection of adjacent partitions, and then outputs a unified pose offset record.

[0043] Furthermore, the surface determination subunit performs panel structure inspection on the front section area of ​​the left side panel, the middle section area of ​​the left side panel, the rear section area of ​​the left side panel, the front section area of ​​the right side panel, the middle section area of ​​the right side panel, the rear section area of ​​the right side panel, as well as the leading edge area, the middle area, and the trailing edge area. The surface damage includes dents, bulges, cracks, perforations, and scratches. The dents refer to areas where the panel surface partially contracts inward and is accompanied by abrupt changes in edge shape. The bulges refer to areas where the panel surface partially bulges outward and changes the original flat contour. The cracks refer to linear cracks that appear on the surface or at the connection points of the panel. The perforations refer to areas where there is a through-hole in the panel surface. The scratches refer to band-shaped abnormal areas left by continuous scraping of the panel surface. In specific implementation, the surface determination subunit reads the partition boundaries and relative position constraints of components in each partition. First, it reconstructs the main boundary and texture direction of the board surface in local image segments. Then, it compares the continuity, edge straightness, and surface texture consistency of the current board surface shape with adjacent normal areas to obtain candidate damaged areas. Subsequently, it classifies the candidate damaged areas into the depression, bulge, crack, perforation, or scratch according to their contour shape, boundary direction, and depth relationship, and writes the classification results into the surface determination record. If the surface damage spans two adjacent segments, the main abnormal partition number is determined according to the segment where the damage center point is located, and the adjacent segments are written into the associated partition record.

[0044] Furthermore, the door lock determination subunit performs door lock closure status determination on the left lock bar area, the right lock bar area, and the door gap area. The door lock closure status is determined based on the lock bar centerline, lock seat position, and door gap boundary. The lock bar centerline refers to the center line of the lock bar body in the image, the lock seat position refers to the installation position of the lock bar corresponding to the door locking component, and the door gap boundary refers to the joint boundary between the left and right door areas. Specifically, the door lock determination subunit first extracts the centerlines of the left and right lock bars within the door area, then compares the relative positions of the lock bar centerlines and the lock seat positions, and simultaneously checks whether the door gap boundary is continuous, straight, or has an abnormal opening width, thereby determining the door lock closure status. The determination results of the door lock closure status include not closed, incomplete closure, and lock rod misalignment. Not closed means the lock rod and lock seat are not locked together and the door gap boundary is noticeably open. Incomplete closure means the lock rod is within the locking range but the door gap boundary still shows a partial opening or the left and right door edges are not flush. Lock rod misalignment means the relative direction of the lock rod centerline deviates from the relative position constraint of the components. If the determination results for the left and right lock rod areas are inconsistent, the conflict verification subunit calls the four corner points of the door frame and the door gap boundary to re-verify the door face posture and then outputs a unified door lock closure status record.

[0045] Further, the corner component determination subunit performs corner component structure checks on the upper front corner component area, the upper rear corner component area, the lower front corner component area, and the lower rear corner component area. Corner component anomalies are determined based on the integrity of the corner component's outer contour, the shape of the opening boundary, and the relative position of the corner. The integrity of the corner component's outer contour refers to the continuity of the corner component's main body boundary in a local image; the shape of the opening boundary refers to the boundary direction and missing state of the corner component's opening; and the relative position of the corner refers to the corresponding position of the corner component relative to the adjacent box edge line under a unified pose coordinate system. Specifically, the corner component determination subunit first extracts the corner component's main body contour of each corner component area, then compares the current contour with the top corner component contour, the bottom corner component contour, and the adjacent edge line. When boundary missing, boundary bending, abnormal opening expansion, or corner component position deviation is found, the corresponding corner component area is marked as an anomaly candidate area. Subsequently, based on the contour gap shape, boundary bending degree, and corner point deviation direction of the anomaly candidate area, the corner component anomalies are classified as defects, deformations, and cracks. The term "defect" refers to a solid missing portion of the corner piece's outer contour; "deformation" refers to an overall distorted state of the corner piece's outer contour or opening boundary; and "crack" refers to a linear crack at the corner piece's boundary. For the corner piece area corresponding to the restricted marker, the corner piece determination subunit only determines the visible boundary and writes the state of the invisible boundary into the local determination record.

[0046] In one engineering embodiment, when the container is hoisted by the yard crane and enters the pre-placement verification position, the partition detection model generated by S300 has given the partition boundaries and component relative position constraints of the door area, side panel area, top panel area, corner fitting area, and bottom connection area. The structural judgment processing link sequentially calls back local image segments according to the partition number. First, the pose judgment subunit reads the unified pose coordinates and container length and height to determine whether the main body of the container has lateral offset, longitudinal offset, tilt offset, and rotation offset after hoisting. Then, the surface judgment subunit checks whether there are dents, bulges, cracks, perforations, and scratches in the left and right side panels and top panel areas. Then, the door lock judgment subunit checks the correspondence between the left and right lock rods and lock seats in the door area and the state of the door gap boundary. Finally, the corner fitting judgment subunit checks the integrity of the outer contour of the corner fittings, the shape of the opening boundary, and the relative position of the corners in the four corner fitting areas. If a corner component area is obstructed by a lifting device, the conflict verification subunit retains the current visible boundary determination result and writes the corner component area into the local determination record and conflict marker record for continued use by the S600 during supplementary verification. Throughout the entire implementation chain, all determination results maintain a one-to-one correspondence with the partition number and the region source segment number, without changing the unified pose coordinates and partition boundaries given in the preceding steps.

[0047] After processing by the pose determination subunit, surface determination subunit, door lock determination subunit, and corner component determination subunit, the result summarization subunit generates the structural self-inspection result. The structural self-inspection result includes anomaly type, anomaly location, anomaly partition number, anomaly image fragment index, and structural risk level. The anomaly type records the specific categories of pose deviation, surface damage, door lock closure status, and corner component anomalies. The anomaly location records the corresponding locations in the left door area, right door area, left lock bar area, right lock bar area, door gap area, left side panel front section area, left side panel middle section area, left side panel rear section area, right side panel front section area, right side panel middle section area, right side panel rear section area, leading edge area, middle area, trailing edge area, upper front corner component area, upper rear corner component area, lower front corner component area, and lower rear corner component area. The anomaly partition number is used to bind the partition object generated by S300. The anomaly image fragment index is used to bind the local image fragment called in the current determination. The structural risk level indicates the determination level of the anomaly at the structural level. The structural self-inspection result serves as the direct input for "inputting the structural self-inspection result into the loading and unloading association judgment process" in S500, and is used for subsequent joint judgment of hoisting point, placement status and fixing status.

[0048] The technical effect of this step is summarized as follows: This step establishes a unified structural judgment link based on the aforementioned partition detection model, encompassing pose displacement, surface damage, door lock closure status, and corner component anomalies. This ensures that different types of anomalies are identified under unified pose coordinates and partition boundary constraints. This step incorporates the anomaly type, anomaly location, anomaly partition number, anomaly image fragment index, and structural risk level into the same structural self-inspection result, allowing subsequent loading / unloading related judgment processing to directly inherit the judgment criteria at the structural level. This step also incorporates restricted markers, local judgment records, and conflict markers into the processing, maintaining continuity in the judgment link under occlusion, gap, and boundary area scenarios.

[0049] S500. Based on the self-inspection results of the structure, perform loading and unloading association judgment processing to obtain the operation abnormality result; The input source for this step is the structural self-inspection result generated by S400. The structural self-inspection result includes anomaly type, anomaly location, anomaly partition number, anomaly image fragment index, and structural risk level. The anomaly partition number is used to maintain a correspondence with the partition detection model generated by S300. Specifically, the structural self-inspection results are input into the loading and unloading association judgment and processing link. The loading and unloading association judgment and processing link consists of an operation information access subunit, a hoisting correspondence subunit, a placement correspondence subunit, a fixing correspondence subunit, a mark generation subunit, and a result summary subunit. The operation information access subunit accesses the hoisting point, placement status, and fixing status. The hoisting correspondence subunit is responsible for reading the corner component abnormality and the positional offset and judging the correspondence with the hoisting point. The placement correspondence subunit is responsible for reading the abnormal part corresponding to the bottom connection area and judging the placement status. The fixing correspondence subunit is responsible for reading the door lock closure status and the position of the locking part in the bottom connection area and judging the fixing status. The mark generation subunit is responsible for generating hoisting abnormality marks, placement abnormality marks, and fixing abnormality marks. The result summary subunit is responsible for generating the operation abnormality results according to the aforementioned marks. Understandably, the lifting point refers to the record of the corresponding position of the lifting tool and the corner of the box body, the placement status refers to the record of the support and fit status after the box body is placed in the target position, and the fixing status refers to the record of the locking status after the box body is placed. The aforementioned three types of operation information, together with the structure self-inspection results, constitute the operation layer judgment input.

[0050] Specifically, the operation information access subunit reads the operation record corresponding to the current container when the container number and operation node are consistent. The operation record includes a hoisting point record, a placement status record, and a fixing status record. The hoisting point record is output by the position acquisition device during the process of the lifting tool entering the clamping position, lifting the lifting tool, and transferring the lifting tool. The content includes the current container number, operation node, corner contact position, and lifting tool action sequence. The placement status record is output by the container placement position detection device. The content includes the current container number, operation node, front connection area support status, and rear connection area support status. The fixing status record is output by the locking status acquisition device. The content includes the current container number, operation node, locking component position, and locking action status. The operation information access subunit pairs the aforementioned operation records with the structural self-inspection results according to the container number, operation node, and time sequence. If multiple operation nodes correspond to the same container number during the pairing process, the operation record closest to the current structural self-inspection result is selected according to the time sequence and written into the associated record. If the current operation record is missing, a missing mark is written and the structural abnormality record is retained for continued circulation.

[0051] Further, the lifting corresponding subunit reads the corner component anomaly, the pose offset, the abnormal location, and the abnormal zone number from the structural self-inspection results, and reads the corner contact position and the lifting tool action sequence from the lifting point record to establish a judgment link between the corner component anomaly, the pose offset, and the lifting point. Specifically, if the abnormal location falls within the upper front corner component area, upper rear corner component area, lower front corner component area, or lower rear corner component area, and the corresponding corner contact position coincides with the abnormal location, the lifting corresponding subunit writes the anomaly into the lifting association record. If the pose offset is a lateral offset, longitudinal offset, tilt offset, or rotational offset, and the offset direction is consistent with the force direction in the lifting tool action sequence, the offset is written into the lifting association record. Subsequently, the mark generation subunit generates lifting anomaly marks according to the lifting association record. The hoisting anomaly marker includes the container number, work node, anomaly part number, anomaly zone number, hoisting point number, and hoisting anomaly category. The hoisting anomaly category is recorded as a corner-related anomaly, an off-center hoisting anomaly, or a corner-pose superposition anomaly. If multiple corner anomalies exist for the same container number but correspond to only one hoisting point, the corner anomaly directly corresponding to that hoisting point is prioritized, and the remaining corner anomalies are written into the accompanying record.

[0052] Further, the placement corresponding subunit reads the relevant abnormal parts and abnormal partition numbers of the bottom connection area from the structural self-inspection results, and reads the support status of the front connection area and the support status of the rear connection area from the placement status record, establishing a correspondence between the bottom connection area and the placement status. Specifically, when the abnormal part falls in the front or rear connection area, and the corresponding area has a missing support, support offset, or inconsistent front and rear support status, the placement corresponding subunit writes the abnormality into the placement association record; when the pose offset is recorded as tilt offset or longitudinal offset in S400, and there is a height difference or contact sequence difference between the support status of the front and rear connection areas, the placement corresponding subunit simultaneously writes it into the placement association record. Subsequently, the tag generation subunit generates placement abnormal tags according to the placement association record. The placement abnormal tag includes box number, work node, abnormal part number, abnormal partition number, front connection area status, rear connection area status, and placement abnormality category, wherein the placement abnormality category is recorded as support position abnormality, front and rear connection inconsistency abnormality, or placement offset abnormality. If the placement status record shows that the box has not been placed yet, the corresponding placement sub-unit will only retain the preliminary record, not output the formal placement anomaly mark, and write the pending placement mark in the record.

[0053] Further, the fixed corresponding subunit reads the door lock closure status, related abnormal parts in the bottom connection area, and the abnormal partition number from the structural self-inspection results, and reads the locking component position and locking action status from the fixed status record to establish a joint judgment link between the door front locking relationship and the bottom locking relationship. Specifically, if the door lock closure status record shows that it is not closed, not fully closed, or the lock rod is misaligned, and the current fixed status record shows that the locking action has been performed, then the fixed corresponding subunit writes the door front abnormality into the fixed association record; if the abnormal part corresponding to the bottom connection area is inconsistent with the locking component position, or the locking component position falls outside the front connection area or the rear connection area, then it is also written into the fixed association record. Subsequently, the tag generation subunit generates a fixed abnormality tag according to the fixed association record. The fixed abnormality tag includes the box number, operation node, abnormal part number, abnormal partition number, locking component position, locking action status, and fixed abnormality category, wherein the fixed abnormality category record is door front locking abnormality, bottom locking abnormality, or door front and bottom overlapping abnormality. If both the front door locking anomaly and the bottom locking anomaly occur simultaneously for the same box number, the tag generation subunit writes the composite anomaly status into the same fixed anomaly tag and retains their respective source records for subsequent historical association processing.

[0054] In one engineering embodiment, after the yard crane lifts the container to the stacking position, the spreader action record forms the lifting point record corresponding to the current container number, the container drop support detection device forms the drop status record, and the locking component acquisition device forms the fixation status record. Simultaneously, the S400 outputs the structural self-inspection results for that container number. If there is a corner component abnormality in the upper front corner component area and the unified pose coordinate corresponds to a lateral offset, the lifting corresponding subunit matches the corner component abnormality and the lateral offset with the current lifting point record and outputs a lifting abnormality mark. If there is an abnormal part in the front connection area and the drop status record shows an abnormal support status in the front connection area, the drop corresponding subunit outputs a drop abnormality mark. If the door lock closure status record shows incomplete closure and the fixation status record shows that the locking action has been completed, the fixation corresponding subunit outputs a fixation abnormality mark. Throughout the implementation process, the three types of marks maintain a one-to-one correspondence with the container number, work node, abnormal part number, and abnormal zone number, without changing the original structural abnormality record output by the S400; only the work layer correspondence is superimposed on the original record.

[0055] After processing by the hoisting corresponding subunit, the placement corresponding subunit, and the fixing corresponding subunit, the result summarization subunit generates the operation anomaly result according to the hoisting anomaly mark, the placement anomaly mark, and the fixing anomaly mark. The operation anomaly result includes the container number, operation node, hoisting anomaly mark, placement anomaly mark, fixing anomaly mark, anomaly part number, anomaly zone number, and operation anomaly category. The hoisting anomaly mark is used to characterize the correspondence between the corner fitting anomaly and the pose offset and the hoisting point. The placement anomaly mark is used to characterize the correspondence between the bottom connection area and the placement state. The fixing anomaly mark is used to characterize the correspondence between the door lock closing state, the locking part position in the bottom connection area, and the fixing state. The operation anomaly category is used to merge the operation layer anomaly states of the current container number under the current operation node. The operation anomaly result serves as the direct input for "inputting the operation anomaly result into historical association processing" in S600, and continues to transmit the container number, the operation node, the anomaly part number, and the operation anomaly category to subsequent hierarchical review and part identification.

[0056] The key technical benefits of this step are that it establishes a correspondence between structural anomalies output by the S400 and hoisting locations, placement statuses, and fixing statuses under the same container number and work node, extending anomaly assessment from the structural layer to the operational layer. This step integrates hoisting anomaly markers, placement anomaly markers, and fixing anomaly markers into the same operational anomaly result, allowing subsequent historical correlation processing to directly inherit the operational layer's assessment criteria. Furthermore, this step maintains a parallel storage relationship between structural anomaly source records and operational information source records, enabling subsequent review processes to trace back to their respective input sources.

[0057] S600. Based on the abnormal operation results, perform historical correlation processing to generate a self-inspection report and handling instructions; The input source for this step is the operation anomaly result generated by S500. The operation anomaly result includes the box number, operation node, hoisting anomaly mark, placement anomaly mark, fixing anomaly mark, anomaly part number, anomaly zone number, and operation anomaly category. Specifically, the abnormal operation results are input into the historical association processing link, which consists of a historical retrieval subunit, a sequential correspondence subunit, a hierarchical review subunit, a location identification subunit, a report generation subunit, an instruction generation subunit, and a structured storage subunit. The historical retrieval subunit is responsible for retrieving historical abnormal records by container number, abnormal location number, operation node, and time sequence. The sequential correspondence subunit is responsible for matching the current abnormal operation results with the loading and unloading sequence. The hierarchical review subunit is responsible for classifying the abnormal operation results into continuing abnormalities, recurring abnormalities, newly added abnormalities, and duplicate abnormalities. The location identification subunit is responsible for generating location identifications by container door area number, side panel area number, top panel area number, corner fitting area number, and bottom connection area number. The report generation subunit is responsible for generating a self-inspection report. The instruction generation subunit is responsible for generating disposal instructions. The structured storage subunit is responsible for writing and saving according to fixed fields. Understandably, the historical anomaly record refers to the set of anomaly records formed under the previous operation nodes for the same container number; the loading and unloading sequence refers to the record of the sequential relationship between the loading and unloading nodes corresponding to the current container number; the hierarchical review refers to reviewing and classifying the correspondence between the current anomaly and the previous anomaly, the current node and the previous node, and the current part and the previous part; and the part identification refers to merging the anomaly part number into the corresponding number in the container door area number, side panel area number, top panel area number, corner piece area number, and bottom connection area number.

[0058] Specifically, after receiving the current container number and the current work node, the historical retrieval subunit first reads the historical ledger corresponding to the current container number, and then retrieves historical anomaly records for the same location according to the anomaly location number and time sequence, forming an associated batch of the current record and the historical records. The historical anomaly record stores the container number, work node, anomaly location number, anomaly type, anomaly level, historical association results, review conclusions, and handling instruction numbers. The time sequence is ordered according to the chronological order of the container number's previous entry, transfer, placement, and fixing processes. If the current anomaly location number is already recorded in the historical ledger, the historical retrieval subunit reads the anomaly type, anomaly level, and handling instruction number for that location under the preceding work node; if the current anomaly location number is not recorded in the historical ledger, a blank historical marker is written into the associated batch. The sequence-corresponding subunit then reads the sequential relationship between the current operation node and historical operation nodes, and sequentially correlates the current anomaly occurrence time with the past anomaly occurrence times. When the current anomaly occurs at the same loading / unloading node as a historical anomaly, it is written to the corresponding record for that node; when the current anomaly occurs at a subsequent loading / unloading node, it is written to the corresponding record for that subsequent node; and when the current anomaly occurs at a preceding loading / unloading node, it is written to the corresponding record for that preceding node. After the above processing, the current operation anomaly result, the historical anomaly record, and the loading / unloading sequence are organized into the same associated batch for use by the hierarchical review subunit.

[0059] Further, the hierarchical review subunit reads the anomaly type, anomaly location number, work node, and historical related records from the associated batch and performs anomaly classification. Specifically, when the current anomaly location number and current anomaly type are consistent with the location and type in the historical anomaly record, and the historical record shows that the anomaly existed in the previous work node without a resolution record, the current anomaly is determined to be a continuing anomaly; when the historical record shows that the same anomaly location number existed before, and the anomaly has been written into the resolution record or the handling completion record, and the same type of anomaly occurs again in the current work node, the current anomaly is determined to be a recurring anomaly; when there is no historical record of the same anomaly location number in the historical ledger, the current anomaly is determined to be a new anomaly; when there is the same anomaly location number and the same anomaly type in the historical ledger, and the anomaly occurs repeatedly in multiple adjacent work nodes, the current anomaly is determined to be a duplicate anomaly. After providing the classification result, the hierarchical review subunit also checks the conflict status between the current anomaly and the historical record. When there is inconsistency in cross-view judgment, anomaly confidence level below the threshold, target partition occlusion, or conflict of the historical anomaly record, supplementary sampling review is triggered. The supplementary sampling and verification process involves the verification scheduling subunit calling the viewpoint image corresponding to the conflict partition and re-executing the viewpoint registration in S200, the partition mapping process in S300, and the structure determination process in S400. The new round of structural anomaly records generated by the supplementary sampling and verification process are written back to the associated batch in this step. Then, the hierarchical verification subunit provides a new classification result for continuing anomalies, recurring anomalies, newly added anomalies, or repeated anomalies, and writes the final classification result into the historical association result field and the verification conclusion field.

[0060] Furthermore, after the hierarchical review subunit outputs the historical association results, the part identification subunit reads the current abnormal part number and abnormal partition number, first determines whether the abnormality belongs to the door area, side panel area, top panel area, corner piece area or bottom connection area, and then generates the corresponding part identification. Specifically, when the abnormal partition number corresponds to the left door area, right door area, left locking bar area, right locking bar area, or door gap area, a door area number is generated and written into the location identifier field; when the abnormal partition number corresponds to the front section of the left side panel, the middle section of the left side panel, the rear section of the left side panel, the front section of the right side panel, the middle section of the right side panel, or the rear section of the right side panel, a side panel area number is generated and written into the location identifier field; when the abnormal partition number corresponds to the front edge area, the middle area, or the rear edge area, a top panel area number is generated and written into the location identifier field; when the abnormal partition number corresponds to the upper front corner piece area, the upper rear corner piece area, the lower front corner piece area, or the lower rear corner piece area, a corner piece area number is generated and written into the location identifier field; when the abnormal partition number corresponds to the front connection area or the rear connection area, a bottom connection area number is generated and written into the location identifier field. If the same abnormal operation result contains multiple abnormal location numbers, the location identifier subunit generates a main location identifier according to the abnormality level and the order of the operation node, and writes the remaining location numbers into the associated location record.

[0061] In one engineering embodiment, when the same container number outputs a placement anomaly mark at the yard placement node and a fixed anomaly mark at the fixed node, the historical retrieval subunit first retrieves the historical anomaly records corresponding to the hoisting node, placement node, and fixed node for that container number, and then the sequential correspondence subunit determines that the current anomaly is in a subsequent node. If the historical ledger shows that the container number had an anomaly in the same bottom connection area in the previous round of operation and no cancellation record was written, the hierarchical review subunit determines the current anomaly as a continuing anomaly; if the historical ledger shows that the same bottom connection area has been dealt with and closed, and the same type of anomaly occurs again, it is determined as a recurring anomaly; if the historical ledger does not contain the current anomaly part number, it is determined as a new anomaly. If the current fixed anomaly mark comes from the door lock closed state, and the historical record shows that the door area has an inconsistent state across viewing angles, the hierarchical review subunit triggers supplementary review, retrieves the view image corresponding to the conflicting partition, and re-executes S200, S300, and S400. The review is completed only after the new structural anomaly record is written back. After the review is completed, the part identification subunit generates a door area number or a bottom connection area number according to the current abnormal partition number, and then the report generation subunit and the instruction generation subunit continue to process it.

[0062] Furthermore, the report generation subunit generates a self-inspection report after obtaining historical correlation results, review conclusions, and location identifiers. This self-inspection report includes the container number, work node, abnormal location number, abnormality type, abnormality level, historical correlation results, review conclusions, and handling instruction number, and simultaneously writes the location identifier field. The instruction generation subunit generates handling instructions according to the abnormality level and work node. Specifically, when the work abnormality result includes a hoisting abnormality mark, a pause hoisting instruction or a manual review instruction is generated; when the work abnormality result includes a placement abnormality mark, a repositioning instruction or a reset detection instruction is generated; when the work abnormality result includes a fixing abnormality mark, a lock review instruction or a prohibition on release instruction is generated. If the same container number contains two or three types of abnormality marks simultaneously under the same work node, the instruction generation subunit outputs the main handling instruction in the order of abnormality level and work node, and writes the remaining handling content into the supplementary instruction record. After receiving the self-inspection report and the handling instruction, the structured storage subunit performs structured storage according to the container number, operation node, location identifier, anomaly type, anomaly level, historical correlation results, and handling instruction. It retains the review conclusion field, the anomaly location number field, and the handling instruction number field, forming a closed-loop record corresponding to the current container number. The self-inspection report and the handling instruction are written into the historical ledger as the final output of this method. Simultaneously, when a supplementary review condition occurs, the corresponding processing links of S200, S300, and S400 are called back.

[0063] The technical effects of this step can be summarized as follows: This step connects the operational anomaly results output by the S500 with historical anomaly records and loading / unloading sequences into the same traceability link, extending anomaly judgment from the current node to past nodes. This step establishes a continuous data transmission relationship between hierarchical review, location identification, self-inspection reports, and handling instructions, ensuring that supplementary sampling review and final storage are within the same closed loop. This step also uniformly writes the anomaly classification results, location numbers, and handling content into the historical ledger, providing a direct entry point for subsequent retrieval of the same container number, location, and operational node.

Claims

1. A self-inspection method for container loading and unloading based on multi-view perception, characterized in that, include: S100. Based on multi-view image acquisition, box-type parameters and operation position information, time alignment and frame filtering processing are performed to obtain the detection input set; S200. Based on the detection input set, perform edge extraction, corner piece contour and door frame feature extraction, and then perform viewpoint registration and box surface stitching to obtain a global view; S300. Based on the global view, perform partition mapping processing to obtain a partition detection model; S400. Based on the partition detection model, perform structural judgment processing to obtain structural self-inspection results; S500. Based on the self-inspection results of the structure, perform loading and unloading association judgment processing to obtain the operation abnormality result; S600. Based on the abnormal operation results, perform historical correlation processing to generate a self-inspection report and handling instructions.

2. The method according to claim 1, characterized in that, The process of multi-view image acquisition, time alignment, and frame selection includes: The multi-view image acquisition includes: front-view, rear-view, left-view, right-view and top-view images are acquired by front-view, rear-view, left-view, right-view and top-view acquisition units respectively, and triggering devices issue trigger signals when the container enters the loading and unloading channel, the spreader enters the working position, the container passes through the middle of the channel and leaves the channel, and time synchronization devices write timestamps for each frame of image. The time alignment and frame filtering process includes: grouping the acquisition batches according to a unified event identifier, sorting the multi-view images under the same identifier by timestamp and matching the time difference between adjacent views, selecting the corresponding frames of the multi-views in the same acquisition period; then calculating the image sharpness, box coverage and occlusion ratio of each candidate frame, removing blurry frames, half-box frames and heavily occluded frames, retaining valid frames and writing valid frame tags. Based on the above processing, a detection input set is obtained, which includes front-view, back-view, left-view, right-view, and top-view images, as well as viewpoint identifiers, frame numbers, box numbers, box lengths, box heights, work position numbers, and valid frame markers.

3. The method according to claim 2, characterized in that, The process of edge extraction, corner contour and door frame feature extraction includes: The edge extraction unit locates the top edge, bottom edge, left edge, and right edge from the main body area of ​​the box; when a break occurs in the edge in a certain view, the corresponding boundary of other views in the same processing batch is called for correction and a break repair mark is written. The contour extraction unit locates the top corner piece contour, bottom corner piece contour, four corner points of the door frame, and door seam boundary from the corner piece area and the door area; The feature verification unit compares the extracted top corner component outline, bottom corner component outline, four corner points of the door frame and door seam boundary with the standard shape constraints corresponding to the box length and box height. When the shape ratio or boundary direction deviates from the preset range, it is marked as a feature to be verified, and adjacent view images are called to re-extract the feature.

4. The method according to claim 3, characterized in that, The process of viewpoint registration includes: A unified pose coordinate system is constructed based on the top edge line, bottom edge line, top corner component outline, bottom corner component outline, and four corner points of the door frame. The unified pose coordinate system includes the corresponding position of each viewpoint to the main body of the box, the relative direction of the key boundaries, and the corresponding order of the corner points.

5. The method according to claim 4, characterized in that, The process of splicing the box surface includes: The effective regions in the multi-view images are integrated according to the six-sided unfolding relationship of the box to generate a global view containing a region index. The region index includes the image source view, the stitching segment number, the box surface position code, and the boundary connection order. Based on the above processing, a global view is obtained, which includes global image number, unified pose coordinates, region index, viewpoint mapping relationship and spliced ​​segment number.

6. The method according to claim 5, characterized in that, The partition mapping process includes: The boundary landing point sub-units are marked sequentially on the unfolded box surface according to the unified pose coordinates and area index, marking the partition boundaries of the box door area, side panel area, top panel area, corner fitting area and bottom connection area. Each partition is written with partition number, partition start and end boundary and partition source segment number. For each component, the corresponding subunit establishes relative position constraints for each zone: the door zone includes the relative position of the lock rod centerline and the door frame edge line, the relative position of the lock seat position and the lock rod centerline, and the relative position of the door seam boundary and the left and right door body boundaries; the side panel zone includes the relative position of the main boundary of the panel surface and the section boundary; the top panel zone includes the relative position of the front edge, middle, and rear edge and the top edge line; the corner piece zone includes the relative position of the corner piece outer contour and the adjacent edge line; the bottom connection zone includes the relative position of the bottom support position, the locking piece position, and the boundaries of the front connection zone and the rear connection zone. The threshold loading subunit loads the corresponding offset threshold and anomaly category identifier according to the partition number. The offset threshold refers to the permissible range of the structural position of each partition deviating from the standard position, and the anomaly category identifier refers to the anomaly type number of each partition that is allowed to enter the subsequent structural judgment and processing. Based on the above processing, a partition detection model is obtained that includes partition boundaries, relative position constraints of components, offset thresholds, anomaly category identifiers, partition numbers, region source fragment numbers, restricted markers, and local mapping markers.

7. The method according to claim 1, characterized in that, The structural determination process includes: The pose determination subunit reads the unified pose coordinates and box-type parameters, measures the lateral offset, longitudinal offset, tilt offset and rotation offset of the box body relative to the standard box body position, and writes them into the pose offset record. The surface determination subunit performs a surface structure inspection on the front, middle, and rear sections of the left side plate and the front, middle, and rear sections, as well as the leading edge, middle, and trailing edge sections of the right side plate, identifying dents, bulges, cracks, perforations, and scratches, and writes the classification results into the surface determination record. The door lock determination subunit extracts the center direction of the left and right lock bars in the door area, compares the relative position of the lock bar center line with the lock seat position, and checks the continuity of the door gap boundary and the opening width. It outputs the door lock closure status record of not closed, not closed in place or lock bar misalignment. The corner piece determination subunit extracts the main outline of the corner piece in each corner piece area, compares the current outline with the outline of the top corner piece, the outline of the bottom corner piece and the adjacent edge lines, classifies the corner piece abnormalities as defects, deformations or cracks, and writes them into the corner piece determination record. When the pose determination results of adjacent partitions are inconsistent or the determination results of the left and right locking bar areas are inconsistent, the conflict verification subunit retrieves the splicing record and view mapping relationship to reposition and outputs a unified determination record. Based on the above processing, a structural self-inspection result is obtained, which includes the anomaly type, anomaly location, anomaly partition number, anomaly image fragment index, and structural risk level.

8. The method according to claim 7, characterized in that, The process of determining and handling loading and unloading associations includes: The corresponding sub-unit for hoisting reads the corner anomalies, pose offsets, and corner contact positions and lifting tool action sequences from the structural self-inspection results and hoisting point records. When the abnormal part coincides with the corner contact position, or the offset direction is consistent with the force direction in the lifting tool action sequence, it is written into the hoisting association record, and the mark generation sub-unit generates a hoisting anomaly mark. The placement corresponding sub-unit reads the relevant abnormal parts of the bottom connection area in the structure self-inspection results and the support status of the front connection area and the support status of the rear connection area in the placement status record; when the abnormal part falls in the front connection area or the rear connection area and there is a lack of support, support offset or inconsistent front and rear support, or the position offset is tilt offset or longitudinal offset and there is a height difference between the front and rear support status, it is written into the placement association record, and the marking generation sub-unit generates a placement abnormal mark; The corresponding subunit reads the door lock closure status, related abnormal parts in the bottom connection area, and the position and locking action status of the locking component in the fixed status record from the structural self-inspection results; when the door lock closure status is not closed, not closed in place, or the lock rod is misaligned and the locking action has been executed, or when the abnormal part in the bottom connection area is inconsistent with the position of the locking component, it is written into the fixed association record, and the fixed abnormality mark is generated by the mark generation subunit; When the conflict verification subunit determines that the results of the left and right lock bar areas are inconsistent, it calls the four corner points of the door frame and the door gap boundary to re-verify the door face posture and output a unified door lock closure status record.

9. The method according to claim 8, characterized in that, Abnormal job results include: The abnormal operation results include container number, operation node, hoisting abnormality mark, placement abnormality mark, fixing abnormality mark, abnormal part number, abnormal zone number, and operation abnormality category.

10. The method according to claim 9, characterized in that, The process of historical association processing includes: The historical retrieval sub-unit retrieves historical abnormal records by box number, abnormal part number, and time sequence, forming a batch linking the current record with the historical record; The hierarchical review subunit classifies the current anomaly as a continuing anomaly, a recurring anomaly, a newly added anomaly, or a repeated anomaly. When there is inconsistency in cross-view judgment, anomaly confidence is lower than the threshold, target partition is occluded, or historical anomaly record conflict occurs, supplementary sampling review is triggered, view registration, partition mapping processing, and structure judgment processing are re-executed, and the new results are written back and then reclassified. The part identification subunit generates the door area number, side panel area number, top panel area number, corner piece area number, or bottom connection area number according to the abnormal partition number, and writes it into the part identification field; The report generation subunit generates a self-inspection report containing the box number, operation node, abnormal part number, abnormality type, abnormality level, historical correlation results, review conclusion, and disposal instruction number; The instruction generation subunit generates handling instructions according to the abnormality level and operation node: when the operation abnormality result contains the hoisting abnormality mark, it generates a pause hoisting instruction or a manual review instruction; when it contains the placement abnormality mark, it generates a repositioning instruction or a reset detection instruction; when it contains the fixing abnormality mark, it generates a lock review instruction or a prohibition release instruction. The structured storage sub-unit completes structured storage according to box number, operation node, location identifier, anomaly type, anomaly level, historical correlation results and handling instructions, forming a closed-loop record.

Images