Container inspection method and system thereof

Abstract

The application provides a container inspection method and system. The method comprises: performing non-contact scanning on the internal and external state of the container of an unmanned truck through a multi-modal perception system, and outputting multi-modal perception data of the container; performing feature extraction and feature splicing based on the multi-modal perception data to generate a multi-dimensional fusion feature vector; inputting the multi-dimensional fusion feature vector into a deep regression detection head, a position positioning detection head and a defect classification detection head for parallel calculation; performing inner product operation on the depth weight and the multi-dimensional fusion feature vector by the deep regression detection head to obtain a defect depth value; performing inner product operation on the position weight and the multi-dimensional fusion feature vector by the position positioning detection head to obtain a defect position value; performing inner product operation on the weight corresponding to each type of defect and the multi-dimensional fusion feature vector by the defect classification detection head to obtain a defect value of each type of defect, and outputting a confidence score of each type of defect detection result through a classifier; and determining the good or bad of the container according to the confidence score.

Description

Technical Field

[0001] This application relates to the field of container inspection technology, and in particular to a container inspection method and system. Background Technology

[0002] Traditionally, container inspection when container trucks pick up empty containers or collect export containers is a crucial part of the port operation process, but it is inefficient and carries certain risks. The specific process typically involves: after the truck driver drives the vehicle to the designated gate or inspection area, they must personally get out of the vehicle and manually open the container door. Then, the inspector, together with the driver, conducts a comprehensive inspection of the container after the door is opened to determine whether it is a good or bad container. They then inform the control room personnel via walkie-talkie or tablet buttons to maintain the current operation or issue a new task to the repair area.

[0003] The traditional container inspection process at inland river terminals relies on manual entry into the container for visual inspection, which is inefficient and risky, and is no longer compatible with the automated operation system of unmanned trucks. Summary of the Invention

[0004] The purpose of this application is to provide a container inspection method and system that can solve at least one of the technical problems mentioned in the prior art.

[0005] One aspect of this application provides a container inspection method. The method includes: activating a multimodal perception system deployed at the inspection station to perform non-contact scanning of the internal and external states of the unmanned container truck container, and outputting multimodal perception data of the container; using a multimodal fusion neural network model to extract and concatenate features based on the multimodal perception data to generate a multidimensional fusion feature vector representing defects; inputting the multidimensional fusion feature vector into three detection heads of the multimodal fusion neural network model for parallel computation, the three detection heads including a depth regression detection head, a location positioning detection head, and a defect classification detection head; the depth regression detection head performing an inner product operation with the depth weight and the multidimensional fusion feature vector to obtain a defect depth value; the location positioning detection head performing an inner product operation with the location weight and the multidimensional fusion feature vector to obtain a defect location value; and the defect classification detection head performing an inner product operation with the corresponding weights of various defects and the multidimensional fusion feature vector to obtain various defect values; feeding the various defect values ​​into the classifier in the multimodal fusion neural network model and outputting confidence scores for various defect detection results; and determining the quality of the container based on the confidence scores of various defect detection results.

[0006] Furthermore, the non-contact scanning of the internal and external states of the unmanned container truck container by activating the multimodal perception system deployed on the inspection gantry at the inspection station and outputting the multimodal perception data of the container includes: reconstructing the three-dimensional point cloud of the container's hexahedron by activating the binocular structured light 3D scanner deployed on the inspection gantry at the inspection station to detect geometric defects of the container; acquiring hyperspectral images of the container's surface using a hyperspectral imager to identify surface damage of the container; acquiring thermal maps of the container's interior using an infrared thermal imager to detect abnormal temperature zones inside the container; and detecting the gas inside the container using a miniature gas sensor array to obtain gas concentration time-series data. The multimodal perception data includes the three-dimensional point cloud, the hyperspectral image, the thermal map, and the gas concentration time-series data.

[0007] Furthermore, the method also includes: using a lidar and a high-definition wide-angle camera deployed on the inspection gantry to assist in positioning the relative position of the container and the inspection gantry, so as to ensure stable scanning posture.

[0008] Furthermore, determining the quality of the container based on the confidence scores of various defect detection results includes: automatically determining the container as good when the confidence score is greater than or equal to a preset first score; automatically determining the container as bad when the confidence score is less than a preset second score, wherein the first score is greater than the second score; and automatically marking the container as a container to be reviewed and pushing it to a remote inspector workstation for remote manual confirmation when the confidence score is greater than or equal to the second score and less than the first score.

[0009] Furthermore, the method also includes: when the confidence score is lower than a preset second score, automatically triggering the multimodal perception system to perform a secondary scan and re-examination of the container from more perspectives.

[0010] Furthermore, the method also includes: constructing a dual electronic fence at the inspection station based on millimeter-wave radar and ultra-wideband positioning technology; when any personnel enter the danger zone defined by the electronic fence, scanning is immediately paused and an alarm is triggered.

[0011] Furthermore, the method also includes: generating a unique digital fingerprint for the entire inspection process data after each container inspection is completed, solidifying the data and forming a traceable data evidence chain; uploading the data evidence chain to the port production scheduling system to form a full life cycle inspection file for the container, which can be accessed by third-party systems according to their permissions.

[0012] Furthermore, the method also includes: during the scanning of the container, controlling the power lock and wheel lock of the unmanned truck carrying the container.

[0013] Another aspect of this application provides a container inspection system. The container inspection system includes a multimodal perception system and an edge intelligent decision-making system. The multimodal perception system is deployed on an inspection gantry at the inspection station, used to perform contactless scanning of the internal and external states of unmanned truck containers and output multimodal perception data of the containers. The edge intelligent decision-making system is deployed on the inspection gantry and includes an AI inspection inference engine, which includes a multimodal fusion neural network model. The multimodal fusion neural network model includes a feature extraction and fusion module, three detection heads, and a classifier. The three detection heads include a depth regression detection head, a location detection head, and a defect classification detection head. The feature extraction and fusion module extracts and concatenates features based on the multimodal sensing data output by the multimodal sensing system to generate a multidimensional fusion feature vector representing defects. The depth regression detection head performs an inner product operation between the depth weights and the multidimensional fusion feature vector to obtain the defect depth value. The location detection head performs an inner product operation between the location weights and the multidimensional fusion feature vector to obtain the defect location value. The defect classification detection head performs an inner product operation between the weights corresponding to each type of defect and the multidimensional fusion feature vector to obtain the value of each type of defect. The classifier outputs confidence scores for each type of defect detection result based on the defect values. The AI ​​container inspection inference engine determines the quality of the container based on the confidence scores of the various defect detection results.

[0014] Furthermore, the multimodal perception system includes a binocular structured light 3D scanner, a hyperspectral imager, an infrared thermal imager, and a miniature gas sensor array. The binocular structured light 3D scanner is used to reconstruct the 3D point cloud of the container's hexahedron, thereby detecting geometric defects in the container. The hyperspectral imager is used to acquire hyperspectral images of the container's surface, thereby identifying surface damage. The infrared thermal imager is used to acquire thermal maps of the container's interior, thereby detecting abnormal temperature zones within the container. The miniature gas sensor array is used to detect the gas inside the container and acquire time-series gas concentration data. The multimodal perception data includes the 3D point cloud, the hyperspectral image, the thermal map, and the time-series gas concentration data.

[0015] Furthermore, the AI ​​container inspection inference engine is used to automatically trigger the multimodal perception system to perform a secondary scan and re-inspection of the container from more perspectives when the confidence score is lower than a preset first score.

[0016] The container inspection method and system of one or more embodiments of this application use a multimodal perception system deployed on the inspection gantry at the inspection station to perform non-contact scanning of the internal and external status of unmanned container trucks. This enables unmanned container trucks to automatically complete a comprehensive inspection of the internal and external status of the container without the driver or inspector entering the container.

[0017] In addition, the container inspection method and system of one or more embodiments of this application can automatically judge the quality of the container based on the confidence score of the defect detection results, without relying on manual clicking of good / bad containers. Attached Figure Description

[0018] Figure 1 This is a flowchart of a container inspection method according to one embodiment of this application.

[0019] Figure 2 This is a schematic block diagram of a container inspection system according to an embodiment of this application.

[0020] Figure 3 This is a diagram of a port intelligent transportation system architecture according to an embodiment of this application.

[0021] Figure 4 This is a flowchart illustrating an embodiment of the automated container inspection process for unmanned trucks according to this application.

[0022] Figure 5 This is a site diagram of one embodiment of this application. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses consistent with some aspects of this application as detailed in the appended claims.

[0024] Although some ports have attempted to use cameras and AI to identify external damage to containers, this approach suffers from the following fundamental flaws: 1. It can only detect defects on the outside of the enclosure and cannot detect the condition inside the enclosure, such as dampness, dirt, foreign objects, structural deformation, etc. 2. It requires manual clicking of "good / bad container", failing to achieve full automation and violating the original design intent of the unmanned truck system; 3. The lack of a structured verification mechanism makes it impossible to distinguish between "real damage" and "temporary stains" on containers, and there is a lack of a traceable and quantifiable chain of evidence. 4. The system is not integrated with the TOS (Terminal Operating System, Port Production Scheduling System) for closed-loop control, and the inspection results cannot automatically trigger task scheduling. 5. The operation requires manual intervention and poses a risk of personal injury.

[0025] In view of this, this application provides an improved container inspection method and system, which can solve at least one of the technical problems mentioned in the above-mentioned related technologies.

[0026] The container inspection methods and systems of various embodiments of this application will be described in detail below with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementation methods can be combined with each other.

[0027] This application provides a method for inspecting containers. Figure 1 A flowchart illustrating a container inspection method according to an embodiment of this application is provided. Figure 1 As shown, a container inspection method according to one embodiment of this application may include steps S1 to S5.

[0028] In step S1, the multimodal perception system of the inspection gantry deployed at the inspection station can be activated to perform contactless scanning of the internal and external status of the unmanned container truck container and output the multimodal perception data of the container.

[0029] In some embodiments, step S1 may further include steps S11 to S14.

[0030] In step S11, the binocular structured light 3D scanner of the inspection gantry deployed at the inspection station can be activated to complete the 3D point cloud reconstruction of the six-sided container, which is used to detect geometric defects of the container.

[0031] A binocular structured light 3D scanner (wavelength, for example, 850nm) can complete the 3D point cloud reconstruction of a container's six sides in 0.5 seconds with an accuracy of ±0.2mm. It can detect geometric anomalies such as dents, deformations, weld cracks, and missing corner pieces in the container.

[0032] In step S12, a hyperspectral image of the container surface can be acquired using a hyperspectral imager to identify damage to the container surface.

[0033] Hyperspectral imagers (wavelengths such as 400-1000nm) can identify oil stains, rust, graffiti, and chemical residues on the surface of containers, and can distinguish between "natural aging" and "human damage" of containers.

[0034] In step S13, an infrared thermal imager can be used to collect a thermal map of the container's interior to detect abnormal temperature zones inside the container.

[0035] Infrared thermal imagers can detect abnormal temperature areas inside the chamber, such as humid areas or residual heat sources.

[0036] In step S14, a miniature gas sensor array can be used to detect the gas inside the container and obtain time-series data of gas concentration.

[0037] Miniature gas sensor arrays can be used to detect substances such as NH3 (ammonia), VOCs (Volatile Organic Compounds), and H2O (water vapor), and to sense odors, chemical leaks, and moldy gases inside the chamber.

[0038] The multimodal sensing data of the container output in step S1 may include three-dimensional point cloud, hyperspectral image, heat map and gas concentration time series data.

[0039] In some embodiments, the container inspection method of this application may further include step S15.

[0040] In step S15, the relative position of the container and the inspection gantry can be assisted by the lidar and high-definition wide-angle camera deployed on the inspection gantry, thereby ensuring the stability of the scanning posture.

[0041] In step S2, a multimodal fusion neural network model can extract and concatenate features based on multimodal perception data to generate a multidimensional fusion feature vector representing the defect.

[0042] Multimodal fusion neural network models can include, for example, CNNs (Convolutional Neural Networks) and Transformers (Transformer Networks). The input to a multimodal fusion neural network model can be a 3D point cloud obtained by a binocular structured light 3D scanner, a hyperspectral image acquired by a hyperspectral imager, a thermal map acquired by an infrared thermal imager, or time-series gas concentration data obtained by a miniature gas sensor array.

[0043] In step S3, the multidimensional fusion feature vector generated in step S2 can be input into the three detection heads of the multimodal fusion neural network model for parallel computation, so as to obtain the defect depth value, defect location value and various defect values ​​respectively.

[0044] Specifically, the three detection heads include a depth regression detection head, a location localization detection head, and a defect classification detection head. The depth regression detection head calculates the defect depth value by performing an inner product operation between the depth weights and the multi-dimensional fusion feature vector; the location localization detection head calculates the defect location value by performing an inner product operation between the location weights and the multi-dimensional fusion feature vector; and the defect classification detection head calculates the value of each type of defect by performing an inner product operation between the corresponding weights of each type of defect and the multi-dimensional fusion feature vector.

[0045] Defects may include, but are not limited to, defects in the box structure and appearance, defects in the box door and seal, defects in the box surface, internal dampness, foreign object residue, and odor inside the box.

[0046] Defects in the enclosure structure and appearance may include: dents, deformations, bulges, etc. on the side panels, top panels, and end panels, affecting structural strength and sealing; holes, cracks, and weld breaks in the enclosure, leading to water leakage; corrosion, perforation, or detachment at the top, side panel welds, bottom beams, and corner post bases, weakening structural strength; cracks, deformation, or missing corner fittings, and bent corner posts, affecting hoisting and stacking safety; damage to the upper and lower side beams and end beams, bending, breakage, or weld breaks, reducing load-bearing capacity.

[0047] Defects in the cabinet door and seals may include: door deformation, inability to close tightly, door warping, misalignment, and failure of weather sealing; aging, detachment, or breakage of the door rubber strip, resulting in loss of waterproof function; damage to the door lock mechanism, such as deformation, breakage, or jamming of the lock rod, hinge, or lock seat, making it impossible to lock or seal; and rust or deformation of the door seal seat, leading to poor sealing.

[0048] Surface defects of the enclosure may include, for example, rust, coating peeling, paint loss, and flaking; oil stains, chemicals, and other foreign matter residues on the enclosure surface.

[0049] Odor defects inside the container may include, for example, chemical leaks inside the container, or the presence of moldy gases or other unusual odors.

[0050] In step S4, the various defect values ​​obtained in step S3 can be fed into the classifier (e.g., Softmax classifier) ​​in the multimodal fusion neural network model. After the classifier performs normalization operations, it outputs the confidence scores of the various defect detection results.

[0051] The confidence scores for various defect detection results can be, for example, scores in the range of 0-1.

[0052] In step S5, the quality of the container can be determined based on the confidence scores of the various defect detection results in step S4.

[0053] For ease of understanding, a specific example of the container inspection method of this application is given below.

[0054] For example, taking the multi-sensor fusion detection of "metal box wall indentation" as an example, the data source may include multimodal sensing data obtained from 2D industrial cameras (capturing texture / light and shadow) and 3D laser equipment (measuring depth / curvature).

[0055] These multimodal sensing data are input into a multimodal fusion neural network model. The model then extracts and concatenates features from this data to generate a multidimensional fusion feature vector F. This feature vector F is a comprehensive encoding of geometry, texture, and spatial location. Different dimensions of the multidimensional fusion feature vector F implicitly represent different metrics.

[0056] For example, F[0~50] can represent three-dimensional geometry (such as curvature changes, point cloud fitting errors); F[51~100] can represent two-dimensional texture anomalies (such as shadow gradients, reflective abrupt changes); F[101~120] can represent spatial location context (such as whether it is close to a weld or edge).

[0057] Assuming the 3D data is very strong (depth 2.5mm), but the 2D texture shows only normal weld slag shadows (texture anomaly score is extremely low). The resulting multidimensional fused feature vector F has a high geometric dimension and a low texture dimension.

[0058] The multidimensional fused feature vector F is input into three detection heads in the multimodal fusion neural network model: a depth regression detection head, a location localization detection head, and a defect classification detection head. The depth regression detection head has a depth weight matrix W_depth, the location localization detection head has a location weight matrix W_box, and the defect classification detection head has a weight matrix W_type corresponding to each type of defect.

[0059] The depth regression detection head can obtain the defect depth value by performing an inner product operation between the depth weight matrix W_depth and the multidimensional fusion feature vector F. For example, assuming that depth is mainly determined by geometry, the depth weight matrix W_depth has a very large weight in the geometric dimension F [0~50] (e.g., 0.9) and a very small weight in the texture dimension F [51~100] (e.g., 0.01). Therefore, after the inner product operation, the defect depth value is obtained as 0.9*F[0~50] + 0.01*F[51~100], outputting 2.5mm.

[0060] The location detection head can perform an inner product operation between the location weight matrix W_box and the multidimensional fusion feature vector F to obtain the numerical value of the defect location. For example, the location weight matrix W_box has the largest weight on F[101~120] (spatial location), which directly maps the (x, y) coordinates and length and width of the defect center on the image.

[0061] The defect classification and detection head performs an inner product operation between the weight matrix W_type corresponding to various defects (such as dents or scratches) and the multi-dimensional fusion feature vector F to obtain the numerical values ​​of various defects. These numerical values ​​are then fed into the classifier in the multi-modal fusion neural network model to obtain the confidence scores of the defect detection results. To distinguish between "dents" and "scratches," the weight matrix W_type corresponding to various defects combines geometric dimensions (dent depth) and texture dimensions (scratches have continuous lines). Assuming that the geometric values ​​are large and the texture values ​​are small in the multi-dimensional fusion feature vector F, the dot product result is biased towards the "dent" class with a probability of 92%. When combining geometric and texture dimensions, the weight matrix W_type corresponding to various defects learns the rule that "only when both the geometric and texture dimensions are activated to a certain threshold is it judged as a true defect." The final output confidence score is only 45%, below the threshold, so the system rejects the defect. The container inspection method of this application can automatically complete the comprehensive inspection of the internal and external conditions of unmanned container trucks without the need for drivers or inspectors to enter the container.

[0062] The container inspection method proposed in this application achieves four-dimensional state perception of the container through three-dimensional structured light reconstruction, multispectral imaging, and gas sensing fusion, enabling quantitative identification and confidence assessment of defects such as container damage, stains, humidity, odor, and foreign objects.

[0063] In some embodiments, step S5 may further include steps S51 to S53.

[0064] In step S51, if the confidence score of the defect detection result is greater than or equal to the preset first score, the container can be automatically determined to be a good container.

[0065] In step S52, if the confidence score of the defect detection result is less than the preset second score, the container can be automatically determined to be a defective container, wherein the first score is greater than the second score.

[0066] In step S53, if the confidence score of the defect detection result is greater than or equal to the second score and less than the first score, the container can be automatically marked as a container to be checked and pushed to the remote inspector workstation for manual remote confirmation.

[0067] The container inspection method proposed in this application can achieve more than 95% automation through AI preliminary judgment + remote manual review, while retaining manual supervision of key links. It complies with the "human-machine collaboration" safety standard and completely eliminates the high-risk action of personnel entering the container.

[0068] For example, the first score can be 0.9, and the second score can be 0.7. When the confidence score of the defect detection result is ≥0.9, it is automatically judged as a "good container" and uploaded to the TOS (Port Production Scheduling System), from which the TOS issues a transportation task; when the confidence score of the defect detection result is ≤0.7 and <0.9, it is automatically marked as a "container awaiting verification" and pushed to the remote container inspector workstation (off-site) for remote manual confirmation; when the confidence score of the defect detection result is <0.7, it is automatically judged as a "bad container" and uploaded to the TOS. The TOS triggers a maintenance work order to the TCS (Truck Control System), and the TCS issues the corresponding control command to the unmanned truck. After receiving the control command, the unmanned truck automatically drives into the repair area.

[0069] The container inspection method of this application does not require manual entry into the container for inspection. Furthermore, it can automatically determine the quality of the container and whether "the container needs to be manually reviewed" based on the confidence score of the defect detection results, without relying on manual clicking of good / bad containers.

[0070] In some embodiments, the container inspection method of this application may further include step S6.

[0071] In step S6, if the confidence score of the defect detection result obtained in step S4 is lower than the preset second score, the multimodal perception system can be automatically triggered to perform a secondary scan and re-inspection of the container from more perspectives, and a 3D defect annotation report can be generated.

[0072] The container inspection method of this application avoids misjudgment and achieves a false detection rate of less than 3% by introducing a self-verification mechanism driven by the confidence score of defect detection results.

[0073] In some embodiments, the container inspection method of this application may further include step S7.

[0074] In step S7, during the scanning of the container, the power lock and wheel lock of the unmanned truck carrying the container can be controlled, and the hazard lights of the unmanned truck can be kept on continuously.

[0075] In some embodiments, the container inspection method of this application may further include steps S81 and S82.

[0076] In step S81, a dual electronic fence can be constructed at the inspection station based on millimeter-wave radar and UWB (Ultra Wideband) positioning technology.

[0077] In step S82, when any person enters the danger zone defined by the electronic fence, the scanning can be paused immediately and an alarm can be triggered, such as an audible and visual alarm.

[0078] In some embodiments, the container inspection method of this application may further include steps S91 and S92.

[0079] In step S91, after each container inspection is completed, a unique digital fingerprint is generated for the entire inspection process data, the data is solidified, and a traceable data evidence chain is formed.

[0080] After each container inspection, an inspection report is generated. This report includes a 3D map of the container, the type of defect, its location, and the corresponding confidence score. Furthermore, a unique digital fingerprint is generated from the entire inspection process data. This digital fingerprint includes information such as timestamps, GPS data, device ID, 3D point cloud hash values, and AI decision logs. All operation logs and security events during the inspection process are stored on the blockchain, establishing a traceable, verifiable, and auditable data evidence chain for the inspection results, meeting port safety and logistics compliance requirements.

[0081] In step S92, the data evidence chain is uploaded to the port production scheduling system to form a full life cycle inspection file for the container. This allows third-party systems such as the port supervision platform, customs, and insurance companies to access the original data according to their permissions, achieving tamper-proof, traceable, and arbitrable data.

[0082] The container inspection method of this application performs non-contact scanning of the internal and external status of unmanned truck containers by activating the multimodal perception system of the inspection gantry deployed at the inspection station, thereby eliminating the need for manual entry into the container for inspection. Furthermore, it can achieve four-dimensional fusion detection of shape, color, temperature, and air, improving the comprehensiveness of container inspection.

[0083] The container inspection method in this application can also be extended to the inspection process of refrigerated containers, dangerous goods containers, special containers, etc.

[0084] This application also provides a container inspection system 200. Figure 2 A schematic block diagram of a container inspection system 200 according to one embodiment of this application is shown. Figure 2 As shown, a container inspection system 200 of one embodiment of this application may include a multimodal perception system 210 and an edge intelligent decision-making system 220.

[0085] The multimodal perception system 210 is deployed on the inspection gantry at the inspection station. It can be used to perform contactless scanning of the internal and external status of unmanned container trucks and output multimodal perception data of the containers.

[0086] The edge intelligent decision-making system 220 is deployed on the inspection gantry. The edge intelligent decision-making system 220 includes an AI inspection inference engine, which includes a multimodal fusion neural network model. The multimodal fusion neural network model may include a feature extraction and fusion module 221, three detection heads and a classifier 225. The three detection heads may include a deep regression detection head 222, a location positioning detection head 223 and a defect classification detection head 224.

[0087] The feature extraction and fusion module 221 can perform feature extraction and feature splicing based on the multimodal sensing data output by the multimodal sensing system 210 to generate a multidimensional fusion feature vector F representing the defect.

[0088] The depth regression detection head 222 can perform an inner product operation between the depth weight W_depth and the multidimensional fusion feature vector F to obtain the defect depth value.

[0089] The location detection head 223 can perform an inner product operation between the location weight W_box and the multi-dimensional fusion feature vector F to obtain the defect location value.

[0090] The defect classification and detection head 224 can perform an inner product operation between the weights W_type corresponding to various defects and the multi-dimensional fusion feature vector F to obtain the numerical values ​​of various defects.

[0091] The classifier 225 can output confidence scores for various defect detection results based on the numerical values ​​of various defects, such as scores in the range of 0-1.

[0092] The AI ​​container inspection inference engine can determine the quality of a container based on the confidence scores of various defect detection results.

[0093] In some embodiments, the multimodal sensing system 210 may include, for example, a binocular structured light 3D scanner 211, a hyperspectral imager 212, an infrared thermal imager, and a miniature gas sensor array 214.

[0094] The binocular structured light 3D scanner 211 can be used to reconstruct the 3D point cloud of a container's six sides, thereby detecting geometric defects in the container.

[0095] The hyperspectral imager 212 can acquire hyperspectral images of the container surface to identify damage to the container surface.

[0096] The infrared thermal imager 213 can collect thermal images of the inside of a container to detect abnormal temperature zones inside the container.

[0097] The miniature gas sensor array 214 can detect the gas inside a container and acquire time-series data of gas concentration.

[0098] The multimodal sensing data of the container output by the multimodal sensing system 210 includes three-dimensional point cloud, hyperspectral image, thermal map and gas concentration time series data.

[0099] In some embodiments, when the confidence score of the defect detection result is greater than or equal to a preset first score (e.g., 0.9), the AI ​​container inspection inference engine can automatically determine that the container is a good container; when the confidence score of the defect detection result is less than a preset second score (e.g., 0.7), the AI ​​container inspection inference engine can automatically determine that the container is a bad container; when the confidence score of the defect detection result is greater than or equal to the second score and less than the first score, the AI ​​container inspection inference engine can automatically mark the container as a container to be checked and push it to the remote inspector workstation for remote manual confirmation.

[0100] In some embodiments, the AI ​​container inspection inference engine can automatically trigger the multimodal perception system 210 to perform a secondary scan and re-inspection of the container from more perspectives when the confidence score is lower than a preset second score.

[0101] In some embodiments, the AI ​​container inspection inference engine generates a unique digital fingerprint for the entire inspection process data after each container inspection, solidifies the data and forms a traceable data evidence chain, and uploads the data evidence chain to the port production scheduling system.

[0102] The container inspection system 200 of this application requires no manual intervention, can perceive the internal and external status of the container in all dimensions, has self-verification capabilities, and can deeply collaborate with the port production scheduling system.

[0103] Figure 3 A diagram illustrating the architecture of a port intelligent transportation system according to an embodiment of this application is provided. Figure 3 As shown, a port intelligent transportation system may include a container inspection gantry system, unmanned trucks, a driver scheduling system (TCS), port systems, and third-party systems.

[0104] The unmanned truck communicates with the Driving Control System (TCS). The unmanned truck includes a perception system, a decision-making system, and a control system. The perception system senses the surrounding environment, locates itself, and collects all external and vehicle status information. The decision-making system receives dispatch tasks from the TCS and combines the perception information to determine driving behavior and operational logic. The control system plans the driving path and speed, controls vehicle steering, acceleration, and deceleration, and precisely executes driving actions.

[0105] The container inspection gantry system includes the multimodal perception system and edge intelligent decision-making system of this application. Additionally, the container inspection gantry system also includes a high-precision positioning system and a communication system. The container inspection gantry system communicates with the intelligent driving dispatch system (TCS), and can upload location information, good / bad container information obtained from the edge intelligent decision-making system, and generated data evidence chain information to the intelligent driving dispatch system (TCS).

[0106] The Intelligent Driving Dispatch System (TCS) is the central cloud brain of the port's unmanned truck system, responsible for the unified dispatch and control of all unmanned truck operations. TCS comprises four main functions: business dispatch, safety monitoring, operations management, and electronic fences. Business Dispatch: It interfaces with the port's Terminal Operations System (TOS), receives loading / unloading / transfer tasks, assigns operational instructions to unmanned trucks, and plans driving routes to ensure the orderly execution of container transport operations. Safety Monitoring: It monitors vehicle location, status, malfunctions, and the surrounding environment in real time, handles abnormal conditions (such as obstacles or vehicle malfunctions), and triggers emergency stops and alarms to ensure port operational safety. Operations Management: It is responsible for fleet maintenance statistics, operational efficiency analysis, and equipment status management (such as energy consumption, mileage, and maintenance), providing data support for port operation optimization. Electronic Fences: Through virtual boundaries, it controls the vehicle's driving range and the dangerous areas entered by personnel, restricting unmanned trucks or personnel from driving or moving within designated areas of the port, preventing boundary crossings and accidental entry into dangerous areas, and is an important means of port safety management. The Intelligent Driving Dispatch System (TCS) can upload the data evidence chain information uploaded by the container inspection gantry system to the Port Production Dispatch System (TOS) in the port system, and can also receive the operation task information issued by the TOS.

[0107] The port system includes the Port Production Scheduling System (TOS), quay cranes, yard cranes, and other systems. Each time a container completes an inspection, the data evidence chain uploaded to the TOS forms a complete lifecycle inspection file for the container. This file supports third-party systems, such as customs, border control, insurance companies, and port monitoring platforms, to access the original data according to their authorized permissions, ensuring tamper-proof, traceable, and arbitrable data.

[0108] Figure 4 A flowchart illustrating an embodiment of the unmanned truck automated container inspection process of this application is disclosed. Figure 4As shown, Step 401: The unloading task is completed, and TOS issues an unloading task; Step 402: The backend confirms whether the unloading task requires container inspection; Step 403: If container inspection is not required, the unloading task is issued to the unmanned truck, which then drives to the unloading location; Step 404: If container inspection is required, TCS issues an inspection task and inspection coordinates to the unmanned truck. After receiving the inspection task and coordinates, the unmanned truck proceeds to the inspection location, stops, locks, and activates its hazard lights; Step 405: The inspection task is then issued to the inspection gantry system, which automatically begins inspecting the unmanned truck's container; Step 406: After inspection, the inspection gantry's decision system reports the inspection report to TOS; Step 407: TOS determines the container is in good condition and sends a good container signal command to TCS; Step 408: After receiving the good container signal command... Step 409: The TCS issues the original task to the unmanned truck, which unlocks and executes the original task; Step 410: The TOS determines that it is a bad container signal and sends a bad container signal instruction to the TCS; Step 411: After receiving the bad container signal instruction, the TCS issues a new task to the unmanned truck, which unlocks and executes the new task; Step 412: The TOS can share the data of each container inspection result with a third-party system for access according to permissions; Step 413: The TCS confirms whether the personnel are in a safe position by setting up an electronic fence. If the personnel are in a safe position, proceed to step 414; Step 415: If the unmanned truck is not locked when the personnel are in the danger zone, the TCS controls the unmanned truck to lock the vehicle; Step 416: When the personnel are in the danger zone, the inspection gantry will sound an alarm; Step 417: The personnel leave the danger zone and are in a safe position.

[0109] Figure 5 A site diagram illustrating one embodiment of this application is shown. (See diagram below.) Figure 5 As shown, 501: The quay crane (shore crane / shore container crane) is installed on the quayside to load containers from the ship onto the unmanned truck; 502: The unmanned truck carrying the container proceeds to the automated inspection area according to instructions; 503: The unmanned truck automatically inspects the container at the automated inspection area and provides the inspection results as good / bad; 504: If the container is detected as good, the unmanned truck automatically proceeds to the container yard and unloads the good container to the container yard via a gantry crane; 505: If the container is detected as bad, the unmanned truck automatically proceeds to the repair area.

[0110] The container inspection method and system provided in this application have been described in detail above. Specific examples have been used to illustrate the container inspection method and system in this application. The descriptions of the embodiments are merely for the purpose of helping to understand the core ideas of this application and are not intended to limit this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the spirit and principles thereof, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for inspecting a container, characterized in that: include: The multimodal perception system deployed on the container inspection gantry at the inspection station is activated to perform non-contact scanning of the internal and external status of the unmanned container truck container and output the multimodal perception data of the container. The multimodal fusion neural network model performs feature extraction and feature concatenation based on the multimodal sensing data to generate a multidimensional fusion feature vector representing the defect; The multidimensional fusion feature vector is input into the three detection heads of the multimodal fusion neural network model for parallel computation. The three detection heads include a depth regression detection head, a location detection head, and a defect classification detection head. The depth regression detection head performs an inner product operation on the depth weight and the multidimensional fusion feature vector to obtain the defect depth value. The location detection head performs an inner product operation on the location weight and the multidimensional fusion feature vector to obtain the defect location value. The defect classification detection head performs an inner product operation on the weights corresponding to each type of defect and the multidimensional fusion feature vector to obtain the value of each type of defect. The various defect values ​​are fed into the classifier in the multimodal fusion neural network model, and the confidence scores of the various defect detection results are output. The quality of the container is determined based on the confidence scores of various defect detection results.

2. The method as described in claim 1, characterized in that: The process of non-contactly scanning the internal and external states of the unmanned truck container by activating the multimodal perception system deployed on the inspection gantry at the inspection station and outputting the multimodal perception data of the container includes: The binocular structured light 3D scanner deployed on the inspection gantry at the inspection station is used to reconstruct the 3D point cloud of the six-sided container and detect the geometric defects of the container. Hyperspectral images of the container surface are acquired using a hyperspectral imager to identify damage to the container surface. The container's interior is captured by an infrared thermal imager to detect abnormal temperature zones. A miniature gas sensor array is used to detect the gas inside the container and acquire time-series data of gas concentration. The multimodal sensing data includes the three-dimensional point cloud, the hyperspectral image, the heat map, and the gas concentration time series data.

3. The method as described in claim 1, characterized in that: The method further includes: The relative position of the container and the inspection gantry is assisted by the LiDAR and high-definition wide-angle camera deployed on the inspection gantry to ensure stable scanning posture.

4. The method as described in claim 1, characterized in that: The determination of the container's quality based on the confidence scores of various defect detection results includes: When the confidence score is greater than or equal to a preset first score, the container is automatically determined to be a good container; When the confidence score is less than a preset second score, the container is automatically determined to be a defective container, wherein the first score is greater than the second score; When the confidence score is greater than or equal to the second score and less than the first score, the container is automatically marked as a container to be checked and pushed to the remote inspector workstation for manual remote confirmation.

5. The method as described in claim 4, characterized in that: The method further includes: When the confidence score is lower than a preset second score, the multimodal perception system is automatically triggered to perform a secondary scan and re-examination of the container from more perspectives.

6. The method as described in claim 1, characterized in that: The method further includes: Based on millimeter-wave radar and ultra-wideband positioning technology, a dual electronic fence is constructed at the box inspection station. When any person enters the danger zone defined by the electronic fence, the scanning will be immediately paused and an alarm will be triggered.

7. The method as described in claim 1, characterized in that: The method further includes: Each time the container inspection is completed, a unique digital fingerprint is generated for the entire inspection process data, the data is solidified, and a traceable data evidence chain is formed. The data evidence chain is uploaded to the port production scheduling system to form a full life cycle inspection file for the container, which can be accessed by third-party systems according to their permissions.

8. The method as described in claim 1, characterized in that: The method further includes: During the scanning of the container, the unmanned truck carrying the container is controlled to lock up its power and wheels.

9. A container inspection system, characterized in that: Including multimodal sensing systems and edge intelligent decision-making systems: The multimodal perception system is deployed on the inspection gantry at the inspection station to perform non-contact scanning of the internal and external status of unmanned container trucks and output multimodal perception data of the containers. The edge intelligent decision-making system is deployed on the inspection gantry. The edge intelligent decision-making system includes an AI inspection inference engine, which includes a multimodal fusion neural network model. The multimodal fusion neural network model includes a feature extraction and fusion module, three detection heads, and a classifier. The three detection heads include a depth regression detection head, a location positioning detection head, and a defect classification detection head. The feature extraction and fusion module is used to extract and concatenate features based on the multimodal sensing data output by the multimodal sensing system to generate a multidimensional fusion feature vector representing the defect. The depth regression detection head is used to perform an inner product operation between the depth weight and the multi-dimensional fused feature vector to obtain the defect depth value; The location detection head is used to perform an inner product operation between the location weight and the multi-dimensional fused feature vector to obtain the defect location value; The defect classification and detection head is used to perform an inner product operation between the weights corresponding to each type of defect and the multidimensional fusion feature vector to obtain the numerical values ​​of each type of defect; the classifier is used to output confidence scores of each type of defect detection result based on the numerical values ​​of each type of defect. The AI ​​container inspection inference engine is used to determine the quality of the container based on the confidence scores of various defect detection results.

10. The system as described in claim 9, characterized in that: The multimodal sensing system includes a binocular structured light 3D scanner, a hyperspectral imager, an infrared thermal imager, and a miniature gas sensor array, wherein... The binocular structured light 3D scanner is used to complete the 3D point cloud reconstruction of the six-sided container, and is used to detect the geometric defects of the container. The hyperspectral imager is used to acquire hyperspectral images of the container body surface to identify damage to the container body surface. The infrared thermal imager is used to collect thermal images of the inside of the container to detect abnormal temperature zones inside the container. The miniature gas sensor array is used to detect the gas inside the container and acquire time-series data of gas concentration. The multimodal sensing data includes the three-dimensional point cloud, the hyperspectral image, the heat map, and the gas concentration time series data.

11. The system as described in claim 9, characterized in that: The AI ​​container inspection inference engine is used to automatically trigger the multimodal perception system to perform a secondary scan and re-inspection of the container from more perspectives when the confidence score is lower than a preset first score.

Images