AI visual identification and multi-terminal cooperation fused shipping cargo supervision method and system

Abstract

The invention discloses a shipping cargo supervision method and system fusing AI visual identification and multi-terminal cooperation, and the method comprises the following steps: S1, synchronously collecting the visual image, three-dimensional point cloud, weight change, cabin pressure and real-time position data of a ship through a visual sensor, a physical sensor and a Beidou / GPS positioning module which are disposed on the ship; s2, performing space-time alignment and feature level fusion on the multi-source heterogeneous data acquired in the S1, and performing feature extraction; and S3, inputting the visual features and the physical features extracted in the step S2 and a preset shipping business rule into a vision-sensor-rule three-mode decision engine for weighted decision making, thereby completing intelligent identification and early warning. According to the invention, through deep fusion of the deep learning model, the multi-source sensor data and the shipping service rule, high-precision decision in a complex environment is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent shipping and logistics supervision technology, specifically to a shipping cargo supervision method and system that integrates AI visual recognition and multi-terminal collaboration. Background Technology

[0002] The current level of intelligent technology in the shipping industry remains insufficient, with numerous pressing challenges. First, manual inspection remains the dominant regulatory method. Most bulk carriers in the Yangtze River basin still rely on the traditional "eye-and-hand log" system to record cargo status. This inefficient approach results in nearly 90% of cargo anomalies occurring in blind spots of manual monitoring. Second, severe data silos exist between various business systems. The lack of effective data exchange mechanisms between ship monitoring systems and key platforms such as port operating systems creates information silos, significantly reducing collaborative operational efficiency. Furthermore, compliance audits with the International Maritime Organization (IMO) still largely depend on manual entry of loading and unloading records. This method is not only prone to errors but also lacks effective traceability, posing a threat to shipping safety management. These problems underscore the urgency and necessity of intelligent transformation in the shipping industry.

[0003] As a major inland waterway in my country, the Yangtze River Basin presents multiple technical challenges to intelligent vessel monitoring due to its complex natural environment. In terms of visual data acquisition, the winding waterways cause continuous swaying of vessels, severely impacting monitoring stability. Strong water surface reflections cause localized overexposure of images during most operating periods, and fogging of monitoring lenses on nearly half of the vessels during the rainy season significantly reduces image clarity. Navigation vibrations further increase image blur. Meanwhile, diverse loading and unloading equipment (such as augers, grain suction machines, and conveyor belts) and different cargo coverings (such as tarpaulins, containers, and bulk cargo) further increase the complexity of image recognition. In terms of network transmission, significant fluctuations in network coverage along the river cause substantial delays in communication base station switching. Limited network bandwidth and the large volume of data generated by vessel monitoring create a significant contradiction, with periodic network outages occurring in some sections. Coupled with frequent fog, heavy rainfall, and other meteorological changes, these objective environmental factors severely test the accuracy, stability, and real-time performance of traditional visual recognition systems. To address these complex challenges, there is an urgent need to overcome existing technological bottlenecks through systematic technological innovation in order to achieve reliable intelligent supervision across all weather conditions and all shipping segments. This solution, by integrating multimodal perception technology and adaptive algorithm optimization, can effectively identify various loading and unloading equipment and cargo under different coverage conditions, providing a comprehensive technical solution for the intelligentization of Yangtze River shipping. Summary of the Invention

[0004] This invention proposes a shipping cargo supervision method and system that integrates AI visual recognition and multi-terminal collaboration. Specifically, it relates to an intelligent waterway cargo supervision system and method that integrates computer vision, edge computing, and multimodal data fusion. It is particularly suitable for intelligent management of the entire cargo transportation process in inland waterway shipping (such as the Yangtze River and Pearl River basins), including vessel hull status identification, loading and unloading operation monitoring, and cargo transportation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration includes the following steps: S1. By deploying visual sensors, physical sensors, and BeiDou / GPS positioning modules on the ship, the ship's visual images, 3D point clouds, weight changes, cabin pressure, and real-time location data are collected simultaneously. S2. Perform spatiotemporal alignment and feature-level fusion on the multi-source heterogeneous data collected in S1, and extract features. S3. The visual and physical features extracted in S2, together with the pre-set shipping business rules, are input into the "vision-sensor-rule" three-modal decision engine for weighted decision-making, thereby completing intelligent recognition and early warning.

[0006] On the other hand, this invention also discloses a shipping cargo supervision system integrating AI visual recognition and multi-terminal collaboration, used to implement the aforementioned shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration, including, It adopts a five-layer distributed architecture design, fully covering the complete technology stack from terminal interaction to infrastructure; In the presentation layer, the system provides access interfaces for multiple terminals, including PC management backend, mobile APP and mini program, and is equipped with a 3D visualization screen to realize the three-dimensional presentation of regulatory data. The API interface layer has built a complete interface service system, including core functional modules such as fine-grained access control, OAuth2.0 authentication, and dynamic rate limiting; The application service layer integrates three major business platforms: intelligent cargo supervision, waterway logistics supervision, and intelligent vessel management. Through AI behavior recognition engine and multimodal data fusion technology, it realizes closed-loop supervision of the entire process of cargo from loading, transportation to unloading. The data service layer adopts a hybrid storage architecture, combining the transaction characteristics of relational databases, the flexible storage of document databases, and the efficient query capabilities of time series databases to build a real-time data processing pipeline. The infrastructure layer is based on the Kubernetes container cloud platform to achieve cloud-native deployment, and effectively responds to fluctuations in the Yangtze River waterway network through intelligent collaboration between edge nodes and cloud computing; The system achieves efficient collaboration among different layers through standardized APIs and possesses core capabilities such as elastic scaling, unified processing of heterogeneous data, and intelligent hierarchical computing.

[0007] Furthermore, the system also includes real-time transmission of cabin data via REST API, push of abnormal alarms via WebSocket, and daily synchronization of blockchain-stored evidence data to the platform to ensure the real-time nature and immutability of regulatory information; Simultaneously, a multi-terminal collaborative system is constructed to provide real-time cabin anomaly alarm pushes for ship owners, open up cargo 3D visualization status query for cargo owners, and equip management with a configurable interface for business rules and response strategies, forming a full-link regulatory closed loop of "data interoperability - intelligent early warning - multi-party collaboration".

[0008] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0009] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0010] As can be seen from the above technical solution, the innovative points of the shipping cargo supervision method and system integrating AI visual recognition and multi-terminal collaboration of the present invention are as follows: (1) Technological Innovation This solution pioneered a "vision-sensor-rule" three-modal decision engine, achieving high-precision decision-making in complex environments through deep integration of deep learning models, multi-source sensor data, and shipping business rules. The developed Maritime-AT shipping-specific adversarial training framework optimizes model robustness for typical interferences such as water surface reflection and fog, improving recognition accuracy by 21.5% under extreme weather conditions. The innovative edge-cloud dynamic load balancing technology automatically switches computing nodes based on network status, ensuring real-time response within 200ms even under fluctuating Yangtze River waterway network conditions.

[0011] (2) Application Innovation By establishing a blockchain-based evidence storage standard for water transport (timestamp + GPS + visual evidence), the loading and unloading process can be reliably traced and tamper-proofed. The developed configurable anomaly response management system supports low-code rule editing, improving management efficiency by 65%. The automatic loading and unloading progress calculation algorithm, through cabin capacity change recognition and equipment action analysis, reduces the loading and unloading record time from 45 minutes to real-time automatic generation, significantly improving logistics efficiency.

[0012] (3) Engineering Innovation The designed shipping-specific data augmentation pipeline integrates algorithms such as HSV transformation and CutMix to solve the problems of insufficient samples and environmental interference; the intelligent log analysis system realizes automatic fault location, reducing the average troubleshooting time from 2 hours to 15 minutes; the automated model deployment toolchain supports one-click processing of the entire process from training to edge deployment, compressing the model iteration cycle to 3 days and significantly reducing operation and maintenance costs.

[0013] Specifically, as follows: (1) It has a strong environmental adaptability and high recognition accuracy. By employing the innovative Maritime-AT adversarial training framework specifically designed for shipping, and a shipping-specific loss function tailored to complex environments such as water surface reflection, fog, and ship sway, the robustness of the AI ​​vision model under extreme weather and harsh operating conditions has been significantly improved. Real-world application data shows that this technology increases the system's average recognition accuracy by 21.5% in typical interference environments such as strong light and fog, effectively overcoming the bottleneck of traditional vision systems experiencing a sharp drop in recognition rate in shipping scenarios.

[0014] (2) Significantly improved decision-making reliability: The pioneering "vision-sensor-rule" trimodal decision engine deeply integrates deep learning, multi-source physical sensor data, and industry business rules. Through cross-validation and weighted fusion of multi-source information, it achieves closed-loop supervision from perception to decision-making. This mechanism reduces the false alarm rate of cargo status in complex environments to below 1.3%, significantly improving the accuracy and credibility of regulatory decisions.

[0015] (3) The system has strong real-time response and good network robustness: The edge-cloud dynamic load balancing technology adopted can intelligently distribute tasks between the shipboard edge node and the cloud computing center according to the real-time fluctuation of the Yangtze River waterway network quality. This ensures that even under conditions of limited network bandwidth or intermittent interruption, the system can still maintain a real-time response within 200 milliseconds, meeting the stringent timeliness requirements of shipping supervision.

[0016] (4) High level of reliable traceability and automation throughout the entire process: By introducing blockchain technology, a standard for the supervision and evidence storage of waterborne cargo has been established. Timestamps, GPS location information and visual evidence are all uploaded to the blockchain, realizing the immutability and reliable traceability of data throughout the loading and unloading process. At the same time, the automatic calculation algorithm for loading and unloading progress enables the real-time automatic generation of operation records, reducing the 45 minutes required for traditional manual recording to almost real-time, greatly improving logistics efficiency and automation level.

[0017] (5) Excellent engineering feasibility and maintainability: The designed shipping-specific data augmentation pipeline effectively alleviates the problem of insufficient samples; the intelligent log analysis system enables rapid fault location, reducing the average troubleshooting time from 2 hours to 15 minutes; the automated model deployment toolchain supports one-click completion of the entire process from training to edge deployment, compressing the model iteration cycle to 3 days. These engineering innovations significantly reduce the long-term operation and maintenance costs and technical barriers of the system, ensuring the large-scale, sustainable deployment and application of the system. Attached Figure Description

[0018] Figure 1 This is a flowchart of an embodiment of the present invention; Figure 2 This is a flowchart of the backpropagation process of visual perception gradient according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the multimodal data fusion process according to an embodiment of the present invention. Figure 4a and Figure 4b This is a schematic diagram of the AI ​​recognition interface in an embodiment of the present invention; Figure 5 This is a schematic diagram of the AI ​​early warning analysis interface; Figure 6 This is a schematic diagram of a visual monitoring interface. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0020] like Figure 1 As shown in the figure, the shipping cargo supervision system integrating AI visual recognition and multi-terminal collaboration described in this embodiment has the following architecture: The intelligent shipping cargo supervision system adopts a five-layer distributed architecture design, comprehensively covering the entire technology stack from terminal interaction to infrastructure. At the presentation layer, the system provides access interfaces for multiple terminals, including a PC management backend, mobile APP, and mini-programs, and is equipped with a 3D visualization screen to present supervision data in a three-dimensional way. The API interface layer constructs a complete interface service system, including core functional modules such as fine-grained access control, OAuth2.0 authentication, and dynamic rate limiting. The application service layer innovatively integrates three major business platforms: intelligent cargo supervision, waterway logistics supervision, and intelligent vessel management. Through AI behavior recognition engine and multimodal data fusion technology, it achieves closed-loop supervision of the entire process of cargo from loading, transportation, to unloading. The data service layer adopts a hybrid storage architecture, combining the transaction characteristics of relational databases, the flexible storage of document databases, and the efficient query capabilities of time-series databases to build a real-time data processing pipeline. The infrastructure layer is based on the Kubernetes container cloud platform for cloud-native deployment, effectively responding to fluctuations in the Yangtze River waterway network through intelligent collaboration between edge nodes and cloud computing. The system achieves efficient collaboration across layers through standardized APIs, possessing core capabilities such as elastic scaling, unified processing of heterogeneous data, and intelligent hierarchical computing. It has been specifically optimized for the characteristics of inland waterway shipping, featuring adaptive network switching and data continuation mechanisms to ensure stable regulatory performance even under complex operating conditions. The architecture diagram is shown below. Figure 1 .

[0021] Multimodal recognition technology This method achieves a closed loop from multimodal perception to intelligent decision-making through the following sequential steps: S1. Multi-source data acquisition and environmental perception By deploying visual sensors (infrared cameras), physical sensors (LiDAR, weighing systems, cabin pressure sensors), and BeiDou / GPS positioning modules on the ship, visual images, 3D point clouds, weight changes, cabin pressure, and real-time location data of the ship are collected simultaneously. Meanwhile, the ship's attitude (roll and pitch) is sensed using an onboard IMU, providing a spatiotemporal reference for subsequent data fusion.

[0022] S2, Multimodal Data Fusion and Feature Extraction Spatiotemporal alignment and feature-level fusion were performed on the multi-source heterogeneous data collected by S1. First, millisecond-level time synchronization was performed based on the PTP protocol; second, coordinate transformation was used to unify all sensor data to the ship's global coordinate system. In the feature extraction stage, visual data was input into a shipping-specific YOLOv8 model trained with Maritime-AT adversarial techniques to extract visual features such as cargo status and equipment type; physical sensor data was filtered, denoised, and quantized to form physical feature vectors describing cargo weight, volume, and sealing performance.

[0023] S3. Intelligent recognition and early warning based on a trimodal decision engine The visual and physical features extracted by S2, along with pre-set shipping business rules, are input into a weighted decision engine based on a "vision-sensor-rule" trimodal approach. This engine calculates a consistency score from multiple sources (e.g., >85% automatically confirms, 40%-85% triggers manual review, <40% triggers an alarm) and ultimately outputs a comprehensive assessment of the cargo's status (e.g., normal, abnormal warehouse opening, risk of moisture damage, suspected theft). This assessment is stored on a blockchain and simultaneously pushed to relevant parties in real-time via a multi-terminal collaborative system.

[0024] The following is a detailed explanation: I. Principles and Design of Visual Perception Module Algorithm Innovative YOLOv8 architecture for shipping A Spectral Attention Module is introduced to enhance effective feature extraction based on water surface reflectivity. Improve the anchor frame design and optimize the anchor point ratio to adapt to the ship's cargo hold structure (12:16 / 19:36 / 40:28). Add a branch of prior knowledge of cabin geometry to improve the accuracy of hatch status recognition.

[0025] Maritime-AT combat training Adversarial example generation: Simulates six typical shipping interference scenarios (strong light / water mist / surge, etc.); A hybrid approach combining physical models and GANs was used to ensure sample authenticity (PSNR>28dB). Loss function design: ; in To detect the loss, To combat the losses; Shipping specialization losses have three sub-losses ( To compensate for the loss of strong light, Loss due to water mist penetration, (to compensate for the loss of exercise) .

[0026] (1) Detection loss

[0027] class DetectionLoss(nn.Module): def __init__(self): super().__init__() self.obj_loss = FocalLoss() # Target existence loss self.box_loss = CIoULoss() # Box regression loss self.cls_loss = BCEWithLogitsLoss() # Classification loss def forward(self, pred, target): return 0.5*self.obj_loss(pred, target) + 0.3*self.box_loss(pred, target) + 0.2*self.cls_loss(pred, target) ... Function Description: Focal Loss is used to solve the problem of imbalanced positive and negative samples. Optimize bounding box regression accuracy using CIoU Loss The weighting ratio for the three types of losses is 5:3:2. (2) Combating losses

[0028] def adversarial_loss(clean_out, adv_out): # Consistency constraint of feature distribution based on KL divergence return F.kl_div( F.softmax(adv_out / τ, dim=1), F.softmax(clean_out / τ, dim=1), reduction='batchmean' ) Key parameters: Temperature coefficient τ = 2.0 Function: To force the model to output a consistent distribution for clean samples and adversarial samples. (3) Shipping specialization losses

[0029] Shipping-specific losses are composite loss functions designed for the special environment of ship monitoring, consisting of three key sub-losses: Glare Loss Compensation Solving the overexposure problem caused by strong reflections from the water surface Lighting compensation effect based on structural similarity (SSIM) Fog Loss Improve recognition capabilities in foggy conditions The L1 norm was used to calculate the pixel-level difference between the dehazed image and the real image. Motion Loss Compensating for image blur caused by hull rolling Using perceptual loss to maintain consistency of high-level semantic features import torch import torch.nn.functional as F from pytorch_msssim import ssim class MaritimeSpecializedLoss(nn.Module): def __init__(self): super().__init__() # Initialize VGG16 for perceptual loss calculation self.vgg = torch.hub.load('pytorch / vision', 'vgg16', pretrained=True).features[:16] for param in self.vgg.parameters(): param.requires_grad = False def glare_loss(self, compensated, target): """Strong light compensates for loss"" return 1 - ssim(compensated, target, data_range=1.0, size_average=True) def fog_loss(self, dehazed, target): "Water mist penetration loss" return F.l1_loss(dehazed, target) def motion_loss(self, deblurred, target): "Movement compensatory loss" # Extracting VGG features feat1 = self.vgg(deblurred) feat2 = self.vgg(target) return F.mse_loss(feat1, feat2) def forward(self, inputs, targets): """ parameter: inputs: A dictionary containing the processed images. - 'compensated': Light compensation result - 'dehazed': Dehazing result - 'deblurred': Deblurred the result targets: real images """ L_glare = self.glare_loss(inputs['compensated'], targets) L_fog = self.fog_loss(inputs['dehazed'], targets) L_motion = self.motion_loss(inputs['deblurred'], targets) return 0.3*L_glare + 0.2*L_fog + 0.5*L_motion Gradient backpropagation process This gradient backpropagation process employs a dual-path mechanism, simultaneously performing clean sample inference and adversarial sample generation after input image processing. Detection loss and adversarial loss are calculated separately, while a shipping-specific loss is dynamically generated based on real-time environmental interference detection. The three types of losses are fused into a total loss with weights of 1:0.5:0.8, and then backpropagated using a gradient truncation strategy (max_norm=2.0). This prioritizes optimizing adversarial interference-sensitive network layers, and a dynamic learning rate adjustment mechanism ensures stable convergence. The entire process achieves synergistic optimization of basic detection accuracy, adversarial robustness, and environmental adaptability, ensuring stable model performance under complex shipboard conditions. The process is as follows: Figure 2 : II. Deep Integration of Sensor Fusion Module This system constructs a multi-dimensional sensor network, achieving comprehensive perception of the ship's status through the collaborative work of an intelligent weighing system (50-ton range, 0.5% accuracy, weight 0.35), a 3D lidar (270° horizontal field of view, 100m ranging, weight 0.25), a cabin pressure monitoring sensor (0-10kPa range, ±50Pa accuracy, weight 0.15), and a historical transportation database (2000+ voyage records, weight 0.25). A spatiotemporal alignment algorithm is employed for precise fusion of multi-source data: first, millisecond-level time synchronization is achieved based on the PTP precise time protocol; then, coordinate transformation unifies the sensor data to the global coordinate system. This algorithm is implemented through the function `sensor_alignment`, which first performs time synchronization (`time_sync`) between visual and sensor data, then combines GPS position and ship attitude (`vessel_attitude`) for spatial coordinate transformation (`coordinate_transform`), finally outputting spatiotemporally aligned fused data, providing a precise and consistent input benchmark for subsequent analysis.

[0030] Multi-dimensional sensor networks:

[0031] Data spatiotemporal alignment algorithm: def sensor_alignment(visual_data, sensor_data): # Time Alignment (PTP Precision Time Protocol) synced_data = time_sync(visual_data, sensor_data) # Spatial coordinate transformation (ship coordinate system → global coordinate system) aligned_data = coordinate_transform( synced_data, gps_position, vessel_attitude ) return aligned_data III. Intelligent Decision-Making Workflow This system employs a three-tiered intelligent decision-making workflow to achieve multimodal data fusion: In the primary fusion layer, spatial registration of visual recognition results with LiDAR point clouds enables centimeter-level (error <5cm) precise object positioning; the intermediate verification layer combines dynamic weighing data with historical transportation records to perform probabilistic analysis of cargo types, and simultaneously utilizes cabin pressure sensor data and visual segmentation results to assess the cargo's sealing status; finally, in the advanced decision-making layer, a comprehensive judgment is generated through cross-validation of multi-source information, such as... Figure 3 The multimodal fusion flowchart shown automatically confirms cargo status when the consistency of data from all sensors is >85%, triggers manual review when it is between 40-85%, and activates an anomaly alarm when it is below 40%, forming a closed-loop monitoring system from perception to decision-making. This workflow significantly improves the accuracy and reliability of cargo monitoring in complex shipping environments, reducing the false alarm rate to 1.3%.

[0032] Hardware deployment solution Table 1 Hardware Deployment List

[0033] 1. System Integration Deeply integrated with the Haohan Intelligent Transportation Platform, it transmits cargo space data in real time via REST API, pushes anomaly alerts via WebSocket, and synchronizes blockchain-stored evidence data to the platform daily to ensure the real-time nature and immutability of regulatory information. At the same time, it builds a multi-terminal collaborative system, providing ship owners with real-time cargo anomaly alert pushes, cargo owners with access to 3D visualization of cargo status, and management with a configurable interface for business rules and response strategies, forming a full-link regulatory closed loop of "data interoperability - intelligent early warning - multi-party collaboration", which significantly improves the collaborative efficiency and transparency of all participants in the shipping supply chain.

[0034] 2. Achieve AI-powered task recognition Traditionally, ship operations at loading and unloading ports rely on ship owners taking photos and sending them to WeChat groups, or port staff taking photos and submitting them offline for approval via reports—a time-consuming and labor-intensive process. By introducing AI algorithms, loading and unloading equipment such as augers, booms, suction cups, and conveyor belts can be identified. Combined with GPS and BeiDou positioning technology, this allows for the identification of loading and unloading operations, as well as the assessment of remaining cargo in the hold to determine the progress of loading and unloading. This significantly improves management efficiency and greatly reduces labor costs. Figure 4a and Figure 4b The image shows a reference example of AI recognition.

[0035] 3. AI Early Warning Analysis The system collects photo and video streams from front-end devices, transmits the data to the back-end, and overlays the results in real time using a YOLO model. This enables timely alerts for anomalies during navigation, such as abnormal warehouse openings, abnormal shutdowns, equipment malfunctions, and unloading outside designated areas. This significantly improves management efficiency. Figure 5 This is an example of the AI ​​early warning analysis interface in this embodiment.

[0036] 4. AI-powered visual monitoring The system allows for real-time, multi-screen monitoring of the transportation status of vessels in operation via onboard camera hardware. It also supports multiple camera positions per vessel and allows for easy switching between viewing modes, thus improving daily management efficiency. Figure 6 This is an example of a visual monitoring interface in this embodiment.

[0037] Implementation results: In its actual deployment at the Yangtze River Shipping Company, this system has achieved significant application results. In terms of operational efficiency, through intelligent identification and automated recording technology, the system enables real-time recording of loading and unloading operations, greatly improving operational efficiency and completely changing the inefficient nature of traditional manual recording methods. Regarding safety, the system, with its advanced anomaly detection capabilities, has significantly reduced the incidence of cargo damage and theft, providing strong protection for cargo transportation safety. In terms of economic benefits, the system, through precise cabin capacity monitoring and intelligent scheduling functions, has effectively improved vessel utilization, saving the company substantial operating costs.

[0038] Currently, this technology system has been successfully applied to the "Haohan Intelligent Transportation" platform of Anhui Yansi Information Technology Co., Ltd., achieving large-scale deployment in the Yangtze River Basin and building a complete intelligent water transport supervision system. The system has demonstrated excellent stability and reliability in actual operation, providing important technical support for promoting the intelligent development of inland waterway transportation. The related technological achievements are providing important references for the formulation of industry standards.

[0039] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0040] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0041] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the shipping cargo supervision methods that integrate AI visual recognition and multi-terminal collaboration described in the above embodiments.

[0042] It is understood that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention, and the explanations, examples, and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0043] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0044] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0045] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0046] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration, characterized in that, Includes the following steps, S1. By deploying visual sensors, physical sensors, and BeiDou / GPS positioning modules on the ship, the ship's visual images, 3D point clouds, weight changes, cabin pressure, and real-time location data are collected simultaneously. S2. Perform spatiotemporal alignment and feature-level fusion on the multi-source heterogeneous data collected in S1, and extract features. S3. The visual and physical features extracted in S2, together with the pre-set shipping business rules, are input into the "vision-sensor-rule" three-modal decision engine for weighted decision-making, thereby completing intelligent recognition and early warning.

2. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 1, characterized in that: S2 includes, First, millisecond-level time synchronization is achieved based on the PTP protocol; second, coordinate transformation is used to unify all sensor data into the ship's global coordinate system. The physical sensor data is then filtered, denoised, and quantized to form physical feature vectors describing the weight, volume, and sealing of the cargo.

3. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 2, characterized in that: S2 includes, In the feature extraction stage, visual data is input into a shipping-specific YOLOv8 model trained with Maritime-AT adversarial techniques to extract visual features such as cargo status and equipment type.

4. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 1, characterized in that: S3 includes a decision engine that calculates a consistency score from multiple sources of information and ultimately outputs a comprehensive judgment on the status of goods. This judgment is stored on the blockchain, and the warning information is pushed to relevant parties in real time through a multi-terminal collaborative system.

5. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 3, characterized in that: The shipping-specific YOLOv8 model introduces a Spectral Attention Module to enhance effective feature extraction based on water surface reflection characteristics; improves anchor frame design and optimizes anchor point ratio to adapt to ship cargo hold structure; and adds a hull geometry prior knowledge branch to improve hatch cover status recognition accuracy.

6. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 5, characterized in that: The Maritime-AT adversarial training steps include: adversarial example generation: Simulate six typical shipping disruption scenarios; A hybrid approach combining physical models and GANs is used to ensure sample authenticity. Loss function design in To detect the loss, To combat the losses; Shipping specialization losses have three sub-losses. To compensate for the loss of strong light, Loss due to water mist penetration, To compensate for the loss through exercise; .

7. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 1, characterized in that: Feature-level fusion includes the construction of a multi-dimensional sensor network, which enables comprehensive perception of the ship's status through the collaborative work of intelligent weighing systems, three-dimensional lidar, cabin pressure monitoring sensors, and historical transportation databases. A spatiotemporal alignment algorithm is used to accurately fuse multi-source data: First, millisecond-level time synchronization is achieved based on the PTP precision time protocol, and then the sensor data is unified to the global coordinate system through coordinate transformation; The spatiotemporal alignment algorithm is implemented through the function sensor_alignment. First, the visual data and sensor data are aligned in time. Then, the spatial coordinates are transformed by combining GPS position and ship attitude. Finally, the spatiotemporally aligned fused data is output, providing an accurate and consistent input benchmark for subsequent analysis.

8. The shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration according to claim 1, characterized in that: The "vision-sensor-rule" three-modal decision engine performs weighted decision-making, including... Multimodal data fusion is achieved through a three-level intelligent decision-making workflow: In the primary fusion layer, centimeter-level precise object positioning is achieved through spatial registration of visual recognition results with LiDAR point clouds. The intermediate verification layer combines dynamic weighing data with historical transportation records to conduct probability analysis of cargo types, and simultaneously uses cabin pressure sensor data and visual segmentation results to assess the cargo sealing status. Ultimately, at the senior decision-making level, a comprehensive judgment is generated through cross-verification of multi-source information. When the consistency of data from various sensors is greater than 85%, the status of the goods is automatically confirmed; when it is between 40% and 85%, manual review is triggered; and when it is less than 40%, an anomaly alarm is activated, forming a closed-loop supervision system from perception to decision-making.

9. A shipping cargo supervision system integrating AI visual recognition and multi-terminal collaboration, used to implement the shipping cargo supervision method integrating AI visual recognition and multi-terminal collaboration as described in any one of claims 1-8, characterized in that, It adopts a five-layer distributed architecture design, fully covering the complete technology stack from terminal interaction to infrastructure; In the presentation layer, the system provides access interfaces for multiple terminals, including PC management backend, mobile APP and mini program, and is equipped with a 3D visualization screen to realize the three-dimensional presentation of regulatory data. The API interface layer has built a complete interface service system, including core functional modules such as fine-grained access control, OAuth2.0 authentication, and dynamic rate limiting; The application service layer integrates three major business platforms: intelligent cargo supervision, waterway logistics supervision, and intelligent vessel management. Through AI behavior recognition engine and multimodal data fusion technology, it realizes closed-loop supervision of the entire process of cargo from loading, transportation to unloading. The data service layer adopts a hybrid storage architecture, combining the transaction characteristics of relational databases, the flexible storage of document databases, and the efficient query capabilities of time series databases to build a real-time data processing pipeline. The infrastructure layer is based on the Kubernetes container cloud platform to achieve cloud-native deployment, and effectively responds to fluctuations in the Yangtze River waterway network through intelligent collaboration between edge nodes and cloud computing; The system achieves efficient collaboration among different layers through standardized APIs and possesses core capabilities such as elastic scaling, unified processing of heterogeneous data, and intelligent hierarchical computing.

10. A shipping cargo monitoring system integrating AI visual recognition and multi-terminal collaboration as described in claim 9, characterized in that, The system also includes real-time transmission of cabin data via REST API, push of abnormal alarms via WebSocket, and daily synchronization of blockchain-stored evidence data to the platform to ensure the real-time nature and immutability of regulatory information. Simultaneously, a multi-terminal collaborative system is constructed to provide real-time cabin anomaly alarm pushes for ship owners, open up cargo 3D visualization status query for cargo owners, and equip management with a configurable interface for business rules and response strategies, forming a full-link regulatory closed loop of "data interoperability - intelligent early warning - multi-party collaboration".

Images