A device tallying identification system for port tallying

Abstract

This invention belongs to the field of port logistics technology, specifically a cargo handling and identification system for port cargo handling equipment. It includes a video acquisition module that acquires continuous video streams from key port operation locations; an image processing module that performs target detection and character recognition to extract visual features and container number information; a temporal state inference module that uses a Bayesian probability model to infer the temporal state of multiple consecutive frames of identification results, eliminating single-frame random errors; and a logistics topology map construction module that establishes a topological relationship model of the port logistics network to characterize the spatial transfer logic of containers between operational stages. The introduction of the temporal state inference module eliminates errors in single-frame image processing and allows for joint inference of multiple consecutive frames of identification results. By utilizing the identification information of historical frames to correct the identification deviation of the current frame, the accuracy of identification can be improved.

Description

Technical Field

[0001] This invention belongs to the field of port logistics technology, specifically a cargo handling identification system for port cargo handling equipment. Background Technology

[0002] Port tallying and identification technology is the backbone of modern port logistics automation. Its technical level directly affects terminal loading and unloading efficiency, yard utilization, and the accuracy of subsequent logistics information. Against the backdrop of continuous growth in global port throughput, achieving efficient, accurate, and automatic container identification has become a key issue for ports to enhance their operational competitiveness. Existing technologies have inherent cognitive blind spots and logical flaws, which restrict the improvement of tallying data accuracy. Existing port tallying and identification systems have evolved from manual tallying to semi-automatic identification to fully automatic intelligent identification. Currently, they adopt a single-frame processing mode from acquisition to identification to output. They acquire images through cameras, use optical character recognition or deep learning algorithms to complete container number identification and container detection, and output the results to the back-end system. Although it is widely used in various port scenarios, it still has logical flaws.

[0003] Existing single-frame processing modes ignore the spatiotemporal continuity of container logistics, severing the correlation between the time and space dimensions of containers. This easily leads to systematic errors such as counting duplication or omission. Furthermore, the recognition results lack a verification mechanism with the terminal operating system, causing operational problems. In multi-machine collaborative scenarios, it also leads to fragmented logistics information, making it difficult to output deterministic data that matches the logistics system. To address this, the present invention provides an equipment tallying and recognition system for port tallying. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0005] The technical solution adopted by this invention to solve its technical problem is: a cargo tallying and identification system for port tallying, comprising:

[0006] The video acquisition module is deployed at key operational locations in the port to collect video stream data containing containers.

[0007] The image processing module parses the acquired video stream data frame by frame, extracting the visual features of the container and the container number character information from each frame of the video stream data;

[0008] The temporal state inference module performs temporal state inference on the identity of containers in multi-frame images based on a Bayesian probability model, thus eliminating the error of single-frame recognition results;

[0009] The logistics topology map construction module establishes a topological relationship model of the port logistics network to represent the spatial transfer logic of containers between operational stages.

[0010] The cross-domain consistency verification module performs consistency matching verification between the time-series state inference results and the logistics topology map, and outputs cargo handling data that conforms to logistics constraints.

[0011] Preferably, the video acquisition module includes multiple camera devices in the port operation area, and the camera devices are deployed in the ship quay crane operation area, the wharf front transportation channel, the yard entrance and exit, the transfer warehouse roadway, and the port gate.

[0012] The acquisition time of the camera equipment is synchronized and calibrated using a unified clock source.

[0013] Preferably, the image processing module extracts video frame images at preset frame intervals using a sliding window method, and the frame intervals are adjusted according to the movement speed of the work scene;

[0014] The extracted video frame images undergo distortion correction, contrast adjustment, and noise reduction.

[0015] Preferably, the image processing module uses a target detection model based on a deep convolutional neural network to identify container regions in the image. The target detection model outputs the detection box coordinates, confidence scores, and category labels, with the category labels used to distinguish different container types.

[0016] Visual features include color histogram features, texture features, and shape contour features, which are used for cross-frame target association and matching;

[0017] The box number character recognition adopts an end-to-end optical character recognition model based on convolutional neural networks.

[0018] Preferably, the timing state inference module models the movement state of the container as a hidden Markov process;

[0019] The prior probabilities of a Bayesian probability model include:

[0020] Transition probability is used to characterize the prior probability that a container maintains the same identity or undergoes a state transition between different frames;

[0021] The emission probability is used to characterize the prior probability of the image processing module outputting the recognition result given the actual state of the container;

[0022] The prior probability matrix is ​​learned from historical data during the system initialization phase.

[0023] Preferably, the temporal state inference module searches for a path in the state sequence that results in a posterior probability using dynamic programming.

[0024] The temporal state inference module also includes a confidence propagation mechanism, which calculates the posterior probability distribution entropy value of each frame recognition result. When the confidence of the image recognition result is lower than a preset threshold, the image result of that frame is marked as low confidence and its weight is adjusted in subsequent processing.

[0025] Preferably, the logistics topology graph construction module abstracts the port operation process into a directed graph structure containing nodes and edges.

[0026] Preferably, the logistics topology graph construction module constructs node and edge attributes using business data from the port operating system and the terminal production system;

[0027] When a business change event pushed by the dock operating system is detected, the corresponding nodes and edges in the topology graph are updated.

[0028] Preferably, the verification process of the cross-domain consistency verification module includes:

[0029] The trajectory generation stage extracts the spatial coordinates of the container in consecutive frames and generates the movement trajectory curve;

[0030] The path matching stage involves matching the movement trajectory with the paths in the logistics topology map based on curve similarity.

[0031] When the movement trajectory does not conform to the logistics topology constraints, it enters the anomaly handling stage, performs secondary verification, and outputs the corrected or marked cargo handling data.

[0032] Preferably, the exception handling phase of the cross-domain consistency verification module includes:

[0033] When low-confidence frames are concentrated in a certain continuous time interval, it is determined that the result of this segment is misidentified due to the image, and the trajectory is corrected by interpolation or extrapolation.

[0034] When low-confidence frames are distributed throughout the entire time interval, it is determined that there is a systematic error in the current identification result, an alarm message is sent to the back-end management system and the downgraded inventory data is output.

[0035] The beneficial effects of this invention are as follows:

[0036] 1. The equipment tallying and identification system for port tallying described in this invention can eliminate errors in single-frame image processing mode by introducing a time-series state inference module. In traditional single-frame processing mode, when a frame fails to be identified due to changes in illumination, angle shift, or temporary occlusion, it will incorrectly determine that the container has left the site or is counted twice as the same container. By using a Bayesian probability model to jointly infer the identification results of multiple consecutive frames and using the identification information of historical frames to correct the identification deviation of the current frame, the accuracy of identification can be improved.

[0037] 2. The equipment tallying and identification system for port tallying described in this invention, through the combined application of logistics topology map and cross-domain consistency verification module, can realize the transition from visual perception to logistics determinism. There is a data inconsistency problem between the tallying data output by the traditional system and the terminal operating system. By performing consistency verification between the identification results and the logistics topology map, the identification results conform to the feasible logistics path constraints, thereby eliminating data errors caused by the uncertainty of visual recognition.

[0038] 3. The equipment tallying and identification system for port tallying described in this invention uses a collaborative working mechanism between a time-series state inference module and a logistics topology map. Based on the time-series state inference module, it identifies the continuous movement trajectory of the same container. The logistics topology map confirms that the movement trajectory conforms to the feasible path of the port logistics network, making the logistics information complete instead of fragmented, and providing the terminal operating system with tallying data that is traceable throughout the entire process. Attached Figure Description

[0039] The invention will now be further described with reference to the accompanying drawings.

[0040] Figure 1 This is a structural block diagram of a cargo tallying and identification system for port cargo tallying according to the present invention;

[0041] Figure 2 This is a flowchart of the timing state inference module in this invention. Detailed Implementation

[0042] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0043] like Figure 1 and Figure 2 As shown, an embodiment of the present invention provides a cargo handling identification system for port cargo handling, comprising a video acquisition module, an image processing module, a time-series state inference module, a logistics topology map construction module, and a cross-domain consistency verification module.

[0044] The video acquisition module is deployed at key operational locations in the port to collect continuous video stream data, including containers.

[0045] The image processing module analyzes the acquired video stream frame by frame, extracting the visual features of the containers and container number characters from each frame.

[0046] The temporal state inference module performs temporal state inference on the identity of containers in consecutive multi-frame images based on a Bayesian probability model, thereby eliminating the random error of single-frame recognition results;

[0047] The logistics topology map construction module establishes a topological relationship model of the port logistics network, representing the spatial transfer logic of containers between various operational stages such as unloading, handling, stacking, and transshipment.

[0048] The cross-domain consistency verification module performs consistency matching verification between the time-series state inference results and the logistics topology map, and outputs cargo handling data that meets the logistics deterministic constraints.

[0049] The video acquisition module is used to collect continuous video stream data containing containers at key operational locations in the port. Multiple camera devices are deployed at key nodes in the port operation area, including ship quay crane operation areas, wharf front transport channels, yard entrances and exits, transfer lanes, and port gates.

[0050] The acquisition time of each camera device is synchronized and calibrated using a unified clock source;

[0051] In some implementations, time synchronization uses the NTP network time protocol. The switch is deployed with an NTP server, and each camera device is configured as an NTP client to request time synchronization from the NTP server, so as to ensure that video data collected from different locations can be accurately aligned in the time dimension.

[0052] For scenarios requiring higher precision, a GPS timing solution can be used, which involves deploying GPS receiver modules at key locations to provide clock signals to the camera equipment.

[0053] The video acquisition module outputs continuous video stream data, which is transmitted in real time to the image processing module for subsequent processing via the network. The video stream transmission adopts the RTSP real-time streaming protocol, the transmission link is an Ethernet switch, and the bandwidth allocation strategy adopts QoS service quality assurance to ensure the real-time performance and stability of video data transmission. The video stream data format is H.264 or H.265 compressed encoding.

[0054] In some implementations, a single camera device generates approximately 500 kilobytes of video data per second, and a 24-port gigabit switch can simultaneously access more than 100 video streams.

[0055] The image processing module receives the continuous video stream output by the video acquisition module, extracts video frame images at preset frame intervals using a sliding window method, and performs preprocessing, target detection, visual feature extraction, and box number character recognition on each frame image. The image processing module is an embedded computing platform.

[0056] The frame interval is dynamically adjusted according to the movement speed of the operation scene. In the operation scene, the operation vehicle travels at a relatively high speed and the container displacement between adjacent frames is large. At this time, the frame interval can be set to 1 to 2 frames to ensure that the container has enough overlapping area in consecutive frames for subsequent association matching.

[0057] In low-speed operation scenarios, the operation vehicle travels slowly, and the frame interval can be set to 3 to 5 frames to reduce the computational load and data storage. The dynamic adjustment of the frame interval is automatically determined by analyzing the statistical characteristics of the target displacement in historical frames.

[0058] The extracted video frame images are processed through a preprocessing pipeline, which includes three processing steps: distortion correction, contrast enhancement, and noise suppression.

[0059] Distortion calibration employs Zhang Zhengyou's calibration algorithm based on camera calibration. Specifically, a checkerboard calibration board is used to acquire multiple calibration images from different angles within the camera's field of view.

[0060] Corner pixel coordinates are extracted using calibrated images to establish the correspondence between object plane coordinates and image plane coordinates;

[0061] The camera's intrinsic parameters, distortion coefficients, and other parameters are solved by the least squares method. The camera's intrinsic parameters include focal length and principal point, and the distortion coefficients include radial distortion and tangential distortion.

[0062] For example, distortion calibration precisely corrects the geometric distortion caused by barrel or pincushion distortion, resulting in an edge position error of less than 2 pixels in the corrected image.

[0063] Contrast enhancement employs an adaptive histogram equalization algorithm. This algorithm divides the image into several sub-regions, performs histogram equalization on each sub-region separately, and then synthesizes the final enhanced image through bilinear interpolation. The parameters for contrast enhancement are set as follows:

[0064] The sub-region size is 8×8 pixels, and the cropping is limited to twice the histogram peak value. After enhancement processing, the visual contrast between the container and the background is improved, which is beneficial to the accuracy of subsequent target detection and character recognition.

[0065] Noise suppression employs a spatial noise reduction method based on Gaussian filtering. Gaussian filtering has a good suppression effect on high-frequency noise while maintaining the clarity of the container edges.

[0066] The preprocessed video frame images enter the target detection stage, where the image processing module uses a target detection model based on a deep convolutional neural network to identify container regions in the image.

[0067] The object detection model can adopt the YOLOv5 architecture, which achieves a good balance between detection accuracy and inference speed.

[0068] The neck network employs a combination of a feature pyramid network and a path aggregation network to achieve multi-scale feature fusion;

[0069] The detection head outputs detection results at three scales, corresponding to large-scale, medium-scale, and small-scale targets, respectively. Preprocessing includes pixel value normalization and image scaling. The model output includes bounding box coordinates, confidence scores, and class labels.

[0070] The model training adopts a transfer learning strategy. Specifically, it is pre-trained on the COCO dataset, which contains tens of millions of labeled images covering common target categories such as people, vehicles, and animals. The pre-trained weights enable the model to have good feature extraction capabilities. Subsequently, container images collected in port scenes are used for fine-tuning training. The fine-tuning dataset contains tens of millions of labeled images, with the labels being the bounding boxes and category labels of the containers. The category labels are used to distinguish between different sizes of standard containers, such as 40-foot standard containers and 45-foot high cube containers.

[0071] In some implementations, the training parameters are set as follows:

[0072] The initial learning rate is 0.001, the batch size is 16, the training epochs are 100, the optimizer can be AdamW, and during model inference, the IoU threshold for non-maximum suppression is set to 0.5, the confidence threshold is set to 0.7, and the container target areas with a confidence level higher than 0.7 are selected.

[0073] For the detected container target area, the image processing module further extracts its visual features and performs container number character recognition. The visual features include color histogram features, texture features, and shape contour features, which are used for subsequent cross-frame target association matching.

[0074] The method for extracting color histogram features is as follows:

[0075] The image within the detection box is converted to the HSV color space, and histograms for the H, S, and V channels are calculated respectively. The quantization level of the H channel histogram is 180, and the quantization levels of the S and V channels are 256 each. The total dimension of the color histogram feature vector is 180+256+256=692. The color histogram feature can characterize the paint color information of the container and distinguish containers of different colors.

[0076] The texture feature extraction method adopts local binary mode. The local binary mode operator takes each pixel as the center, takes the gray value of its neighboring pixels and compares it with the center pixel to obtain a binary code. The binary code is converted into a decimal number and a histogram is calculated to obtain the local binary mode feature. The local binary mode feature vector has a dimension of 256, which can characterize the texture details of the container surface.

[0077] The shape contour feature extraction method uses the histogram of oriented gradients. The image within the detection box is scaled, and then the gradient direction of the pixels is calculated and the histogram of gradient direction is statistically analyzed. The histogram of oriented gradients represents the edge contour information of the container.

[0078] After normalization, the three visual feature vectors are concatenated into a unified multidimensional feature vector with a total dimension of 692+256+3780=4728. This feature vector is used for subsequent cross-frame target association matching and temporal state inference.

[0079] The box number character recognition adopts an end-to-end optical character recognition model based on convolutional neural networks. The model architecture consists of two parts: convolutional layers and fully connected layers. The convolutional layer includes 5 convolutional blocks, each of which contains a convolutional layer, a batch normalization layer, and a ReLU activation function. The fully connected layer includes 3 fully connected layers, and the number of output layer nodes corresponds to the number of box number character categories.

[0080] The training data for the container number character recognition model includes more than 100,000 images of port containers labeled with container number characters. The character categories cover all characters and numbers that may appear in the container number, including 26 uppercase English letters, 10 Arabic numerals, and punctuation marks that may appear in the container number format.

[0081] The training data is augmented through data augmentation, including operations such as random rotation, random scaling, random brightness adjustment, and random noise addition.

[0082] In some implementations, the training parameters are set as follows:

[0083] The initial learning rate is 0.0001, the batch size is 32, the training epochs are 200, and the optimizer can be AdamW.

[0084] The model outputs the identified box number string and its corresponding confidence score. Identification results with a confidence score higher than 0.8 are entered into the subsequent processing flow, while identification results with a confidence score lower than 0.8 are marked as low confidence and their weight is reduced in the time-series state inference module.

[0085] The recognition results output by the image processing module are transmitted to the temporal state inference module in a structured data format. The data format includes the following fields: frame timestamp, camera number, detection box coordinates, visual feature vector, box number string, box number confidence, and box type.

[0086] The temporal state inference module performs temporal state inference on the identity of containers in consecutive multi-frame images based on a Bayesian probability model, eliminating the random error of single-frame recognition results. The temporal state inference module is implemented based on the TensorFlow or PyTorch framework, and the parameters of the Bayesian probability model are continuously optimized through a combination of offline learning and online updates.

[0087] Bayesian probabilistic models model the movement state of containers as hidden Markov processes, which include a sequence of hidden states and a sequence of observable states.

[0088] Hidden state sequences are used to characterize the true identity and spatial location of containers;

[0089] The observable state sequence is used to characterize the visual recognition results output by the image processing module.

[0090] The prior probabilities of a Bayesian probability model are constructed based on historical statistical data, and there are two types of prior probabilities:

[0091] That is, the transition probability and the emission probability.

[0092] The transition probability characterizes the prior probability that a container maintains the same identity or undergoes a state transition between consecutive frames;

[0093] In port logistics scenarios, container state transitions mainly fall into three categories: continuous appearance of the same target, new target entering the field of view, and existing target leaving the field of view. The transition probability matrix is ​​obtained by analyzing the distribution patterns of container trajectories in historical port operation data. The specific analysis method is as follows:

[0094] Collect all identification results records from the past three months and connect the target trajectories of the same box number in chronological order; count the number of state transitions and calculate the frequency of each type of transition as an estimate of the transition probability.

[0095] In some implementations, the typical values ​​for the transition probability matrix are as follows:

[0096] The transition probabilities of maintaining the current state, the transition probabilities of a new target appearing, and the transition probabilities of a target disappearing reflect the typical patterns of container movement in port operations:

[0097] In most cases, containers maintain the same identity in consecutive frames, but occasionally a new container will enter the frame or an existing container will leave the frame.

[0098] The emission probability represents the prior probability of the image processing module outputting various recognition results given the actual state of the container. The emission probability is obtained by analyzing the consistency between historical recognition results and real labeled data. The specific analysis method is as follows:

[0099] The recognition results of the image processing module are compared with the real data labeled by humans, and the frequency of each recognition result under the given real conditions is statistically analyzed as an estimate of the emission probability.

[0100] The calculation of the transmission probability needs to consider various error scenarios: normal recognition, that is, the actual state is consistent with the recognition result; character recognition error, that is, the box number character is partially incorrect; and complete error, that is, the recognition result is completely different from the actual state. The frequency of occurrence of each type of error is obtained through statistical analysis to form a transmission probability matrix.

[0101] The prior probability matrix is ​​learned from historical data during the system initialization phase and is continuously updated during system operation;

[0102] During system operation, the latest state transition frequency and identification error frequency are statistically analyzed every 24 hours, and the prior probability matrix is ​​dynamically updated so that the model can adapt to changes in port operation patterns.

[0103] The temporal state inference module uses the Viterbi algorithm to solve for the optimal state sequence. The Viterbi algorithm searches for the optimal path that maximizes the posterior probability among all possible state sequences through dynamic programming.

[0104] The specific implementation steps of the Viterbi algorithm are as follows:

[0105] The first step is initialization. Set the total number of frames in the sequence to T and the total number of possible states of the system to N. Create two matrices, each with the dimension of the total number of states N × the total number of frames T. The first matrix is ​​used to store the optimal partial path probability of each state at each time step, and the second matrix is ​​used to store the reverse index pointers required for state backtracking.

[0106] For the first time step of the sequence, the partial path probability of each state is initialized. This probability is obtained by multiplying the initial probability of the corresponding state by the emission probability of that state generating the current observation.

[0107] The second step is to recursively calculate the partial path probability for each state, from the second time step to the last time step:

[0108] For each state at the current time, its optimal partial path probability is equal to the optimal partial path probabilities of all states at the previous time, multiplied by the transition probability of the corresponding state to the current state, and the maximum value is taken, and then multiplied by the emission probability of the current state generating the current observation.

[0109] Simultaneously, record the state at the previous moment that caused the product to reach its maximum value, and store it in the reverse index pointer matrix.

[0110] The third step is to terminate the judgment. At the last moment of the sequence, select the maximum value from the optimal partial path probabilities of all states. This probability is the total probability of the optimal path, and determine the state at the last moment corresponding to the maximum probability.

[0111] The fourth step is path backtracking. Based on the reverse index pointer matrix, starting from the optimal state determined at the last moment, backtracking forward moment by moment to determine the optimal state at each moment in turn, and finally obtaining the complete sequence of optimal states.

[0112] The optimal state sequence output by the Viterbi algorithm is the true identity attribution result of the container in multiple consecutive frames of images. For each frame of image, the system synchronously outputs the posterior probability distribution of each state corresponding to that frame, and uses the entropy value as the confidence evaluation index of the recognition result.

[0113] The temporal state inference module integrates a confidence propagation mechanism to quantify the reliability of the state inference results. This mechanism assesses confidence by calculating the entropy of the posterior probability distribution corresponding to each frame's recognition result.

[0114] The lower the entropy value, the higher the certainty and credibility of the inference result;

[0115] The higher the entropy value, the lower the certainty and credibility of the inference result.

[0116] The entropy value is calculated as follows: sum the product of the posterior probability of each state and the logarithm of that probability, then take the negative of the sum to obtain the entropy value of the current frame. The system has a preset confidence threshold.

[0117] When the entropy value is less than 0.5, the recognition result of the current frame is determined to be of high confidence.

[0118] When the entropy value is in the range of 0.5 to 1.5, it is judged as having medium confidence level, excluding extreme values;

[0119] When the entropy value is greater than 1.5, it is judged as low confidence.

[0120] When the confidence level of the recognition result of a certain frame image is lower than the preset threshold, the module automatically marks the frame result as low confidence and reduces its weight in subsequent processing;

[0121] In the recursive calculation of the Viterbi algorithm, the emission probability of low-confidence frames is multiplied by a weight factor of 0.5 to reduce the contribution of that frame to the final result. This confidence propagation mechanism effectively eliminates the random error of single-frame recognition results and improves the robustness of the recognition system in complex environments.

[0122] The logistics topology graph construction module establishes a topological relationship model of the port logistics network, abstracting the port operation process into a directed graph structure containing nodes and edges;

[0123] The logistics topology graph construction module can use Neo4j graph database as the storage backend, supporting efficient graph querying and dynamic updates.

[0124] The nodes in the logistics topology graph represent logistics operation entities, including:

[0125] Vessel berth node: Attributes include berth number, berth length, water depth, berthing capacity, etc. Each vessel berth has a unique geographic coordinate attribute, which is used for subsequent location mapping;

[0126] Quay crane loading / unloading point node: Attributes include quay crane number, location coordinates, and operational capacity. The reachability between the quay crane loading / unloading point and the ship berth is connected by edges.

[0127] Yard bay node: Attributes include bay number, yard area, layer limit, number of container bays, etc. Yard bays are the basic modules for container storage;

[0128] Lane node for transfer stack: Attributes include lane number, start coordinates, end coordinates, speed limit, etc.

[0129] Port gate node: Attributes include gate number, opening time, throughput, etc.

[0130] The edges of the logistics topology graph represent the transfer paths and relationships between logistics operation entities, including:

[0131] Unloading path side: Connecting the ship berth and the quay crane loading and unloading point, indicating the transfer path of containers from the ship to the dock front;

[0132] Transport path edge: Connects the quay crane loading and unloading point and the yard bay, indicating the transfer path of the container from the quay front to the yard;

[0133] Transfer path edge: connects two yard bays, indicating the transfer movement of containers within the yard;

[0134] Port access path: connects the yard bay to the port gate, indicating the transfer path of containers from the yard to the port.

[0135] Each edge is assigned the following attributes:

[0136] Transfer time is the average transfer time based on historical operation data.

[0137] Transfer distance, a straight-line distance calculated based on geographic coordinates;

[0138] Throughput capacity, the maximum number of containers that can pass through the route per hour.

[0139] The node and edge attributes of the logistics topology map are automatically constructed by analyzing the business data of the port operating system and the terminal production system. The system obtains ship berth and loading / unloading plan data from the ship planning module of the terminal operating system, yard bay occupancy information from the yard management module, and port vehicle scheduling plan from the port scheduling module. Based on this business data, the logistics topology map construction module automatically generates possible transfer paths between each operation link.

[0140] The specific business data access method is as follows: Establish a data interface with the terminal operating system to receive business data such as ship plans, yard plans, and port arrival plans. Ship plans include information such as ship berthing and departure times, container loading and unloading volume, and container space distribution.

[0141] The yard plan includes information such as bay space occupancy, planned container turnover, and stack transfer instructions;

[0142] The port access plan includes information such as port vehicle reservations and gate openings. The logistics topology map construction module analyzes this business data, extracts logistics entities and transfer relationships, and automatically updates the map structure.

[0143] The logistics topology map construction module also includes a dynamic update mechanism to adapt to real-time changes in port operation processes. This dynamic update mechanism adopts an event-driven architecture, namely:

[0144] Event listening: The module subscribes to the event queue of the terminal operating system and listens for business change events. The types of business change events include: ship schedule change events (adjustment of ship berthing and departure times), yard bay space change events (bay space being occupied or released), port route change events (port route adjustment), etc.

[0145] Event parsing: When a business change event is detected, the module immediately parses the event content and extracts the entity type and change parameters.

[0146] Graph updates are performed based on the changes. When a ship's schedule changes, the attributes of the corresponding ship berth node and the attributes of related edges are updated. When a storage yard bay changes, the occupancy status of the corresponding bay node is updated. When a port route changes, port route edges are updated or added.

[0147] The dynamic update mechanism can ensure the synchronization between the logistics topology map and the actual operation process. When the port operation process changes, the map can be updated within minutes, ensuring that the cross-domain consistency verification module can obtain the latest topology constraint information.

[0148] The cross-domain consistency verification module performs consistency matching verification between the container identification results output by the time-series state inference module and the logistics topology map. The logic of the cross-domain consistency verification module is as follows:

[0149] The movement trajectory of any container in the port logistics network must follow the feasible path defined by the logistics topology map. Any identification result that does not conform to the logistics topology constraints should be judged as abnormal and trigger the verification process.

[0150] The cross-domain consistency verification module's verification process is divided into three stages: trajectory generation stage, path matching stage, and exception handling stage.

[0151] During the trajectory generation stage, the cross-domain consistency verification module extracts the spatial coordinates of the container in each frame based on the continuous frame recognition results output by the temporal state inference module, and connects the position coordinates of the same container in multiple frames in chronological order to generate the movement trajectory curve of the container.

[0152] The method for obtaining spatial location coordinates is to map the coordinates of the center point of the detection box in the image to the pixel coordinate system;

[0153] Transform pixel coordinates to the camera coordinate system using camera calibration parameters;

[0154] The camera coordinate system is transformed to the port geographic coordinate system using geographic coordinate registration parameters. The method for obtaining the geographic coordinate registration parameters is as follows:

[0155] Several control points were set up at key locations in the port, and the geographical coordinates and image pixel coordinates of each control point were measured. A coordinate transformation model was established by fitting the least squares method.

[0156] The method for generating the motion trajectory curve is to sort all the identification results of the same box number in chronological order, extract the spatial location coordinates of each frame, and use spline interpolation or linear interpolation to connect the discrete coordinate points into a continuous trajectory curve.

[0157] For two frames with a large time interval, the rationality of the path needs to be considered during interpolation, and constraints can be imposed by combining feasible paths in the logistics topology map.

[0158] During the path matching phase, the cross-domain consistency verification module matches the generated movement trajectory with feasible paths in the logistics topology map. The matching algorithm is based on curve similarity measurement, specifically using Fraser distance or Hausdorff distance.

[0159] The Fraser distance is calculated by representing the trajectory curve as a sequential sequence of points, and the feasible path as a sequential sequence of points as well.

[0160] The Fraser distance is calculated by finding the maximum Euclidean distance between corresponding points on two curves under matching methods that satisfy the point order constraint, and then taking the minimum of this maximum distance among all legal matching methods.

[0161] The smaller the value of the Fraser distance, the more similar the shapes and directions of the two curves are.

[0162] Hausdorff distance is calculated by representing the trajectory curve and feasible paths as sets of points, respectively.

[0163] For each point on the trajectory curve, first calculate its Euclidean distance to all points on the feasible path and take the minimum value;

[0164] Then, taking the maximum of the minimum values ​​for all points on the trajectory, we obtain the Hausdorff distance. The Hausdorff distance is also used to measure the similarity between two curves.

[0165] In some implementations, the threshold for determining the Fraser distance is set to 50 meters;

[0166] When the Fraser distance between the movement trajectory to be verified and a preset feasible path is less than 50 meters, the movement trajectory is determined to meet the logistics topology constraints and the verification is passed.

[0167] When the Fraser distance between the movement trajectory and all preset feasible paths is greater than 50 meters, the movement trajectory is determined to be inconsistent with the logistics topology constraints, and the system enters the anomaly handling stage.

[0168] The specific steps of path matching are to determine the possible starting path based on the starting position of the trajectory, usually selecting all path segments within 100 meters around the starting coordinates as candidate paths.

[0169] Calculate the Fraser distance between the trajectory and each candidate path, and sort them in ascending order of distance;

[0170] If the minimum distance is less than the threshold, the verification passes and the matching result is output.

[0171] If the minimum distance is greater than the threshold, a second check is performed on the path from the second smallest distance to the first smallest distance; if the distance of all candidate paths is greater than the threshold, the process enters the anomaly handling stage.

[0172] During the anomaly handling phase, the cross-domain consistency verification module performs a secondary verification on the identification results that do not conform to the logistics topology constraints. The secondary verification first checks the confidence information output by the time-series state inference module.

[0173] If low-confidence frames are concentrated within a certain continuous time interval, it is determined that the results for that segment may be misidentified due to image quality issues. Image quality issues may be caused by the following factors:

[0174] Strong light and backlighting can cause recognition difficulties, cameras may temporarily lose sight of the target, and dirt on the container surface can affect character recognition. The verification module attempts to correct the trajectory segment by interpolation or extrapolation. The interpolation method uses valid trajectory points before and after the segment to generate the position of the intermediate point through linear interpolation or spline interpolation. The extrapolation method uses the trend of the trajectory to extrapolate and predict the position of the missing point.

[0175] If low-confidence frames are scattered throughout the entire time interval, it is determined that the current recognition result may contain systematic errors. Systematic errors may be caused by the following factors:

[0176] If the target detection model misses or falsely detects, the character recognition model has not been updated for a long time, resulting in a decrease in recognition rate, or the lighting conditions change, the verification module sends alarm information to the back-end management system. The alarm content includes the anomaly type, the abnormal time period, the abnormal trajectory description, etc. At the same time, it outputs the downgraded inventory data and marks the suspicious interval in the data.

[0177] The specific method for downgrading is to set a flag requiring manual review in the data quality indicator field for data marked as suspicious.

[0178] The data is compared a second time with the actual operation data in the terminal operating system. If the comparison is consistent, the data is retained.

[0179] If the comparison is inconsistent, it will be marked as data anomaly and sent to manual review.

[0180] The cargo handling data processed by the cross-domain consistency verification module has a high degree of logistics certainty. This is because each piece of data output by the verification module has undergone noise reduction processing for time-series state inference and verification filtering for logistics topology constraints, which improves its consistency with the actual logistics situation.

[0181] The final output cargo handling data includes the following fields: container number, identification location, identification time, vessel of origin, target yard, and data quality indicators, including confidence score, verification status, and suspicious interval markers.

[0182] Example:

[0183] This embodiment constructs an application scenario for a cargo identification system for a container terminal of an international hub port. The terminal has an annual throughput of over 5 million TEUs and an average daily loading and unloading volume of over 30,000 TEUs during peak operating hours. The system is deployed in three key operating areas of the terminal: the ship quay crane operation area, the terminal front transportation channel, the yard entrance and exit, and the port gate.

[0184] The system is deployed using the port tallying and identification scheme of this invention. The image processing module can be based on an embedded computing platform, running the YOLOv5 object detection model and the CRNN character recognition model. The temporal state inference module is based on the Bayesian probability model, with parameters learned from historical data and continuously updated. The logistics topology map construction module is built based on the Neo4j graph database and obtains business data from the terminal operating system in real time. The cross-domain consistency verification module realizes trajectory generation, path matching and anomaly handling functions.

[0185] The video acquisition module continuously acquires video streams from several points according to preset parameters, with a data transmission latency of less than 200 milliseconds, meeting the requirements for real-time processing.

[0186] The image processing module parses the received video stream frame by frame. The YOLOv5 target detection model has a detection accuracy of 96.3% and a recall of 94.8% in port scenarios. The CRNN character recognition model has a recognition accuracy of 93.5%. For clear container number images under standard working conditions, the recognition accuracy can reach over 97%.

[0187] The temporal state inference module performs Bayesian probability inference on the recognition results of multiple consecutive frames. The typical values ​​of the transition probability matrix constructed by the system are:

[0188] The probability matrix for maintaining the current state is 0.85, for a new target appearing it is 0.10, and for a target disappearing it is 0.05. This matrix is ​​obtained through analysis of historical data.

[0189] The probability of normal recognition is 0.88, the probability of character recognition error is 0.09, and the probability of complete error is 0.03. The accuracy of the state sequence output by the Viterbi algorithm reaches 98.2%, which is 13.2 percentage points higher than the accuracy of 85% for single-frame recognition.

[0190] The confidence propagation mechanism of the temporal state inference module effectively identified low-confidence frames. During system operation, a total of 12,000 low-confidence frames were marked, accounting for 2.4% of the total number of frames. These low-confidence frames were mainly concentrated in periods with complex lighting conditions, such as sunrise and sunset, and periods of strong light.

[0191] The logistics topology graph construction module successfully built a port logistics network topology graph containing 326 nodes and 487 edges. The data interface between the module and the terminal operating system is running normally. A total of 15,000 business change events have been processed, with an average processing latency of less than 30 seconds.

[0192] The cross-domain consistency verification module performed logistics topology consistency verification on all identification results, with a pass rate of 97.8%. The 2.2% of identification results that failed the verification entered the anomaly handling process. After secondary verification and analysis, 1.5% of them were confirmed as identification errors and 0.7% were confirmed as system anomalies.

[0193] The final output of cargo handling data was 99.5% consistent with the data in the terminal operating system. When the terminal operators manually checked the data, they found that the differences were mainly due to a few special cases, and the system's recognition accuracy had reached a high level.

[0194] Comparative example:

[0195] A comparative test was conducted using a traditional port cargo tallying and recognition system based on single-frame image processing. The traditional system deployed the same number of cameras for video acquisition, but the image processing module directly output the single-frame recognition result without performing temporal state inference. The system does not have the functions of logistics topology mapping and cross-domain consistency verification, and only filters the results based on simple confidence threshold filtering.

[0196] During the same three-month operating cycle, the traditional system also handled more than 50,000 ship loading and unloading operations.

[0197] The parameters of the video acquisition module are the same as those in the embodiment, and the same video data was acquired.

[0198] The target detection and character recognition models of the image processing module are the same as those in the example. The single-frame recognition accuracy is 85%. Due to the lack of a temporal state inference mechanism, the system directly outputs the single-frame recognition results.

[0199] Traditional systems only use simple confidence threshold filtering for result processing. The confidence threshold is set to 0.8. When the confidence of the recognition result is lower than 0.8, it is discarded directly, and when the confidence is higher than 0.8, it is output directly.

[0200] Due to the lack of time-series inference and logistics topology verification, the cargo handling data output by traditional systems contains obvious errors. Statistical results show that:

[0201] Duplicate counting errors accounted for 7.2% of the total number of identifications, meaning that the same container was repeatedly counted as multiple independent targets in different frames; omission counting errors accounted for 8.5% of the total number of identifications, meaning that some containers were missed due to single-frame identification failure; and logistics logic errors accounted for 5.3% of the total number of identifications, meaning that the output identification results did not match the actual logistics path.

[0202] Table 1: Comparison of Embodiments of the Invention with Comparative Examples

[0203] Evaluation indicators Comparative Example Example Technological improvement effect Single frame recognition accuracy 85% 85% Same hardware foundation Accuracy after time series inference not applicable 98.2% Achieving something from nothing Improved recognition accuracy benchmark value +13.2% promote Repeat count error rate 7.2% 0.8% Reduced by 89% Omission Count Error Rate 8.5% 1.1% Reduced by 87% Logistics logic error rate 5.3% 0.4% Reduced by 92% Consistency of inventory data 86.2% 99.5% An increase of 15.4% Low confidence frame processing Simply discard Confidence propagation weighting Intelligent processing Anomaly detection and handling Not available Secondary verification and alarm reliability improvement Multi-machine collaboration capability Not available Cross-domain association tracking Achieve collaboration System response time real time Increased by approximately 50ms Acceptable

[0204] As can be seen from the above comparative data, by working together with the time-series state inference module and the logistics topology map, the recognition accuracy of the traditional single-frame processing system can be improved. The accuracy is greatly improved by using the time-series inference of the Bayesian probability model. This improvement is directly due to the joint noise reduction processing of the recognition results of multiple consecutive frames by the time-series state inference.

[0205] The improved consistency between tally data and the terminal operating system demonstrates the high reliability of the double-verified tally data, which can be directly used as input for key decisions in the terminal operating system.

[0206] Furthermore, in terms of system architecture design, this invention adopts a distributed design that combines microservice architecture with message queues. Each functional module runs as an independent microservice and transmits data through message queues. The message queues can use Apache Kafka to support high-throughput real-time data stream transmission. The microservice architecture enables each module to be independently expanded and upgraded, improving the maintainability and scalability of the system.

[0207] In terms of data storage, the intermediate results of the time-series state inference module can be stored using a Redis in-memory database, which supports millisecond-level data read and write latency. The logistics topology map can be stored using a Neo4j graph database, which supports efficient graph queries and path searches. The sorting results data can be stored using a MySQL relational database, which supports complex transaction processing and data analysis.

[0208] In terms of communication protocols, data interaction between modules adopts the gRPC remote procedure call protocol. gRPC is based on the HTTP / 2 protocol and supports features such as bidirectional streaming and header compression, which has high communication efficiency. The data format can use Protocol Buffers, which has higher serialization and deserialization efficiency than JSON.

[0209] Video streaming uses TLS encryption to prevent data from being eavesdropped on or tampered with. The API interface uses OAuth2.0 authentication, so only authorized users can access system functions. Sensitive operations such as parameter modification and model update require secondary verification to ensure system security.

[0210] In terms of operation and maintenance monitoring, the system includes Prometheus monitoring and Grafana visualization components. Monitoring metrics include CPU utilization, memory usage, GPU utilization, model inference latency, message queue backlog, etc. When the metrics exceed the preset threshold, alarms are automatically triggered, and multiple notification methods such as DingTalk, SMS, and email are supported.

[0211] In summary, this invention, through the construction of a comprehensive technical solution integrating a time-series state inference module, a logistics topology map construction module, and a cross-domain consistency verification module, can achieve a technological leap from single-frame image processing to multi-frame time-series inference, and from visual perception to logistics determinism. The background technical contradictions resolved by this invention are clear and explicit, including counting errors caused by single-frame processing mode, the disconnect between visual recognition and logistics determinism, and the data link disconnection problem in multi-machine collaborative scenarios.

[0212] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cargo tallying identification system for port tallying equipment, characterized in that, include: The video acquisition module is deployed at key operational locations in the port to collect video stream data containing containers. The image processing module parses the acquired video stream data frame by frame, extracting the visual features of the container and the container number character information from each frame of the video stream data; The temporal state inference module performs temporal state inference on the identity of containers in multi-frame images based on a Bayesian probability model, thus eliminating the error of single-frame recognition results; The logistics topology map construction module establishes a topological relationship model of the port logistics network to represent the spatial transfer logic of containers between operational stages. The cross-domain consistency verification module performs consistency matching verification between the time-series state inference results and the logistics topology map, and outputs cargo handling data that conforms to logistics constraints.

2. The equipment tallying and identification system for port tallying as described in claim 1, characterized in that, The video acquisition module includes multiple camera devices in the port operation area. The camera devices are deployed in the ship quay crane operation area, the transport channel at the front of the wharf, the yard entrance and exit, the transfer roadway, and the port gate. The acquisition time of the camera equipment is synchronized and calibrated using a unified clock source.

3. The equipment tallying and identification system for port tallying as described in claim 1, characterized in that, The image processing module extracts video frame images at preset frame intervals using a sliding window method. The frame intervals are adjusted according to the movement speed of the work scene. The extracted video frame images undergo distortion correction, contrast adjustment, and noise reduction.

4. A cargo tallying identification system for port tallying equipment according to claim 1, characterized in that, The image processing module uses a target detection model based on a deep convolutional neural network to identify container regions in the image. The target detection model outputs the detection box coordinates, confidence score, and category label. The category label is used to distinguish different container types. Visual features include color histogram features, texture features, and shape contour features, which are used for cross-frame target association and matching; The box number character recognition adopts an end-to-end optical character recognition model based on convolutional neural networks.

5. A cargo tallying identification system for port tallying equipment according to claim 1, characterized in that, The time-series state inference module models the movement state of the container as a hidden Markov process. The prior probabilities of a Bayesian probability model include: Transition probability is used to characterize the prior probability that a container maintains the same identity or undergoes a state transition between different frames; The emission probability is used to characterize the prior probability of the image processing module outputting the recognition result given the actual state of the container; The prior probability matrix is ​​learned from historical data during the system initialization phase.

6. A cargo tallying identification system for port tallying equipment according to claim 5, characterized in that, The temporal state inference module searches for a path in the state sequence that maximizes the posterior probability using dynamic programming. The temporal state inference module also includes a confidence propagation mechanism, which calculates the posterior probability distribution entropy value of each frame recognition result. When the confidence of the image recognition result is lower than a preset threshold, the image result of that frame is marked as low confidence, and its weight is adjusted in subsequent processing.

7. A cargo tallying identification system for port tallying equipment according to claim 1, characterized in that, The logistics topology graph construction module abstracts the port operation process into a directed graph structure containing nodes and edges.

8. A cargo tallying equipment identification system for port tallying according to claim 7, characterized in that, The logistics topology graph construction module constructs node and edge attributes using business data from the port operating system and the terminal production system. When a business change event pushed by the dock operating system is detected, the corresponding nodes and edges in the topology graph are updated.

9. A cargo tallying equipment identification system for port tallying according to claim 1, characterized in that, The verification process of the cross-domain consistency verification module includes: The trajectory generation stage extracts the spatial coordinates of the container in consecutive frames and generates the movement trajectory curve; The path matching stage involves matching the movement trajectory with the paths in the logistics topology map based on curve similarity. When the movement trajectory does not conform to the logistics topology constraints, it enters the anomaly handling stage, performs secondary verification, and outputs the corrected or marked cargo handling data.

10. A cargo tallying equipment identification system for port tallying according to claim 9, characterized in that, The exception handling phase of the cross-domain consistency verification module includes: When low-confidence frames are concentrated in a certain continuous time interval, it is determined that the result of this segment is misidentified due to the image, and the trajectory is corrected by interpolation or extrapolation. When low-confidence frames are distributed throughout the entire time interval, it is determined that there is a systematic error in the current identification result, an alarm message is sent to the back-end management system and the downgraded inventory data is output.

Images