Water transportation wharf cargo throughput rapid detection method based on multispectral remote sensing

Abstract

This invention relates to the field of network communication control technology, specifically a rapid detection method for cargo throughput at water transport terminals based on multispectral remote sensing. The method includes the following steps: acquiring remote sensing imagery and ship behavior data; establishing a shipyard mapping relationship; identifying dynamic regions of yard reflectivity; marking loading and unloading response blocks; analyzing berthing behavior; filtering abnormal events; matching shipyard trajectories; confirming throughput correspondence; classifying reflectivity patterns; marking cargo range and category; and recording throughput change trends. This invention introduces a correlation mapping mechanism between ship behavior and dynamic yard reflectivity in a time series, and utilizes multi-band reflectivity change trends for jump identification and frequency clustering block positioning. This achieves dynamic response identification of loading and unloading areas and precise marking of cargo flow areas, enabling continuous tracking and classification recording of cargo distribution trends. While ensuring identification accuracy, it enhances dynamic adaptability and processing efficiency across multiple time periods.

Description

Technical Field

[0001] This invention relates to the field of network communication control technology, and in particular to a rapid detection method for cargo throughput at water transport terminals based on multispectral remote sensing. Background Technology

[0002] The field of network communication control technology involves the design of various protocols and communication processing strategies for managing and scheduling data transmission in communication networks. Core aspects include the parsing and execution of communication protocols, the identification and scheduling of network data packets, flow control and error checking of data transmission, and command control and information feedback mechanisms in remote communication. This technical field systematically covers the orderly flow and response control of multi-source information in complex network environments through preset communication rules. Especially in scenarios such as big data collection, multi-terminal interconnection, remote sensing, and intelligent analysis, network communication control technology provides stable guarantees for efficient transmission and centralized processing between devices. Among these, the traditional rapid detection method for cargo throughput at waterway terminals refers to using the waterway terminal area as the detection object, determining the quantity or type of cargo at the port and estimating the throughput scale through image recognition or statistical data analysis. The technical issue addressed by this method is to achieve rapid identification and estimation of changes in the type and quantity of cargo in the terminal area. Traditional methods rely on manual report statistics, customs declaration data integration, or manual interpretation and estimation of the number of ships and storage yard areas in satellite remote sensing images. In practice, this mainly involves extracting data from single-spectral-band images and then classifying them using thresholds for ship identification or cargo zoning. Some methods combine manually drawn samples with image model training for image region recognition and counting, and use cargo unit volume conversion to estimate throughput. These methods involve specific steps such as manual feature extraction, fixed algorithm recognition processes, or rule template matching.

[0003] Traditional methods for detecting cargo throughput at waterway terminals rely on static information processing mechanisms such as manual report compilation, declaration data integration, and image interpretation. These methods are insufficient to meet the rapid response requirements for cargo flow in complex terminal environments. The manual feature extraction process is subject to subjective bias, affecting recognition accuracy. The use of a single spectral band limits the ability to distinguish fine-grained changes in cargo material and distribution. Threshold classification methods cannot dynamically adapt to image changes and are unable to track subtle fluctuations during cargo loading and unloading. Fixed recognition processes and template matching modes are not adaptable enough to multiple time frames and multiple scene changes, resulting in problems such as lag in throughput estimation, fuzzy cargo type recognition, and lack of dynamic response capabilities. These issues affect the efficiency of terminal operations and the real-time and accuracy of management decisions. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a rapid detection method for cargo throughput at waterway terminals based on multispectral remote sensing, comprising the following steps: To achieve the above objectives, the present invention adopts the following technical solution: a rapid detection method for cargo throughput at waterway terminals based on multispectral remote sensing, comprising the following steps: S1: Acquire remote sensing image sequences of water transport terminals, extract the heading angle sequence and positioning trajectory of ships in the berth area, and construct the mapping index relationship between ship behavior and yard images by aligning the heading timestamp with the image frame timestamp, and generate a ship and yard synchronous observation index group. S2: Based on the ship and yard synchronous observation index group, obtain the multi-band reflectance value sequence of the pixel group corresponding to the stacking area in the remote sensing image sequence of the water transport terminal, compare the reflectance jump direction between adjacent bands in turn and identify the section with prominent jump gradient, and generate a dynamic annotation layer for loading and unloading in the yard. S3: Based on the dynamic labeling layer for loading and unloading in the yard, extract the trend of the ship's heading angle change in the berthing area, and determine whether berthing has occurred by comparing whether the ship's berthing position is consistent in the images of the previous and next frames, and generate a list of abnormal berthing ships. S4: Call the reflectivity dynamic area marked in the dynamic labeling layer of the yard loading and unloading, and match it with the ship trajectory frame number in the abnormal berthing ship mark list to identify whether there is a matching yard response block in the corresponding berth area, and generate a throughput behavior confirmation mapping list.

[0005] As a further aspect of the present invention, the ship-yard synchronous observation index group includes a ship positioning trajectory sequence, a heading angle time index, a frame image time index, a berth image spatial mapping relationship, and a shipyard correspondence table. The dynamic annotation layer for loading and unloading in the yard includes reflectivity abrupt change blocks, abrupt gradient peak markers, frequency cluster response regions, continuous abrupt change verification markers, and a dynamic evolution time sequence diagram. The list of abnormal berthing ship markers includes a direction fluctuation frame number index, a berth spatial position change marker, a heading and displacement contradiction judgment result, an abnormal event number, and a time frame comparison result. The throughput behavior confirmation mapping list includes the coordinate range of the intersection area, the time continuity verification result, the berth response matching marker, the throughput behavior identification number, and a shipyard corresponding time mapping table.

[0006] As a further aspect of the present invention, the specific steps of S1 are as follows: S101: Acquire the remote sensing image sequence of the water transport terminal, delineate the berth area of ​​the multi-frame image, perform mask projection operation on the pixels of the corresponding area in each frame image, perform coordinate correction processing on the edge pixels of the berth area of ​​the image in combination with the berth space boundary configuration parameters, and serialize and mark the corrected berth area image according to the frame time order to generate a berth area sequence image set. S102: Based on the berth area sequence image set, detect the ship image contour features in the corresponding berth area in each frame image, extract the contour principal axis direction and combine it with the image frame timestamp, perform matching calculation with the ship identification number according to the heading angle change trend sequence corresponding to the time point, and generate a berth ship heading angle sequence set. S103: Call the berth vessel heading angle sequence set, perform sequence alignment operation according to the timestamp index and the frame timestamp in the berth area sequence image set, retrieve the berth image matrix at the corresponding time point in multiple frames, extract the vessel positioning trajectory and aggregate the vessel number, positioning coordinates and corresponding image frame index information at the time point, and establish a ship and field synchronous observation index group.

[0007] As a further aspect of the present invention, the specific steps of S2 are as follows: S201: Based on the ship and field synchronous observation index group, retrieve the image frame of the stacking area that matches the index frame in the remote sensing image sequence of the water transport terminal, extract the multi-band data vector of the pixel group corresponding to the stacking area in the image frame, extract the reflectance value on the channel according to the band channel order, and generate a stacking area band reflectance value sequence set. S202: Call the set of reflectance values ​​of the stacked area bands, perform difference calculation on the reflectance value vectors between adjacent bands in turn, and determine the jump trend according to the direction of reflectance change. Select the region where the reflectance change amplitude is greater than the jump gradient judgment threshold and mark the position index. Calculate the pixel index distribution density of the jump gradient in multiple frames of images, locate the region with concentrated change density, and generate a high-frequency gradient aggregation distribution map of the stacked area. S203: Based on the high-frequency gradient aggregation distribution map of the stacking area, the reflectance value change sequence of the marked area is compared between multiple frames of images. The consistency value of the jump direction and the fluctuation value of the jump amplitude of the pixels in the continuous frames are calculated. It is determined whether the continuous mutation judgment threshold and the stability change amplitude limit value are met at the same time. If they are met, they are marked as cargo flow reaction blocks. The pixel positions that meet the conditions are aggregated to generate a dynamic labeling layer for loading and unloading in the yard.

[0008] As a further aspect of the present invention, the specific steps of S3 are as follows: S301: Based on the dynamic labeling layer for loading and unloading in the yard, retrieve the corresponding time frame index number, call the heading angle parameter value of the matching frame number in the berth vessel heading angle sequence set, perform differential calculation on the heading angle values ​​in continuous time frames, filter the frame index numbers whose heading angle change amplitude exceeds the direction fluctuation threshold, and generate a list of heading angle fluctuation frame numbers. S302: Call the heading angle fluctuation frame number list, extract the center coordinates of the ship's outline in the corresponding images of multiple frames, calculate the Euclidean distance of the ship's pixel position between adjacent time frames and determine whether it is within the boundary of the same berth area, filter the frame index numbers of the center coordinates that change across berths, and generate a berth area spatial displacement frame number list. S303: According to the list of spatial displacement frame numbers in the berth area, if the trend of the heading angle change deviates from the spatial displacement direction in the image, the ship number corresponding to the time frame is added to the anomaly list, the number information of the discrepancies and the frame index are aggregated, and an anomaly berthing ship mark list is generated.

[0009] As a further embodiment of the present invention, in the list of heading angle fluctuation frame numbers, the heading angle fluctuation threshold is the frame index number corresponding to the time frame where the absolute value of the heading angle change is greater than 15 degrees. In the list of spatial displacement frame numbers for the berth area, the calculation of the Euclidean distance of the pixel position is based on the distance between two points in the image coordinate system where the coordinates of the ship's outline center are in continuous time frames, and the determination of whether it is within the boundary of the same berth area is based on the pixel boundary range of the berth area boundary. The process of adding the ship number corresponding to the time frame to the anomaly list involves comparing the angle between the direction of the change in the heading angle corresponding to each frame in the heading angle fluctuation frame number list and the pixel displacement direction of the corresponding frame in the berth area spatial displacement frame number list. When the angle value is greater than 60 degrees, it is determined that there is a deviation between the heading angle change trend and the spatial displacement direction in the image.

[0010] As a further aspect of the present invention, the specific steps of S4 are as follows: S401: Call the vessel number and corresponding trajectory frame number in the abnormal berthing vessel marking list, combine the reflectivity dynamic area dataset marked in the yard loading and unloading dynamic marking layer, extract the pixel position index and corresponding time frame number of the marked area, and generate a yard response and trajectory frame corresponding index table. S402: Based on the index table corresponding to the yard response and trajectory frames, perform pixel spatial overlap detection on the positioning coordinates of the yard response block and the abnormally berthed vessel in the berth area, retrieve the time frame index of the intersection area, and judge the trajectory overlap of the pixel position change trend of the intersection area in the continuous frames. Filter the berth image frames that meet the intersection area and are continuous in time axis, and generate a list of overlapping frames of ship trajectory and yard response. S403: Call the list of overlapping frames between the ship trajectory and the yard response, extract the corresponding ship number and yard response area number, and aggregate them in time frame order to form a set of four-tuples of ship number, frame number index, berth number and yard area number. Assign a unique mapping identifier to each matching relationship and generate a throughput behavior confirmation mapping list.

[0011] As a further aspect of the present invention, in the index table corresponding to the yard response and trajectory frame, the pixel position index of the reflectivity dynamic region dataset is a set of two-dimensional pixel coordinates extracted in the image coordinate system, and the time frame number is limited to the image frame number that intersects with the frame number in the list of abnormally berthed vessels. The process of pixel spatial overlap detection is based on whether there are at least 3 consecutive pixels in each time frame that intersect the outline of the berth area's reaction block and the positioning coordinates of the abnormally berthed vessel. The determination of trajectory overlap is based on the fact that the change in the Euclidean distance of the pixel positions in the intersection area is less than a set threshold in 3 consecutive frames. The time frames included in the list of overlapping frames between the ship trajectory and the yard response must simultaneously meet the requirements of spatial overlap determination and continuous arrangement of time frame numbers according to the image acquisition sequence. Furthermore, the berth number in the quadruple set is determined based on the berth area to which the corresponding frame number in the list of spatial displacement frames of the berth area belongs.

[0012] As a further aspect of the present invention, the method further includes step S5: S5: Based on the throughput behavior confirmation mapping list, read the band jump curve shape within the pixel distribution range of the loading and unloading area, perform a classification operation on the reflectivity mode, and determine the cargo distribution range and category label corresponding to each loading and unloading based on the mapped material type and response area range. Combined with the time frame change trend, obtain the cargo throughput change record. The cargo throughput change record includes reflectivity pattern classification results, material type labels, response area pixel distribution, cargo distribution time series curve, and loading / unloading category dynamic labels.

[0013] As a further aspect of the present invention, the specific steps of S5 are as follows: S501: Based on the throughput behavior confirmation mapping list, read the corresponding yard response area number and time frame index, identify the pixel position of the loading and unloading area in the corresponding image frame, extract the reflectance value sequence in the multi-band image, calculate the pixel jump amplitude sequence in band order and construct the band jump curve morphology vector to generate a loading and unloading area band jump morphology sequence set. S502: Call the band jump morphology sequence set of the loading and unloading area, perform a combination feature determination on the band jump curve morphology vector of the number of jump nodes, jump direction sequence and amplitude change range, perform a matching operation based on the reflectivity mode features, mark the category number of the response area, record the area boundary index, and generate a cargo distribution category label set. S503: Based on the cargo distribution category label set, extract the classification number and boundary index sequence in continuous time frames, extract the boundary change trajectory of the same category label, calculate the increase or decrease difference of the number of pixels in the same category area between each frame, and construct the time change sequence of multiple category labels to obtain cargo throughput change records.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, a correlation mapping mechanism between ship behavior and yard reflectivity dynamics in a time series is introduced. By leveraging multi-band reflectivity change trends for jump identification and frequency clustering block positioning, dynamic response identification of loading and unloading areas and precise marking of cargo flow areas are achieved. By combining time axis continuity and spatial trajectory changes, trend analysis and anomaly identification of berthing behavior are performed. The yard reaction area and ship activity trajectory are matched to construct a correspondence between throughput behavior. Furthermore, reflectivity pattern classification and cargo type determination are performed by combining band change curves, enabling continuous tracking and classification recording of cargo distribution change trends. This enhances dynamic adaptability and processing efficiency across multiple time periods while ensuring identification accuracy. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a detailed schematic diagram of S1 of the present invention; Figure 3 This is a detailed schematic diagram of S2 of the present invention; Figure 4 This is a detailed schematic diagram of S3 of the present invention; Figure 5 This is a detailed schematic diagram of S4 of the present invention; Figure 6 This is a detailed schematic diagram of S5 of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] Please see Figure 1 This invention provides a rapid detection method for cargo throughput at waterway terminals based on multispectral remote sensing, comprising the following steps: S1: Acquire remote sensing image sequences of water transport terminals, extract the heading angle sequence and positioning trajectory of ships in the berth area, and construct the mapping index relationship between ship behavior and yard images by aligning the heading timestamp with the image frame timestamp, and generate a ship and yard synchronous observation index group. S2: Based on the ship and yard synchronous observation index group, obtain the multi-band reflectance value sequence of the pixel group corresponding to the stacking area in the remote sensing image sequence of the water transport terminal, compare the reflectance jump direction between adjacent bands in turn and identify the segment with prominent jump gradient, locate the area where high frequency gradient changes are concentrated, mark it as the reaction area where there is cargo flow, verify whether there is a continuous reflectance change trend in the area through inter-frame comparison operation, and generate a dynamic annotation layer for loading and unloading in the yard. S3: Based on the dynamic labeling layer of loading and unloading in the yard, extract the trend of the ship's heading angle change in the berthing area, identify the frame number of the direction change fluctuation in multiple time frames, and determine whether berthing has occurred by comparing whether the berth position of the ship in the images of the previous and next frames is consistent. If the change trend does not match the spatial displacement direction, the behavior is marked as abnormal berthing, and an abnormal berthing ship labeling list is generated. S4: Call the reflectivity dynamic area marked in the dynamic annotation layer of the yard loading and unloading, and match it with the ship trajectory frame number in the abnormal berthing ship mark list. Identify whether there is a matching yard response block in the corresponding berth area. If there is an overlapping area and the two are continuous in time axis, mark the ship and yard combination and generate a throughput behavior confirmation mapping list. S5: Based on the throughput behavior, confirm the mapping list, read the band jump curve shape within the pixel distribution range of the loading and unloading area, classify the reflectivity mode, and determine the cargo distribution range and category label corresponding to each loading and unloading based on the mapped material type and response area range. Combined with the time frame change trend, obtain the cargo throughput change record. The ship-yard synchronous observation index group includes ship positioning trajectory sequences, heading angle time index, frame image time index, berth image spatial mapping relationship, and shipyard correspondence table. The dynamic annotation layer for loading and unloading in the yard includes reflectivity mutation blocks, jump gradient peak markers, frequency cluster response regions, continuous mutation verification markers, and dynamic evolution time sequence diagrams. The abnormal berthing ship marker list includes direction fluctuation frame number index, berth spatial position change markers, heading and displacement contradiction judgment results, abnormal event numbers, and time frame comparison results. The throughput behavior confirmation mapping list includes cross-area coordinate range, time continuity verification results, berth response matching markers, throughput behavior identification numbers, and shipyard corresponding time mapping table. The cargo throughput change record includes reflectivity pattern classification results, material type labels, response area pixel distribution, cargo distribution time sequence curves, and loading and unloading category dynamic labels.

[0023] Please see Figure 2 The specific steps of S1 are as follows: S101: Acquire the remote sensing image sequence of the water transport terminal, delineate the berth area of ​​the multi-frame image, perform mask projection operation on the pixels of the corresponding area in each frame image, perform coordinate correction processing on the edge pixels of the berth area of ​​the image in combination with the berth space boundary configuration parameters, and serialize and mark the corrected berth area image according to the frame time order to generate a berth area sequence image set. The monitoring process requires selecting a specific waterway terminal area, such as a typical berth layout area within a port. High-resolution image sequences acquired at multiple time points are downloaded from a remote sensing image service platform. A fixed daily time interval, such as 8:00 AM and 2:00 PM, is chosen to acquire images continuously for a week, resulting in an image sequence of at least 14 frames. Remote sensing image processing tools are used to register the images, adjusting them to a unified spatial reference. After processing, berth areas are identified using vector boundary data or image recognition methods, such as by analyzing the bright areas in the images. The outline of the berth area is determined by extracting structural lines. After delineating the berth area, a mask layer needs to be constructed for this area. The mask is used to preserve the berth portion and shield non-target areas, ensuring that the processing focuses on the berth area. The mask is applied to each frame of the image to obtain an image containing only the berth portion. Then, combined with existing berth space boundary parameters, such as the actual length, width, direction, and center point coordinates of the berth, the coordinates of the boundary area after masking are corrected so that the berth pixel positions in the image correspond to the standard positions. After correction, each frame of the image is uniformly numbered according to the shooting time and arranged in chronological order to generate a berth area sequence image set.

[0024] S102: Based on the berth area sequence image set, detect the ship image contour features in the corresponding berth area in each frame image, extract the contour principal axis direction and combine it with the image frame timestamp, perform matching calculation with the ship identification number according to the heading angle change trend sequence corresponding to the time point, and generate a berth ship heading angle sequence set; To identify the silhouette features of ships within the berth in each frame, target detection tools, such as deep learning detection models, are used to annotate and identify the ships in the image. After identification, the silhouette information of each target region is extracted, the boundary points of the silhouette are identified, and the principal axis direction of the silhouette is analyzed using image analysis methods. This direction represents the tilt angle of the ship in the image. For example, in frames 20250101-0800, the ship numbered M001 is identified. Its silhouette extends slightly to the left from top to bottom, and the estimated principal axis direction is 25 degrees. In frames 20250101-1400, the ship's main axis direction deflects to 35 degrees. Based on this, the main axis angle change information in consecutive frames is obtained to form a sequence of the ship's heading angle changing over time. At the same time, each target is assigned a ship identification number, which is established by judging the spatial position and shape similarity between consecutive frames. Assuming that the ship with the number M001 appears in a series of frames, its main axis angle changes smoothly over time without any sudden changes. The main axis direction can be paired with the shooting time to construct a correspondence table between heading angle and time, and generate a set of heading angle sequences for berthed ships.

[0025] S103: Call the berth vessel heading angle sequence set, perform sequence alignment operation based on the timestamp index and the frame timestamp in the berth area sequence image set, retrieve the berth image matrix corresponding to the time point in multiple frames, extract the vessel positioning trajectory and aggregate the vessel number, positioning coordinates and corresponding image frame index information of the time point, and establish a ship and field synchronous observation index group. Align the time index with the time in the berth area image frames one by one to establish the correspondence between berth image frames and time. For example, extract specific time information from the name of each frame and convert it into a standard format. Match the time points in the heading angle sequence with the time points of the image frames one by one, select the corresponding image frames, and retrieve the pixel matrix of the corresponding berth area in the image. In the pixel matrix, combine the position coordinate information associated with the ship number to extract the center point coordinates of the ship in each image frame and construct the continuous positioning trajectory of the ship in multiple image frames. For example, the ship numbered M001 is located at five different berth area coordinate points in five images. The trajectory path can be marked accordingly. Aggregate the information of each time point, including the ship number, the timestamp of the berth image frame, the berth image frame index, and the positioning coordinates in the image, to construct a structured observation data record. Store each record one by one and establish a ship and field synchronous observation index group.

[0026] Please see Figure 3 The specific steps of S2 are as follows: S201: Based on the ship and field synchronous observation index group, retrieve the image frames of the stacking area that match the index frame in the remote sensing image sequence of the water transport terminal, extract the multi-band data vector of the pixel group corresponding to the stacking area in the image frame, extract the reflectance value on the channel according to the band channel order, and generate a sequence set of band reflectance values ​​of the stacking area. The process involves identifying the name of the remote sensing image frame associated with each index record and its corresponding time point. Then, the corresponding image frame is retrieved from the entire remote sensing image sequence to identify the pixel range corresponding to the stacked area in the image. For example, the location of the area can be extracted from the image using a preset stacked area boundary template or by combining it with an automatic recognition method. The pixel set of the stacked area in the image is read frame by frame, and the original reflectance values ​​of the area in each band channel are extracted. Taking a common remote sensing image such as GF-2 as an example, commonly used multi-band data includes blue, green, red, and near-infrared channels, corresponding to four channels, and each pixel... Each of these four channels has a reflectance value, forming a vector of length 4. This process is repeated in the frame image to extract the reflectance information of pixels in the same area under multiple bands and record it as a set of reflectance vectors with time labels. For example, the reflectance of the stacked area band extracted in frame time 20250101-0800 is [0.21, 0.35, 0.42, 0.53], while in 20250101-1400 it is [0.25, 0.38, 0.40, 0.51], generating a sequence set of reflectance values ​​of the stacked area band.

[0027] S202: Call the reflectance value sequence set of the stacked area bands, perform difference operation on the reflectance value vectors between adjacent bands in turn, and determine the jump trend according to the direction of reflectance change. Select the region where the reflectance change amplitude is greater than the jump gradient judgment threshold and mark the position index. Calculate the pixel index distribution density of the jump gradient in multiple frames of images, locate the region with concentrated change density, and generate a high-frequency gradient cluster distribution map of the stacked area. The process involves sequentially performing interpolation on adjacent bands in each reflectance vector. Specifically, it calculates the reflectance difference between adjacent channels from each pixel's reflectance vector, generating a reflectance change sequence. For example, from [0.21, 0.35, 0.42, 0.53], the interpolation sequence becomes [0.14, 0.07, 0.11]. The sign of the interpolation is recorded to determine the direction of change. A threshold for gradient jump is then used for comparison. For instance, if the threshold is set to 0.10, the first and third terms in the above interpolation sequence are greater than the threshold, indicating a significant band jump at the pixel's location. The corresponding pixel location is then marked with an index. This process is repeated across image frames, counting the number of marked pixel indices in each frame. The spatial distribution of marked pixels is then summarized in the image. Density statistics are performed based on the frequency of pixel index occurrences. For example, if pixels at a certain location in a stacked area all exhibit jumps within 10 consecutive frames, the index occurrence frequency is 10 / 10 = 1, indicating a high-frequency region. A high-frequency gradient aggregation distribution map of the stacked area is then generated.

[0028] S203: Based on the high-frequency gradient aggregation distribution map of the stacking area, the reflectance value change sequence of the marked area is compared between multiple frames of images. The consistency value of the jump direction and the fluctuation value of the jump amplitude of the pixels in the continuous frames are calculated. It is determined whether the continuous mutation judgment threshold and the stability change amplitude limit value are met at the same time. If they are met, they are marked as cargo flow reaction blocks. The pixel positions that meet the conditions are aggregated to generate a dynamic labeling layer for loading and unloading in the yard. The reflectance change sequence of corresponding pixels in multiple frames of images within each marked region is analyzed. The consistency of reflectance change direction in consecutive image frames is compared, and the consistency value of jump direction is statistically calculated. If a pixel shows the same reflectance increase or decrease trend in multiple frames, the pixel has a high consistency value. At the same time, the jump amplitude fluctuation value, that is, the stability of reflectance change amplitude in time series, is calculated. The stability is judged by analyzing the fluctuation range of changes between frames. The above two statistical results are compared with the set continuous change judgment threshold and stability change amplitude limit value. The jump consistency is set to be greater than 0.8 and the fluctuation value is less than 0.05 as the judgment criteria. When a pixel meets both conditions, the area where the pixel is located is determined to be a cargo flow reaction block. The pixel positions that meet the conditions are selected in the image frame, aggregated, and connected to form closed or continuous area boundaries to generate a dynamic labeling layer for yard loading and unloading.

[0029] Please see Figure 4 The specific steps of S3 are as follows: S301: Based on the dynamic labeling layer of loading and unloading in the yard, retrieve the corresponding time frame index number, call the heading angle parameter value of the matching frame number in the berth vessel heading angle sequence set, perform differential calculation on the heading angle values ​​in continuous time frames, filter the frame index numbers whose heading angle change amplitude exceeds the direction fluctuation threshold, and generate a list of heading angle fluctuation frame numbers. Identify the corresponding time frame index number in the labeled layer, that is, read the layer markers of each frame image showing dynamic changes in loading and unloading, and extract the associated image frame number. For example, if the labeled layer shows obvious loading and unloading activities in frames 20250101-0800 and 20250101-1400, then record the frame number as the index. Then, retrieve the heading angle value that matches the frame number from the berth vessel heading angle sequence set, and extract the heading angle list of consecutive frames in chronological order. For two frames that are adjacent in time, calculate the heading angle change value, that is, the difference in heading angle between the two frames. For example, if the heading angle is 25 degrees in frame t1 and 38 degrees in frame t2, the change range is 13 degrees. After repeating the calculation operation, compare the heading angle change value in consecutive frames with the preset direction fluctuation threshold. Assuming the threshold is 10 degrees, the frame pair is considered to have a change exceeding the threshold, and frame t2 is recorded as an abnormal change frame, generating a heading angle fluctuation frame number list.

[0030] S302: Call the heading angle fluctuation frame number list, extract the center coordinates of the ship's outline in the corresponding images of multiple frames, calculate the Euclidean distance of the ship's pixel position between adjacent time frames and determine whether it is within the boundary of the same berth area, filter the frame index numbers of the center coordinates that change across berths, and generate a berth area spatial displacement frame number list. The center coordinates of the ship's outline are extracted from each included frame. The center coordinates are the position of the center point of the outline's bounding box. The coordinate point pairs between consecutive frames are processed in chronological order to calculate the pixel-level displacement distance of the ship between two time points. The Euclidean distance formula is used for measurement, and the magnitude of each displacement distance is recorded. Then, based on the area where the ship's center point is located, it is determined whether it falls within the boundary of the same berth area. The berth boundary information can be obtained from the berth area division layer. The determination method is to compare whether the center point coordinates still belong to the polygon area of ​​the same berth. For example, if the center point of a ship is located in berth 1 in frame t1 and in berth 2 in frame t2, it has moved across berths. The determination requires a spatial position matching operation for each center point. The analysis process is repeated in the retrieved frames to filter the frame indexes of frames with center coordinate jumps and cross-berths, and a list of berth area spatial displacement frame numbers is generated.

[0031] S303: Based on the list of spatial displacement frame numbers in the berth area, if the trend of the heading angle change deviates from the spatial displacement direction in the image, the ship number corresponding to the time frame is added to the anomaly list, the number information of the discrepancies and the frame index are aggregated, and an anomaly berthing ship mark list is generated. The trend of heading angle change and the direction of ship spatial displacement are compared and analyzed separately. The center coordinates of the ship in consecutive frames are extracted according to the time sequence to deduce the actual movement direction in the image. At the same time, the heading angle change trend of the ship in the corresponding frame is collected from the berth ship heading angle sequence. The deviation of the two directions is compared. If the direction indicated by the ship's heading angle deviates from the direction of spatial displacement in the image, that is, the directions are inconsistent, it indicates that the ship is exhibiting abnormal berthing behavior in the image. The ship number information corresponding to the image frame is recorded together with the frame index and added to the abnormal list. If the ship numbered M001 has a heading due north in frames 20250101-0800, but the center coordinates in the frame are significantly shifted to the east, the heading and displacement are inconsistent and it is judged as abnormal. The ship numbers that meet this condition are aggregated with the frame numbers to generate an abnormal berthing ship marker list.

[0032] Please see Figure 5 The specific steps of S4 are as follows: S401: Call the vessel number and corresponding trajectory frame number in the abnormal berthing vessel marker list, combine the reflectivity dynamic area dataset marked in the dynamic annotation layer of the yard loading and unloading, extract the pixel position index and corresponding time frame number of the annotation area, and generate a yard response and trajectory frame corresponding index table. The system needs to read the records in the list. Each record contains the ship number and the time frame number of the anomaly. The corresponding trajectory information needs to be retrieved from the berth ship positioning data. The positioning data includes the center coordinates of the ship, the frame number, and the berth number in each frame. After the positioning trajectory is extracted, the reflectivity dynamic area data marked in the dynamic annotation layer of the yard loading and unloading is retrieved to determine the yard area with significant dynamic response in each frame image. The two-dimensional position index value of the pixels in the area (e.g., the horizontal and vertical pixel coordinates in the image) is extracted. At the same time, the time frame number corresponding to the area is recorded. The above two types of data are compared by time frame, and an index mapping between the ship trajectory frame number and the yard dynamic area frame number is constructed. A relation index table is established by traversal. The ship number and the frame number are paired with the pixel position index of the yard dynamic area in that frame. For example, if the number M001 appears in frame t1, the corresponding yard area number is Y001. The correspondence is recorded, and a yard response and trajectory frame correspondence index table is generated.

[0033] S402: Based on the index table corresponding to the yard response and trajectory frames, perform pixel spatial overlap detection on the positioning coordinates of the yard response block and the abnormally berthed vessel in the berth area, retrieve the time frame index of the intersection area, and judge the trajectory overlap of the pixel position change trend of the intersection area in the continuous frames. Filter the berth image frames that meet the intersection area and are continuous in time axis, and generate a list of overlapping frames of ship trajectory and yard response. Pixel spatial overlap detection is performed on the yard reaction block of the berth area within the frame and the positioning coordinates of the ship. The set of marked pixel positions of the yard area in each frame image is read. At the same time, the center pixel coordinates of the abnormally berthed ship in each frame image are extracted. Then, the spatial judgment logic is used to compare whether the ship's center coordinates fall within the range of the yard area pixel set. If there is an overlap, that is, the ship's center pixel and the dynamic area of ​​the yard have at least one pixel position, then each frame is recorded as a cross frame index. The overlap relationship is analyzed to see if it exists continuously in multiple frames. For example, if a ship is in the yard reaction area in three consecutive frames t1, t2 and t3, it is considered to have a trajectory and yard response overlap trend. The continuous relationship is recorded and the frame index that meets the continuous overlap condition of the time axis is filtered to form a list of overlapping frames of ship trajectory and yard response.

[0034] S403: Call the list of overlapping frames between ship trajectory and yard response, extract the corresponding ship number and yard response area number, aggregate them in time frame order to form a set of four-tuples of ship number, frame number index, berth number and yard area number, assign a unique mapping identifier to each matching relationship, and generate a throughput behavior confirmation mapping list. Extract the corresponding ship number and yard area number, and then combine them with the berth number of the ship in the image frame. Through data aggregation, a combined data structure containing four items is formed: ship number, time frame index, berth number, and yard area number. A unique identifier is assigned to each set of data as a unique mapping identifier for throughput behavior. For example, if number M001 is located at berth P03 in frame 20250101-0800 and overlaps with yard area Y007, then the mapping identifier T001 is assigned to this set of data and recorded as a set item (M001, 20250101-0800, P03, Y007, T001). Frames that meet the overlap relationship are processed sequentially, and the four-tuple combination and unique mapping identifier are summarized to generate a throughput behavior confirmation mapping list.

[0035] Please see Figure 6 The specific steps of S5 are as follows: S501: Based on the throughput behavior confirmation mapping list, read the corresponding yard response area number and time frame index, identify the pixel position of the loading and unloading area in the corresponding image frame, extract the reflectance value sequence in the multi-band image, calculate the pixel jump amplitude sequence in band order and construct the band jump curve morphology vector to generate a loading and unloading area band jump morphology sequence set. The system reads the list of storage yard response area numbers and corresponding time frame indices one by one, locates the specific pixel positions in the image frames that identify the loading and unloading activity areas. The area boundary information is provided by the dynamic annotation layer of the storage yard. First, the area in the image is extracted and processed. Then, a set of pixels is selected within the area. The reflectance values ​​of the pixels in each channel are read from the multi-band remote sensing image. For example, for the Y007 area in a certain image frame, the pixels are extracted and the reflectance values ​​of the red, green, blue, and near-infrared channels are obtained in channel order. The extraction operation is repeated in the frame images taken multiple times in the same area to form the reflectance sequence data of each channel in the time series. Then, the jump amplitude is calculated for the multi-band data of each pixel in channel order, that is, the amount of reflectance change between adjacent bands, to obtain a continuous jump sequence, which is used to represent the response characteristics of each pixel in different spectral channels. The jump amplitude results are arranged according to pixels and combined into a morphological vector of the band jump curve. The vector reflects the spectral response characteristic state of the pixel in the current frame. The band jump morphological vectors generated by the pixels in multiple image frames are integrated to generate a set of band jump morphological sequences for the loading and unloading area.

[0036] S502: Call the band jump morphology sequence set of the loading and unloading area, perform a combination feature determination on the band jump curve morphology vector of the number of jump nodes, jump direction sequence and amplitude change range, perform matching operation based on reflectivity mode features, mark the category number of the response area, record the area boundary index, and generate a cargo distribution category label set. For each set of transition curve morphology vectors, feature combination judgment analysis is performed, mainly including the statistics of the number of transition nodes, the extraction of the sequential combination of transition directions, and the identification of the changing trend of transition amplitude in each interval. For example, if a vector contains three transition nodes, presenting a "rising-falling-rising" directional sequence, and the maximum amplitude interval is located in the near-infrared segment, then the pattern feature combination is a set of key recognition parameters. Multiple typical transition pattern sample libraries are preset, and the transition features of each set of curves are compared and analyzed with the classified features in the sample library for matching operations. The cargo category of the current area is determined by the sample category number that is closest to the feature pattern. For example, if the transition morphology highly matches the standard "container reflection feature" template, then the category number C03 is assigned, and the result is written into the annotation record. At the same time, the boundary index of the current area in the image frame is retained, that is, the pixel coordinate range covered by the recorded area. The processed record items all contain the category number and the spatial coverage area, and are written into the cargo distribution category annotation set one by one to generate the cargo distribution category annotation set.

[0037] S503: Based on the cargo distribution category label set, extract the classification number and boundary index sequence in continuous time frames, extract the boundary change trajectory of the same category label, calculate the increase or decrease difference of the number of pixels in the same category area between each frame, and construct the time change sequence of multi-category labels to obtain cargo throughput change records. The classification number and corresponding boundary index data are extracted from consecutive image frames. The focus is on the change of the regional boundary of the same category label between consecutive frames. The spatial position trajectory of the boundary pixels is recorded. The positions of the same category in different time frames are compared to determine whether the region has undergone morphological changes such as displacement, expansion or contraction. Then, the total number of pixels occupied by the category in each frame is counted. The difference in the number of pixels of the same category region between consecutive frames is calculated. For example, if the C03 category region is 1500 pixels in frame t1 and 1850 pixels in frame t2, the increment is 350 pixels. The process is applicable to category numbering. Time series records are established for different categories. The time change curve of multi-category labels is formed by accumulating the difference between each frame to obtain the cargo throughput change record.

[0038] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A rapid detection method for cargo throughput at waterway terminals based on multispectral remote sensing, characterized in that, Includes the following steps: S1: Acquire remote sensing image sequences of water transport terminals, extract the heading angle sequence and positioning trajectory of ships in the berth area, and construct the mapping index relationship between ship behavior and yard images by aligning the heading timestamp with the image frame timestamp, and generate a ship and yard synchronous observation index group. S2: Based on the ship and yard synchronous observation index group, obtain the multi-band reflectance value sequence of the pixel group corresponding to the stacking area in the remote sensing image sequence of the water transport terminal, compare the reflectance jump direction between adjacent bands in turn and identify the section with prominent jump gradient, and generate a dynamic annotation layer for loading and unloading in the yard. S3: Based on the dynamic labeling layer for loading and unloading in the yard, extract the trend of the ship's heading angle change in the berthing area, and determine whether berthing has occurred by comparing whether the ship's berthing position is consistent in the images of the previous and next frames, and generate a list of abnormal berthing ships. S4: Call the reflectivity dynamic area marked in the dynamic labeling layer of the yard loading and unloading, and match it with the ship trajectory frame number in the abnormal berthing ship mark list to identify whether there is a matching yard response block in the corresponding berth area, and generate a throughput behavior confirmation mapping list.

2. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 1, characterized in that, The ship-yard synchronous observation index group includes ship positioning trajectory sequence, heading angle time index, frame image time index, berth image spatial mapping relationship, and shipyard correspondence table. The dynamic annotation layer for loading and unloading in the yard includes reflectivity mutation block, jump gradient peak marker, frequency cluster response region, continuous mutation verification marker, and dynamic evolution time sequence diagram. The abnormal berthing ship marker list includes direction fluctuation frame number index, berth spatial position change marker, heading and displacement contradiction judgment result, abnormal event number, and time frame comparison result. The throughput behavior confirmation mapping list includes cross area coordinate range, time continuity verification result, berth response matching marker, throughput behavior identification number, and shipyard corresponding time mapping table.

3. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Acquire the remote sensing image sequence of the water transport terminal, delineate the berth area of ​​the multi-frame image, perform mask projection operation on the pixels of the corresponding area in each frame image, perform coordinate correction processing on the edge pixels of the berth area of ​​the image in combination with the berth space boundary configuration parameters, and serialize and mark the corrected berth area image according to the frame time order to generate a berth area sequence image set. S102: Based on the berth area sequence image set, detect the ship image contour features in the corresponding berth area in each frame image, extract the contour principal axis direction and combine it with the image frame timestamp, perform matching calculation with the ship identification number according to the heading angle change trend sequence corresponding to the time point, and generate a berth ship heading angle sequence set. S103: Call the berth vessel heading angle sequence set, perform sequence alignment operation according to the timestamp index and the frame timestamp in the berth area sequence image set, retrieve the berth image matrix at the corresponding time point in multiple frames, extract the vessel positioning trajectory and aggregate the vessel number, positioning coordinates and corresponding image frame index information at the time point, and establish a ship and field synchronous observation index group.

4. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 3, characterized in that, The specific steps of S2 are as follows: S201: Based on the ship and field synchronous observation index group, retrieve the image frame of the stacking area that matches the index frame in the remote sensing image sequence of the water transport terminal, extract the multi-band data vector of the pixel group corresponding to the stacking area in the image frame, extract the reflectance value on the channel according to the band channel order, and generate a stacking area band reflectance value sequence set. S202: Call the set of reflectance values ​​of the stacked area bands, perform difference calculation on the reflectance value vectors between adjacent bands in turn, and determine the jump trend according to the direction of reflectance change. Select the region where the reflectance change amplitude is greater than the jump gradient judgment threshold and mark the position index. Calculate the pixel index distribution density of the jump gradient in multiple frames of images, locate the region with concentrated change density, and generate a high-frequency gradient aggregation distribution map of the stacked area. S203: Based on the high-frequency gradient aggregation distribution map of the stacking area, the reflectance value change sequence of the marked area is compared between multiple frames of images. The consistency value of the jump direction and the fluctuation value of the jump amplitude of the pixels in the continuous frames are calculated. It is determined whether the continuous mutation judgment threshold and the stability change amplitude limit value are met at the same time. If they are met, they are marked as cargo flow reaction blocks. The pixel positions that meet the conditions are aggregated to generate a dynamic labeling layer for loading and unloading in the yard.

5. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 4, characterized in that, The specific steps for S3 are as follows: S301: Based on the dynamic labeling layer for loading and unloading in the yard, retrieve the corresponding time frame index number, call the heading angle parameter value of the matching frame number in the berth vessel heading angle sequence set, perform differential calculation on the heading angle values ​​in continuous time frames, filter the frame index numbers whose heading angle change amplitude exceeds the direction fluctuation threshold, and generate a list of heading angle fluctuation frame numbers. S302: Call the heading angle fluctuation frame number list, extract the center coordinates of the ship's outline in the corresponding images of multiple frames, calculate the Euclidean distance of the ship's pixel position between adjacent time frames and determine whether it is within the boundary of the same berth area, filter the frame index numbers of the center coordinates that change across berths, and generate a berth area spatial displacement frame number list. S303: According to the list of spatial displacement frame numbers in the berth area, if the trend of the heading angle change deviates from the spatial displacement direction in the image, the ship number corresponding to the time frame is added to the anomaly list, the number information of the discrepancies and the frame index are aggregated, and an anomaly berthing ship mark list is generated.

6. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 5, characterized in that, In the list of heading angle fluctuation frame numbers, the heading fluctuation threshold is the frame index number corresponding to the time frame where the absolute value of the heading angle change is greater than 15 degrees. In the list of spatial displacement frame numbers for the berth area, the calculation of the Euclidean distance of the pixel position is based on the distance between two points in the image coordinate system where the coordinates of the ship's outline center are in continuous time frames, and the determination of whether it is within the boundary of the same berth area is based on the pixel boundary range of the berth area boundary. The process of adding the ship number corresponding to the time frame to the anomaly list involves comparing the angle between the direction of the change in the heading angle corresponding to each frame in the heading angle fluctuation frame number list and the pixel displacement direction of the corresponding frame in the berth area spatial displacement frame number list. When the angle value is greater than 60 degrees, it is determined that there is a deviation between the heading angle change trend and the spatial displacement direction in the image.

7. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 5, characterized in that, The specific steps of S4 are as follows: S401: Call the vessel number and corresponding trajectory frame number in the abnormal berthing vessel marking list, combine the reflectivity dynamic area dataset marked in the yard loading and unloading dynamic marking layer, extract the pixel position index and corresponding time frame number of the marked area, and generate a yard response and trajectory frame corresponding index table. S402: Based on the index table corresponding to the yard response and trajectory frames, perform pixel spatial overlap detection on the positioning coordinates of the yard response block and the abnormally berthed vessel in the berth area, retrieve the time frame index of the intersection area, and judge the trajectory overlap of the pixel position change trend of the intersection area in the continuous frames. Filter the berth image frames that meet the intersection area and are continuous in time axis, and generate a list of overlapping frames of ship trajectory and yard response. S403: Call the list of overlapping frames between the ship trajectory and the yard response, extract the corresponding ship number and yard response area number, and aggregate them in time frame order to form a set of four-tuples of ship number, frame number index, berth number and yard area number. Assign a unique mapping identifier to each matching relationship and generate a throughput behavior confirmation mapping list.

8. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 7, characterized in that, In the index table corresponding to the yard response and trajectory frames, the pixel position index of the reflectivity dynamic region dataset is a set of two-dimensional pixel coordinates extracted in the image coordinate system, and the time frame number is limited to the image frame number that intersects with the frame number in the list of abnormally berthed vessels. The process of pixel spatial overlap detection is based on whether there are at least 3 consecutive pixels in each time frame that intersect the outline of the berth area's reaction block and the positioning coordinates of the abnormally berthed vessel. The determination of trajectory overlap is based on the fact that the change in the Euclidean distance of the pixel positions in the intersection area is less than a set threshold in 3 consecutive frames. The time frames included in the list of overlapping frames between the ship trajectory and the yard response must simultaneously meet the requirements of spatial overlap determination and continuous arrangement of time frame numbers according to the image acquisition sequence. Furthermore, the berth number in the quadruple set is determined based on the berth area to which the corresponding frame number in the list of spatial displacement frames of the berth area belongs.

9. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 1, characterized in that, The method further includes step S5: S5: Based on the throughput behavior confirmation mapping list, read the band jump curve shape within the pixel distribution range of the loading and unloading area, perform a classification operation on the reflectivity mode, and determine the cargo distribution range and category label corresponding to each loading and unloading based on the mapped material type and response area range. Combined with the time frame change trend, obtain the cargo throughput change record. The cargo throughput change record includes reflectivity pattern classification results, material type labels, response area pixel distribution, cargo distribution time series curve, and loading / unloading category dynamic labels.

10. The rapid detection method for cargo throughput at a waterway terminal based on multispectral remote sensing according to claim 9, characterized in that, The specific steps of S5 are as follows: S501: Based on the throughput behavior confirmation mapping list, read the corresponding yard response area number and time frame index, identify the pixel position of the loading and unloading area in the corresponding image frame, extract the reflectance value sequence in the multi-band image, calculate the pixel jump amplitude sequence in band order and construct the band jump curve morphology vector to generate a loading and unloading area band jump morphology sequence set. S502: Call the band jump morphology sequence set of the loading and unloading area, perform a combination feature determination on the band jump curve morphology vector of the number of jump nodes, jump direction sequence and amplitude change range, perform a matching operation based on the reflectivity mode features, mark the category number of the response area, record the area boundary index, and generate a cargo distribution category label set. S503: Based on the cargo distribution category label set, extract the classification number and boundary index sequence in continuous time frames, extract the boundary change trajectory of the same category label, calculate the increase or decrease difference of the number of pixels in the same category area between each frame, and construct the time change sequence of multiple category labels to obtain cargo throughput change records.

Images