Automatic intelligent monitoring method and system for ship

Abstract

The invention discloses an automatic intelligent monitoring method and system for a ship, and relates to the technical field of intelligent monitoring. According to the invention, the high-precision intelligent monitoring of the cargo fixing state is realized by establishing dual references of the cargo area image and the pressure characteristics; by extracting a local change area in real time and performing lossless compression, the data transmission and storage burden is greatly reduced, and the real-time performance and integrity of the system are guaranteed. By dynamically selecting the master camera and the slave camera and combining with the posture change of the ship body, the cargo monitoring requirements under different inclination conditions can be flexibly met, and it is ensured that a key area is always in a high-quality visible range. According to the invention, through the frame-level integration and sub-region source tracking technology, the monitoring image has good spatial continuity and traceability, and the accuracy and operability of situation awareness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring, and particularly to a ship automation intelligent monitoring method and system. Background Art

[0002] With the development of the large-scale and intelligent shipping of ships, a variety of emerging technologies have gradually been introduced in the field of ship cargo monitoring, including cargo status detection based on image recognition, pressure sensor data acquisition and analysis, and environmental parameter assisted monitoring. These technical means have played a positive role in improving the efficiency of cargo loading management and ensuring the safety of cargo transportation. Especially in unattended or remote monitoring scenarios, automated monitoring with the help of visual data and sensor data has become an important part of the ship intelligent management system. In the prior art, for the status monitoring of the cargo hold or cargo area, it mainly focuses on judging the distribution characteristics of the cargo based on image processing, regulating the cargo hold environment based on temperature and environmental parameters, and applying deep learning classifiers to realize the recognition of the cargo hold status.

[0003] For example, CN119762821A discloses an intelligent monitoring method and system for cargo hold working data. It obtains the cargo hold image through a camera, extracts features via an edge detection algorithm, divides sub-regions to count the pixel distribution, and judges the uniformity of the cargo distribution. Subsequently, according to the cargo distribution type, combined with the data of temperature sensors and infrared imaging devices, the uniform temperature or non-uniform temperature is calculated by weighted calculation, and the wind speed of the cooling fan is adjusted accordingly to achieve the intelligent control of the cargo hold temperature. This method can optimize the cargo hold environment to a certain extent and ensure the safety of the cargo, but its core monitoring index is still mainly the environmental temperature, and it fails to conduct fine-grained real-time monitoring of the position change or structural state change of the cargo itself, lacking the ability to quickly capture and respond to local abnormal changes of the cargo.

[0004] CN117036816A discloses a ship cargo hold status detection method. It extracts the depth features, contour features and texture features of the cargo hold image based on deep learning, and uses a trained classifier to judge whether the cargo hold status is an empty hold or a non-empty hold. Although this method realizes the automatic recognition of the cargo hold status and reduces the labor cost, its granularity is relatively coarse, and it can only judge the occupancy status of the overall cargo hold, and cannot identify local cargo movement, fixed state change or abnormal risks in specific areas at the detail level, which limits its application effect in refined management and real-time dynamic monitoring.

[0005] Generally speaking, the existing ship cargo monitoring technologies generally have the following deficiencies: They overly rely on a single type of environmental parameter as the judgment basis and cannot accurately perceive the structural changes of the cargo itself; Image processing mostly aims at overall distribution or state classification and lacks detailed tracking of local area and local feature changes; For the changes in the cargo fixation state caused by hull inclination and sea condition changes during ship navigation, the response mechanism is not timely enough, and there is a lack of fusion processing means for visual and sensor data, resulting in insufficient monitoring accuracy and dynamic adaptability, etc. Therefore, a ship automation intelligent monitoring solution is needed. Summary of the Invention

[0006] In view of the problem of low monitoring accuracy of local abnormal changes in existing cargo, the present invention is proposed.

[0007] Therefore, the problem to be solved by the present invention is how to achieve frame-level integration and priority-based transmission of cargo area images and visual data, and significantly improve the monitoring accuracy, response speed and data transmission efficiency.

[0008] To solve the above technical problems, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a ship automation intelligent monitoring method, which includes extracting an image feature reference frame and a cargo fixation point pressure value feature set of the deck cargo area under steady-state conditions, and constructing a local structural similarity matrix SSIM0 and a pressure reference vector P0; When the real-time pressure value P of any cargo fixation point r changes and satisfies |P r -P0|≥T p At this time, the current frame image is called, and the structural similarity is calculated with the reference frame within the cargo area, and the local image blocks that satisfy the SSIM difference Δ s ≥T a Among them, T p is the change threshold, and T a is the difference threshold; For the local image blocks, a lossless compression algorithm is used to generate compression blocks, and a cargo area image frame is constructed; The hull inclination data θ is obtained in real time, and the main camera and the slave camera are dynamically selected at the main control end according to the change trend of θ, and a unified visual data frame is synthesized; The cargo area image frame and the visual data frame are frame-level integrated to generate a composite situation map frame, where each sub-region retains the source tracking identifier and is transmitted to the monitoring system in the order of priority.

[0010] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the construction of the local structure similarity matrix SSIM0 includes: when the ship is static and the goods are not displaced, using a camera to synchronously collect the initial image frames of the goods area; dividing the initial image frames into a set of local sub-blocks within the goods area according to a preset resolution, obtaining a set of dynamic sub-blocks through edge gradient analysis, and using the standard structure similarity SSIM algorithm to calculate the local structure features block by block to generate the local structure similarity matrix SSIM0.

[0011] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the extraction of the local image blocks includes: according to the fixed points of the goods, based on the division of the preset resolution, extracting the set of goods sub-blocks G in the current frame r , while maintaining the same spatial index as the set of dynamic sub-blocks; for each goods sub-block, calculating the SSIM difference Δ according to the reference frame and SSIM0 s ; if Δ s ≥T a , then extract the image block.

[0012] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the construction of the goods area image frames includes: respectively applying a lossless compression algorithm based on entropy coding to the local image blocks, compressing to generate a corresponding set of compressed blocks, and maintaining the spatial index of each block consistent with the local image blocks; extracting the set of unchanged areas that satisfy Δ s <T a , directly caching them to form a set of cached areas; according to the spatial index, performing position splicing on the set of compressed blocks and the set of cached areas to form a complete goods area image frame, where the changed areas are filled after decompressing the set of compressed blocks, and the unchanged areas are directly filled by the set of cached areas.

[0013] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the dynamic selection of the main camera and the slave camera includes: continuously collecting the current hull inclination data θ, and at the same time establishing a sliding window to cache the historical inclination sequence θ h ; performing weighted sliding mean processing on θ h to determine the main tilt direction, and dynamically selecting the main camera and the slave camera at the master control end according to the main direction.

[0014] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the dynamic selection of the main camera and the slave camera further includes: if the current inclination exceeds the self-set threshold θ0, then only the main camera outputs a complete image frame during the data extraction stage, and at the same time instructing the slave camera to only extract the perspective fragment area that does not overlap with the main camera.

[0015] As a preferred solution of the ship automation intelligent monitoring method of the present invention, the following steps are included: generating a spatial association index table according to the original acquisition information of the cargo area image frame and the unified visual data frame, establishing a mapping relationship between each local area and its source identifier, and forming a regional mapping relationship table; based on the regional mapping relationship, performing block fusion on the cargo monitoring area in the cargo area image frame and the hull environment area in the unified visual data frame, maintaining the independent identifier of each sub-block, and constructing a composite situation map frame.

[0016] In a second aspect, the present invention provides a ship automation intelligent monitoring system, which includes:

[0017] An image feature extraction module, configured to extract an image feature reference frame and a cargo fixed point pressure value feature set under steady state conditions of the deck cargo area, and construct a local structural similarity matrix SSIM0 and a pressure reference vector P0;

[0018] A pressure change detection module, configured to, when the real-time pressure value P of any cargo fixed point r changes and satisfies |P r -P0|≥T p , call the current frame image, perform structural similarity calculation with the reference frame within the cargo area, and extract the local image blocks that satisfy the SSIM difference Δ s ≥T a , where T p is a change threshold, and T a is a difference threshold;

[0019] A local compression module, configured to, for the local image blocks, generate compression blocks by using a lossless compression algorithm and construct a cargo area image frame;

[0020] An inclination data acquisition module, configured to acquire the hull inclination data θ in real time, dynamically select the main camera and the slave camera at the main control end according to the change trend of θ, and synthesize a unified visual data frame;

[0021] A map frame integration module, configured to perform frame-level integration on the cargo area image frame and the visual data frame to generate a composite situation map frame, where each sub-region maintains a source tracking identifier and is transmitted to the monitoring system in the order of priority.

[0022] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and the following steps are included: when the computer program instructions are executed by the processor, the steps of the ship automation intelligent monitoring method described in the first aspect of the present invention are implemented.

[0023] Fourthly, the present invention provides a computer-readable storage medium, on which a computer program is stored, wherein: when the computer program instructions are executed by a processor, the steps of the ship automation intelligent monitoring method as described in the first aspect of the present invention are implemented.

[0024] The beneficial effects of the present invention are as follows: By establishing a dual benchmark of the cargo area image and pressure characteristics, the present invention realizes high-precision intelligent monitoring of the cargo fixation state; by extracting the local change area in real time and performing lossless compression, the data transmission and storage burden is greatly reduced, ensuring the real-time performance and integrity of the system. By dynamically selecting the main camera and the slave camera and combining with the hull attitude change, the monitoring requirements of the cargo under different inclination conditions can be flexibly adapted to ensure that the key area is always within the high-quality visible range. Through the frame-level integration and sub-region source tracking technology, the monitoring images of the present invention have good spatial continuity and traceability, improving the accuracy and operability of the situation awareness.

[0025] In summary, the present invention effectively improves the intelligent monitoring efficiency and reliability of the ship deck cargo area in a dynamic environment, reduces the intensity of manual inspection, reduces the safety risks caused by abnormal displacement of the cargo, and provides a more intelligent and automated guarantee means for ship transportation operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0027] Figure 1 It is a flowchart of the ship automation intelligent monitoring method;

[0028] Figure 2 It is a construction process of the local structure similarity matrix of the ship automation intelligent monitoring method;

[0029] Figure 3 It is a structure diagram of the ship automation intelligent monitoring system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention in conjunction with the drawings in the specification.

[0031] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0032] Secondly, as used herein, an "embodiment" or "embodiments" refers to specific features, structures, or characteristics that may be included in at least one implementation of the present invention. The phrase "in an embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it an individual or alternative embodiment that is mutually exclusive with other embodiments.

[0033] As mentioned in the above background art, the existing ship cargo monitoring technologies generally have the following deficiencies: relying too much on a single type of environmental parameter as the judgment basis, unable to accurately perceive the structural changes of the cargo itself; image processing mostly targets overall distribution or state classification, lacking detailed tracking of local area and local feature changes; for the changes in the cargo fixation state caused by hull inclination and sea condition changes during ship navigation, the response mechanism is not timely enough, lacking means of fusing visual and sensor data, resulting in insufficient monitoring accuracy and dynamic adaptability, etc. Therefore, a ship automation intelligent monitoring solution is needed.

[0034] Figure 1 It is a flowchart of a ship automation intelligent monitoring method according to an embodiment of the present invention. As Figure 1 shown, in the ship automation intelligent monitoring method, it includes:

[0035] S1: Extract the image feature reference frame and the cargo fixation point pressure value feature set of the deck cargo area under steady-state conditions, and construct the local structural similarity matrix SSIM0 and the pressure reference vector P0.

[0036] It should be noted that in this step, the steady-state condition refers to data collection when the ship is in a stationary or non-significant motion state to ensure that the collected data is not affected by external disturbances (such as wave and wind speed changes). The setting of this steady-state condition is to ensure the stability and consistency of the image and pressure data and avoid feature distortion caused by dynamic changes. In conventional operations, without the steady-state requirement, it is easy to cause image blurring or abnormal pressure, thus affecting the accuracy of subsequent feature extraction. Therefore, in the operation of the present invention, it is required to monitor the roll, pitch, and heave amplitudes in real time through the ship attitude monitoring system. When the monitored values are all less than the preset threshold (for example, within ±0.5°), it is determined as the steady-state condition and data collection is started.

[0037] S1.1: The construction of the local structural similarity matrix SSIM0 includes the following steps:

[0038] When the ship is static and the goods are not displaced, a camera is used to synchronously collect the initial image frames of the goods area, and a pressure sensor at the fixed point of the goods synchronously records the initial pressure data set;

[0039] The initial image frames are divided into a set of local sub - blocks within the goods area according to a preset resolution, and static redundant sub - blocks with gradient magnitudes lower than the threshold T e are screened out through edge gradient analysis, and only the sub - block set with significant dynamic features, that is, the dynamic sub - block set, is retained;

[0040] Based on the dynamic sub - block set, the standard Structural Similarity (SSIM) algorithm is used to calculate the local structural features sub - block by sub - block, generating a local structural similarity matrix SSIM0, and the matrix elements correspond to the feature reference values of each sub - block within the dynamic sub - block set.

[0041] It should be noted that the overall arrangement of the local structural similarity matrix SSIM0 is kept consistent with the sub - block division of the goods area to ensure the consistency of spatial features.

[0042] Furthermore, for each sub - block obtained by division, the present invention performs an edge gradient analysis operation, specifically using the Sobel operator to calculate the gradient magnitude within the sub - block.

[0043] Moreover, as an important index for measuring the consistency of brightness, contrast, and structural information between two images, SSIM is widely used in the field of image quality evaluation. However, its conventional use is usually for the whole image. The present invention refines it to the sub - block level to accurately characterize the stability of local features. The specific operation is to take each dynamic sub - block, calculate its three components of brightness, contrast, and structure, and the three parts are combined according to the multiplication rule to obtain the final structural similarity index, and finally synthesize the SSIM value to form a local structural feature vector.

[0044] Synchronous with the extraction of image features is the acquisition of the pressure value feature set at the fixed points of the goods. Specifically, a high - precision pressure sensor array is arranged below the goods, at the lashing points, and at the force - bearing support points, and each sensor independently records the pressure value per unit time.

[0045] S1.2: The construction of the pressure reference vector P0 includes the following steps:

[0046] Outliers are removed from the initial pressure data set, and the removal criterion is the pressure values greater than or less than the mean of the initial pressure data set ± 3 times the standard deviation range. After removal, a stable pressure reference vector for the fixed points of the goods is generated, and each element of the vector corresponds to a fixed point within the goods image area, and the vector elements maintain a one - to - one correspondence with the positions of each fixed point within the image area.

[0047] In summary, the present invention can accurately extract the most representative image structure features and pressure reference features in the deck cargo area under static and stable conditions of the cargo. By constructing the local structure similarity matrix, the effective separation of the dynamic features inside the cargo area and the static background is achieved, significantly improving the sensitivity and accuracy of subsequent change detection. The synchronous establishment of the pressure reference vector enables the initial stable state of each cargo fixed point to be quantitatively recorded, providing a reliable physical reference baseline for change monitoring.

[0048] S2: When the real-time pressure value P of any cargo fixed point r changes and satisfies |P r -P0|≥T p At this time, the current frame image is called, and the structural similarity calculation is performed with the reference frame in the cargo area, and the local image blocks that satisfy the SSIM difference Δ s ≥T a are extracted, where T p is the change threshold, and T a is the difference threshold.

[0049] Preferably, the extraction of the local image blocks that satisfy the SSIM difference Δ s includes the following steps:

[0050] a. According to the cargo fixed points, based on the above-mentioned division with the same preset resolution, the set G of cargo sub-blocks in the current frame is extracted r , while keeping the same spatial index as the set of dynamic sub-blocks;

[0051] b. For each cargo sub-block, calculate the SSIM difference Δ s =SSIM r -SSIM0 based on the reference frame and SSIM0, where SSIM r is the structural similarity index between the current frame image and the reference image in the local area;

[0052] c. If Δ s ≥T a , then the image block is extracted.

[0053] This operation method can more sensitively detect the real changes caused by cargo displacement, deformation, etc. compared with the simple method based only on pixel differences, while suppressing the false changes caused by factors such as light fluctuations and small camera jitters, thus greatly improving the robustness and practical applicability of change detection.

[0054] Furthermore, to further improve the accuracy of change detection, it is necessary to perform spatial correspondence verification on the positions of these changed sub-blocks and the positions of the triggered cargo fixed points, specifically including:

[0055] d. Compare the extracted image blocks with the triggered |Pr -P0|≥T p Perform spatial correspondence on the positions of the cargo fixing points, check whether the changed blocks are concentrated near the physical change points, filter out the mis-detected blocks unrelated to the pressure change, and update them to the final local image blocks.

[0056] It should be noted that if a changed block is within the preset physical expansion range near the fixing point, it is considered a valid changed block. Among them, the physical expansion range needs to be dynamically set in combination with the actual cargo size, which is not limited in this embodiment; if the changed block is far from all triggered fixing points and there is no corresponding pressure change record, it is initially determined as a mis-detected block and needs to be excluded. This process adopts a fast nearest neighbor query algorithm based on Euclidean distance in technical implementation to reduce the computational overhead during large-scale image block screening.

[0057] Through this method of joint verification based on physics and images, it is possible to effectively eliminate the isolated false changed blocks caused by environmental interference or measurement errors, and greatly enhance the spatial consistency and physical reliability of the final change detection results.

[0058] S3: For the local image blocks, use a lossless compression algorithm to generate compressed blocks and construct an image frame of the cargo area.

[0059] Preferably, the construction of the image frame of the cargo area includes the following steps:

[0060] S3.1: Apply a lossless compression algorithm based on entropy coding to the local image blocks respectively to compress and generate a corresponding set of compressed blocks, and keep the spatial index of each block consistent with the local image block. Each local image block is independently compressed to keep the spatial index unchanged, ensuring that the corresponding area can be accurately located during subsequent image frame reconstruction.

[0061] S3.2: Extract the set of unchanged areas that satisfy Δ s <T a and directly cache them without any compression processing to form a set of cache areas. The purpose is to retain the original pixel accuracy, reduce the redundant computational overhead, and accelerate the subsequent assembly process of the image frame.

[0062] Furthermore, after each preliminary comparison between the image frame of the cargo area and the unified visual data frame, for the unchanged areas, in addition to caching the original area data, it is also necessary to perform consistency verification based on key environmental features (such as the lighting characteristics of the hull bulkhead, the change of the environmental background texture, etc.) to determine whether the unchanged area is still visually consistent with the current environmental state. If the environmental consistency detection result shows that the change amplitude exceeds the preset threshold (for example, the number of feature point drifts exceeds a certain proportion or the texture distribution change rate exceeds the standard), then even if the change amplitude of the area content is small, the area is forcibly marked as an area to be updated and re-included in the subsequent fusion processing to avoid the deviation of the overall situation map caused by cache expiration.

[0063] S3.3: According to the spatial index, splice the positions of the compressed block set and the cache area set to form a complete image frame of the cargo area, where the changed area is filled after decompressing the compressed block set, and the unchanged area is directly supplemented by the cache area set.

[0064] Specifically, the decompressed compressed blocks are used to fill the corresponding changed positions, and the unchanged area is directly pasted and supplemented. Finally, the image frame of the cargo area is completely constructed. This image frame takes into account the compact representation of the changed area and the efficient retention of the unchanged area, while ensuring visual coherence, significantly reducing the data volume.

[0065] S3.4: Through communication scheduling, the image frame of the cargo area is packaged in slices, and the slice size is dynamically selected according to the bandwidth status of the shore-based terminal and then sent. An index table is attached during the sending process for restoration at the shore-based terminal.

[0066] In summary, the present invention can significantly reduce the data redundancy of the image frame of the cargo area while ensuring the integrity of the image information. Due to the adoption of the strategy of lossless compression of the local changed area and direct caching of the unchanged area, the image frame of the cargo maintains spatial continuity and data consistency during construction, avoiding the resource consumption problem caused by full-scale compression. At the same time, the slice packaging and dynamic slice size adjustment mechanism enable the image data to more flexibly adapt to the transmission requirements under different bandwidth conditions, effectively improving the robustness of the overall system in a complex communication environment.

[0067] S4: Real-time obtain the hull inclination data θ, dynamically select the main camera and the slave camera at the master control end according to the change trend of θ, and synthesize a unified visual data frame.

[0068] S4.1: Continuously collect the current hull inclination data θ, and at the same time establish a sliding window to cache the historical inclination sequence θ h 。

[0069] It should be noted that the inclination data θ specifically refers to the attitude change angle of the hull relative to the horizontal direction, usually collected by an inertial measurement unit or a high-precision inclination sensor. To ensure the continuity and timeliness of the data, the inclination acquisition frequency should be set not less than 10Hz to promptly reflect the dynamic changes of the hull under the action of the external environment. The real-time obtained inclination data includes not only the transverse inclination and the longitudinal inclination, but also whether to introduce the yaw angle needs to be determined according to the actual application requirements. However, in the solution of the present invention, the two indicators of the transverse inclination and the longitudinal inclination are mainly concerned to ensure that the collected data is accurate and can reflect the actual hull state.

[0070] Based on the acquisition of inclination data, a historical inclination sequence is cached by establishing a sliding window for trend analysis. This caching mechanism is different from the traditional single-point sampling method, which can better reflect the trend characteristics of inclination changes and improve the accuracy and stability of dynamic master-slave camera selection.

[0071] S4.2: Perform weighted moving average processing on θ h to determine the main tilt direction, and dynamically select the main camera and slave camera on the master control end according to the main direction.

[0072] Specifically, in order to further highlight the influence of recent data and reduce the unnecessary interference of historical data on current judgment, weighted moving average processing is performed on θ h In the operation, the weighted moving average can set a weight decreasing rule. For example, the closer the inclination data is to the current time, the greater the weight, and the farther the distance, the smaller the weight. Linear decay or exponential decay models can be used, and this embodiment is not limited to a unique one.

[0073] Using the weighted moving average can effectively eliminate the interference of occasional outliers on trend determination, ensuring that the master control end makes subsequent dynamic camera selection based on more stable and accurate main tilt direction information.

[0074] S4.3: If the current inclination exceeds the self-set threshold θ0, only the complete image frame output by the main camera is retained in the data extraction stage, and at the same time, the slave camera is instructed to only extract the perspective fragment area that does not overlap with the main camera.

[0075] Specifically, the main tilt direction of the current hull tilt is determined according to the above weighted moving average, that is, to judge whether the hull tilt is mainly horizontal or vertical.

[0076] Generally, if the absolute value of the transverse inclination is greater than the absolute value of the longitudinal inclination, it is determined that the transverse tilt is the main one, otherwise the longitudinal tilt is the main one. After establishing the main direction, the master control end dynamically selects the main camera and slave camera according to the preset camera deployment strategy. The specific operations include: real-time adjustment according to the change of the main direction. For example, when the hull tilts to the starboard side, the corresponding camera on the starboard side is preferentially selected as the main camera, and at the same time, the camera on the port side or the stern is used as the slave camera for auxiliary coverage. The selection strategy here is different from the conventional fixed main camera mode, which can adapt to the change of the hull attitude and ensure the integrity and continuity of visual coverage of key areas.

[0077] Furthermore, if the detected current inclination angle exceeds the self-set threshold, a more streamlined and efficient strategy is adopted during the data extraction phase. Specifically, only the image frames completely output by the main camera are retained to ensure the integrity and continuity of the image data in the core area; for the slave camera, only the fragment area within its field of view that does not overlap with the main camera is extracted, avoiding redundant data transmission and improving the overall data processing and transmission efficiency. This mechanism can significantly reduce the load in the case of sudden large tilts, ensuring the stability and real-time performance of the visual data link when the hull attitude changes violently.

[0078] S4.4: Based on the preset spatial mapping matrix, the complete image of the main camera and the fragments extracted from the slave camera are stitched according to the spatial alignment rules to generate a unified visual data frame for subsequent transmission and processing.

[0079] Specifically, the spatial mapping matrix is usually calibrated during the system initialization phase, and the calibration content includes the positional relationship, attitude relationship between the cameras, and the overlapping area of the fields of view.

[0080] Specifically in the operation, first, the complete image of the main camera is spatially located to determine its basic reference system in the unified visual frame; subsequently, according to the spatial mapping matrix, geometric transformations such as rotation and translation are performed on the fragments extracted from the slave camera to complete the position matching in the coordinate system of the main camera. After spatial alignment, each image fragment is seamlessly stitched according to the preset rules to form a unified visual data frame.

[0081] The generation of the unified visual data frame not only ensures the spatial continuity and integrity of the visual information but also lays a solid foundation for subsequent data transmission, hull state assessment, and navigation safety monitoring. Compared with the traditional fixed acquisition and rough stitching methods, the present invention effectively improves the adaptive ability of the system to complex dynamic environments by combining real-time inclination data to dynamically adjust the visual perception strategy, significantly enhancing the real-time performance, robustness, and reliability of the overall system.

[0082] S5: The image frames of the cargo area and the visual data frames are integrated at the frame level to generate a composite situation map frame, where each sub-region retains the source tracking identifier and is transmitted to the monitoring system in the order of priority.

[0083] It should be noted that the integration of the two types of data frames aims to achieve global dynamic perception of the cargo state and the hull environment through spatial superposition and information fusion, thereby enhancing the risk monitoring ability during navigation. During the integration process, the source tracking identifiers of each sub-region must be strictly ensured, that is, each part of the image information must carry its source camera number, shooting timestamp, and geospatial position index, so as to achieve accurate positioning during subsequent data backtracking, anomaly analysis, etc.

[0084] Preferably, the generation of the composite situation map frame includes the following steps: According to the original acquisition information of the cargo area image frame and the unified visual data frame, generate a spatial association index table, establish the mapping relationship between each local area and its source identifier, and form a regional mapping relationship table; Based on the regional mapping relationship, perform block fusion on the cargo monitoring area in the cargo area image frame and the hull environment area in the unified visual data frame, keep the independent identifiers of each sub-block, and construct a composite situation map frame.

[0085] It should be noted that during the integration, it is also necessary to preferentially select the areas that pass the consistency check for composite processing, and actively trigger resampling or refusion for the cached areas that fail the consistency check to ensure that the finally generated composite situation map frame comprehensively and accurately reflects the actual hull environment conditions.

[0086] Preferably, considering that in the application scenario of the present invention, the cargo area and the hull environment area are respectively from different camera nodes, and there are local misalignments caused by factors such as shooting angle, position micro-deviation, and lens distortion during the acquisition process. Therefore, if the global affine transformation is directly used, it is easy to cause the accumulation of fusion errors, resulting in local image misalignment and ghosting, which in turn affects the accuracy and real-time performance of subsequent risk monitoring. Therefore, the present invention proposes a fusion method based on local feature point matching to achieve high-precision alignment and splicing of each sub-region, ensuring that the composite situation map frame has higher consistency and traceability. The block fusion operation of the present invention is not a simple image overlay or splicing, but based on the spatial index, the corresponding sub-blocks are aligned in position and edge transition processed in the unified visual reference system, and the resolution is adjusted according to actual needs to ensure visual consistency and overall continuity. It avoids problems such as edge breakage and visual jump in the traditional large-block superposition method. The finally formed composite situation map frame not only completely reflects the cargo state and the hull environment, but also can quickly locate the source and attributes of each piece of image through the embedded independent identifier.

[0087] Furthermore, after the construction of the composite situation map frame is completed, according to the preset priority rules, the priority of each sub-region in the composite situation map frame is evaluated respectively, including but not limited to the cargo risk area and the abnormal hull movement area, to generate a priority list. Among them, during the evaluation process, it is preferably set with multiple classification criteria. For example, the first level is the high-risk area (abnormal cargo stacking or loose binding area), the second level is the medium-risk area (the area where the local tilt of the hull intensifies), and the third level is the low-risk area (the area with normal environment), etc. Specifically, in operation, the risk level of each sub-region can be automatically determined by combining real-time sensing data and image analysis results to form a priority list.

[0088] Furthermore, in the order of the priority list, each sub-region of the composite situation map frame is packaged into a layered data packet and sent to the shore-based monitoring system through the communication module in turn for dynamic assessment of navigation risks.

[0089] Among them, in the data packaging operation, the data in the high-priority area is preferentially encapsulated into small-volume and fast-transmission data units to ensure that key risk information can reach the monitoring center first, enabling timely response and handling. The data in the medium- and low-priority areas is then transmitted in batches at a later time to reduce the occupation of communication bandwidth and improve the overall system transmission efficiency. This hierarchical packaging and transmission strategy is different from the traditional full-volume transmission mode and can flexibly adjust the data stream according to the dynamic changes of navigation risks, greatly enhancing the information accessibility and emergency response ability of the system in case of emergencies.

[0090] In summary, through a series of meticulous operations such as spatial indexing, regional mapping, priority management, and hierarchical packaging and transmission, the present invention not only realizes the deep integration of cargo status and hull environment information, but also significantly improves the timeliness of data processing and the accuracy of risk identification, providing strong support for the navigation safety of ships.

[0091] Furthermore, as Figure 3 shown, this embodiment also provides a ship automation intelligent monitoring system, including:

[0092] An image feature extraction module 100, configured to extract the image feature reference frame and the cargo fixed-point pressure value feature set in the deck cargo area under steady-state conditions, and construct a local structural similarity matrix SSIM0 and a pressure reference vector P0;

[0093] A pressure change detection module 200, configured to, when the real-time pressure value P of any cargo fixed point r changes and satisfies |P r - P0| ≥ T p , call the current frame image, perform structural similarity calculation with the reference frame within the cargo area, and extract the local image blocks that satisfy the SSIM difference Δ s ≥ T a , where T p is the change threshold, and T a is the difference threshold;

[0094] A local compression module 300, configured to, for the local image blocks, generate compressed blocks using a lossless compression algorithm and construct an image frame of the cargo area;

[0095] An inclination data acquisition module 400, configured to continuously obtain the hull inclination data θ in real time, dynamically select the main camera and the slave camera at the master control end according to the change trend of θ, and synthesize a unified visual data frame;

[0096] A frame integration module 500, configured to perform frame-level integration on the image frame of the cargo area and the visual data frame to generate a composite situation map frame, where each sub-region maintains the source tracking identifier and is transmitted to the monitoring system in the order of priority.

[0097] This embodiment also provides a computer device applicable to the situation of the ship automation intelligent monitoring method, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the ship automation intelligent monitoring method proposed in the above embodiment.

[0098] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad set on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0099] This embodiment also provides a storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the ship automation intelligent monitoring method proposed in the above embodiment.

[0100] In summary, the present invention realizes high-precision intelligent monitoring of the cargo fixation state by establishing a dual benchmark of the cargo area image and the pressure characteristics; by real-time extracting the local change area and performing lossless compression, it greatly reduces the data transmission and storage burden, and ensures the real-time performance and integrity of the system. By dynamically selecting the main camera and the slave camera and combining with the hull attitude change, it can flexibly adapt to the cargo monitoring requirements under different inclination conditions, ensuring that the key area is always within the high-quality visible range. Through the frame-level integration and sub-region source tracking technology, the monitoring image has good spatial continuity and traceability, improving the accuracy and operability of the situation awareness.

[0101] In summary, the present invention effectively improves the intelligent monitoring efficiency and reliability of the ship deck cargo area in a dynamic environment, reduces the intensity of manual inspection, reduces the safety risks caused by abnormal displacement of the cargo, and provides a more intelligent and automated guarantee means for ship transportation operations.

[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for automatic intelligent monitoring of ships, characterized in that: Including: Extract the image feature reference frame of the deck cargo area under steady-state conditions and the feature set of the cargo fixing point pressure values, and construct the local structural similarity matrix SSIM0 and the pressure reference vector P0; When the real-time pressure value P of any cargo fixing point r changes and satisfies |P r - P0| ≥ T p At this time, the current frame image is called, and the structural similarity calculation is performed with the reference frame within the cargo area, and the local image block satisfying the SSIM difference Δ s ≥ T a is extracted, where T p is the change threshold, and T a is the difference threshold; For local image blocks, use a lossless compression algorithm to generate compressed blocks and construct the image frames of the cargo area; Obtain the hull inclination data θ in real time, dynamically select the main camera and the slave camera at the master control end according to the change trend of θ, and synthesize a unified visual data frame; Perform frame-level integration on the image frames of the cargo area and the visual data frames to generate a composite situation map frame, where each sub-region retains the source tracking identifier and is transmitted to the monitoring system in the order of priority.

2. The ship automation intelligent monitoring method according to claim 1, characterized in that: The construction of the local structural similarity matrix SSIM0 includes: when the ship is static and the cargo has no displacement, use the camera to synchronously collect the initial image frames of the cargo area; divide the initial image frames into a set of local sub-blocks within the cargo area according to the preset resolution, obtain the dynamic sub-block set through edge gradient analysis, and use the standard structural similarity SSIM algorithm to calculate the local structural features block by block to generate the local structural similarity matrix SSIM0.

3. The ship automation intelligent monitoring method according to claim 2, characterized in that: The extraction of the local image patches includes: based on the fixed points of the goods and the division with a preset resolution, extracting the set G of goods sub-patches in the current frame, while maintaining the same spatial index as the set of dynamic sub-patches; for each goods sub-patch, calculating the SSIM difference Δ according to the reference frame and SSIM0. r , and keeping the spatial index consistent with the set of dynamic sub-patches; for each goods sub-patch, calculating the SSIM difference Δ according to the reference frame and SSIM0. s ; if Δ s ≥T a , then extract the image patch.

4. The ship automation intelligent monitoring method according to claim 3, characterized in that: The construction of the cargo area image frame includes: applying a lossless compression algorithm based on entropy coding to local image patches respectively, compressing to generate a corresponding set of compressed patches, and keeping the spatial index of each patch consistent with the local image patch; extracting a set of unchanged regions that satisfy Δ s <T a , directly caching them to form a set of cached regions; according to the spatial index, splicing the positions of the set of compressed patches and the set of cached regions to form a complete cargo area image frame, where the changed regions are filled after decompressing the set of compressed patches, and the unchanged regions are directly filled by the set of cached regions.

5. The ship automation intelligent monitoring method according to claim 1, characterized in that: The dynamic selection of the main camera and the slave camera includes: continuously collecting the current hull inclination data θ, and at the same time establishing a sliding window to cache the historical inclination sequence θ h ; performing weighted sliding mean processing on θ h to determine the main tilt direction, and dynamically selecting the main camera and the slave camera at the main control end according to the main direction.

6. The ship automation intelligent monitoring method according to claim 5, characterized in that: The dynamic selection of the main camera and the slave camera further includes: if the current inclination exceeds the self-set threshold θ0, only retain the complete image frames output by the main camera during the data extraction stage, and at the same time instruct the slave camera to only extract the perspective fragment area that does not overlap with the main camera.

7. The ship automation intelligent monitoring method according to claim 1, wherein: Generate a spatial association index table according to the original acquisition information of the image frames of the cargo area and the unified visual data frames, establish the mapping relationship between each local area and its source identifier, and form a regional mapping relationship table; Based on the regional mapping relationship, perform block fusion on the cargo monitoring area in the image frames of the cargo area and the hull environment area in the unified visual data frames, keep the independent identifiers of each sub-block, and construct a composite situation map frame.

8. A ship automation intelligent monitoring system, based on the ship automation intelligent monitoring method according to any one of claims 1 to 7, characterized in that: Also including: An image feature extraction module for extracting the image feature reference frame of the deck cargo area under steady-state conditions and the feature set of the cargo fixing point pressure values, and constructing the local structural similarity matrix SSIM0 and the pressure reference vector P0; A pressure change detection module, which is used to, when the real-time pressure value P of any cargo fixing point r changes and satisfies |P r - P0| ≥ T p , call the current frame image and perform structural similarity calculation with the reference frame within the cargo area, and extract the local image blocks that satisfy the SSIM difference Δ s ≥ T a , where T p is the change threshold, and T a is the difference threshold; A local compression module for using a lossless compression algorithm to generate compressed blocks for local image blocks and constructing the image frames of the cargo area; An inclination data acquisition module for obtaining the hull inclination data θ in real time, dynamically selecting the main camera and the slave camera at the master control end according to the change trend of θ, and synthesizing a unified visual data frame; A frame integration module for performing frame-level integration on the image frames of the cargo area and the visual data frames to generate a composite situation map frame, where each sub-region retains the source tracking identifier and is transmitted to the monitoring system in the order of priority.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the ship automation intelligent monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the ship automation intelligent monitoring method according to any one of claims 1 to 7.

Images