Ship cabin panoramic visual information construction method and system based on multiple cameras

Abstract

The invention discloses a ship cabin panoramic visual information construction method and system based on multiple cameras. The method comprises the following steps: S1, acquiring size and shape data of a cabin, and determining a visual field range according to use requirements; s2, determining the arrangement mode and the number of cameras according to the size and shape data of the cabin and the field of view, wherein the arrangement mode of the cameras comprises annular arrangement, latticed arrangement or arrangement according to the geometrical shape of the cabin; and S3, installing cameras, and configuring the cameras to work in a linkage manner so as to track the image change in the cabin and realize panoramic information capture. According to the invention, real-time monitoring of ship pictures and comprehensive cabin information acquisition can be realized, dangerous information can be responded more timely, and a more efficient, safer and more environment-friendly ship operation mode is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent ship operation and maintenance technology, and more specifically, to a method and system for constructing panoramic visual information of ship cabins based on multiple cameras. Background Technology

[0002] As ships continue to develop towards intelligence, the number of personnel on board is decreasing, but the demand for real-time monitoring and maintenance of the ship's safe operation status is increasing.

[0003] Traditional methods of cabin information monitoring and data collection have certain limitations. For example, cameras capture images from fixed locations, and there is no linkage between the various monitoring images, which makes it difficult to locate and track abnormal situations and reduces the efficiency of handling ship emergency situations. Summary of the Invention

[0004] In view of the shortcomings of existing technologies, this invention provides a method and system for constructing panoramic visual information of ship cabins based on multiple cameras. This invention enables real-time monitoring of ship images and comprehensive acquisition of cabin information, allowing for more timely response to hazardous information and achieving a more efficient, safer, and more environmentally friendly ship operation mode.

[0005] The technical means employed in this invention are as follows:

[0006] A method for constructing panoramic visual information of ship cabins based on multiple cameras includes the following steps:

[0007] S1. Obtain the size and shape data of the cabin, and determine the field of view based on usage requirements;

[0008] S2. Determine the arrangement and number of cameras based on the dimensions, shape data, and field of view of the cabin;

[0009] S3. Install cameras and configure them to work in tandem to track image changes within the cabin and capture panoramic information, including:

[0010] S31. The images captured by each camera are stitched and optimized to combine local images extracted from multiple cameras into a panoramic view, providing more comprehensive cabin information.

[0011] S32. Merge the video information collected by each camera to generate a panoramic video.

[0012] Furthermore, the number of cameras is calculated using the following formula:

[0013]

[0014] Where w and h represent the width and height of the camera image, respectively. This represents the tangent of half the camera's field of view, and the ceil function rounds up.

[0015] Furthermore, the cameras are configured to work in tandem, including setting each camera to trigger other cameras to track the image as soon as any one camera detects a change, thereby capturing the entire process of the change.

[0016] Furthermore, the local images extracted from multiple cameras are combined into a panoramic view, including:

[0017] Feature extraction: Perform feature extraction processing on each local image to obtain image feature points;

[0018] Feature matching: Matching feature points between different images to find corresponding feature point pairs in multiple images;

[0019] Geometric correction: Geometric correction is performed using corresponding feature point pairs in multiple images to estimate the transformation relationship between different images;

[0020] Image stitching: Based on the estimated transformation relationship, multiple partial images are aligned and merged to generate a stitched complete image.

[0021] Furthermore, combining local images extracted from multiple cameras into a single panoramic view also includes:

[0022] Image optimization: The generated stitched image is optimized and adjusted, including removing artifacts at the stitching edges and adjusting brightness and contrast.

[0023] Furthermore, the video information collected by each camera is merged, including:

[0024] Data loading: Loads multiple video files to be spliced. Each video file is usually treated as a continuous video stream, consisting of a series of image frames.

[0025] Frame-level stitching: For each video file, it is read frame by frame starting from the first frame. During the stitching process, each frame of the video is added to the final stitched video one by one.

[0026] Audio processing: Extract the audio from the original video and splice it with the corresponding time segments of the spliced ​​video;

[0027] Merge long videos: All video files are spliced ​​and merged at the frame level to obtain a long video;

[0028] Save as a new video: Saves the merged long video as a new video file.

[0029] Furthermore, the method also includes:

[0030] S4. Monitor the panoramic video in real time, detect abnormal situations in the ship's cabins through image analysis and pattern recognition technology, and trigger an alarm.

[0031] This invention also discloses a multi-camera-based panoramic visual information construction system for ship cabins, comprising:

[0032] A vision installation system is used to acquire the size and shape data of the cabin and determine the field of view according to the usage requirements; a camera placement unit is used to determine the placement method and number of cameras based on the size and shape data of the cabin and the field of view.

[0033] A visual panoramic information construction system is used to configure various cameras to work in tandem, thereby tracking image changes inside the cabin and capturing panoramic information. Specifically, it includes:

[0034] The image stitching module is used to stitch and optimize images captured by various cameras, combining local images extracted from multiple cameras into a panoramic view, providing more comprehensive cabin information.

[0035] The video merging module is used to merge video information collected by various cameras to generate a panoramic video.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] This invention fuses perspectives from multiple cameras to provide panoramic monitoring, covering all areas inside the cabin. It creates a more comprehensive and accurate panoramic image of the cabin's interior. This multi-view approach provides richer information, helping managers better understand the cabin's situation.

[0038] This invention utilizes real-time image processing technology to analyze captured images in real time, enabling functions such as real-time target recognition and anomaly detection. Real-time detection of anomalies improves ship safety, allowing ship management personnel to promptly identify and address problems.

[0039] The present invention provides a method for constructing panoramic visual information of ship cabins based on multiple cameras. This method has potential in real-time monitoring, data analysis and innovative applications. It can remotely access and manage monitoring data, and significantly improve the safety and efficiency of ship operations. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating a method for constructing panoramic visual information of a ship's cabin based on multiple cameras, as described in this embodiment.

[0042] Figure 2 This is a schematic diagram of the camera arrangement in the embodiment.

[0043] Figure 3 This is a flowchart of the image stitching process in the embodiment.

[0044] Figure 4 This is a flowchart of the video synthesis process in the embodiment.

[0045] Figure 5 This is a schematic diagram of a multi-camera-based panoramic visual information construction system for ship cabins, as described in one embodiment. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] like Figure 1 As shown, this invention provides a method for constructing panoramic visual information of ship cabins based on multiple cameras, mainly including the following steps:

[0049] S1. Obtain the size and shape data of the cabin and determine the field of view based on usage requirements.

[0050] S2. Determine the arrangement and number of cameras based on the dimensions, shape data, and field of view of the cabin, such as... Figure 2 As shown, the cameras can be arranged in a ring, a grid, or according to the geometry of the cabin.

[0051] This step involves installing cameras in suitable locations within the enclosed compartments of a ship. Due to the large size of the compartments and the numerous pieces of equipment on board, multiple cameras are necessary. The number and arrangement of cameras depend on the size and shape of the compartment and the required field of view. Common multi-camera arrangements include circular, grid, or configurations based on the geometry of the compartment. A circular arrangement involves cameras evenly distributed along a circular or elliptical path. This arrangement is suitable for scenarios requiring omnidirectional coverage. A grid arrangement involves cameras arranged in a regular grid pattern, typically evenly distributed horizontally and vertically. This arrangement is suitable for scenarios requiring coverage of large areas. A geometrical arrangement places the cameras according to the specific geometry of the compartment or scene. This arrangement usually requires custom design to maximize the fulfillment of specific shooting needs. For example, each compartment on a ship has a different size and shape, so the camera placement must be based on the compartment's geometry. For instance, a circular arrangement of cameras is typically suitable for circular or near-circular compartments. This arrangement provides omnidirectional field of view coverage and is suitable for scenarios requiring monitoring in all directions within the compartment. In a circular arrangement, cameras are evenly distributed along the edges or circumference of the cabin to ensure relatively uniform field of view coverage. These cameras can effectively monitor activities throughout the cabin and provide comprehensive security surveillance. Alternatively, a grid-like arrangement of cameras in ship cabins is suitable for larger, rectangular, or square cabins. This arrangement, by forming a grid of cameras within the cabin, achieves uniform coverage of the entire cabin area, ensuring that every area is effectively monitored. In a grid arrangement, cameras are typically evenly distributed horizontally and vertically, forming a grid structure. This method is suitable for cabins requiring detailed division and monitoring, providing a relatively uniform field of view distribution and ensuring appropriate coverage of each area. It is important to note that the specific camera arrangement also depends on the specific size and shape of the cabin, as well as the monitoring requirements.

[0052] Because the size, volume, and environment of each compartment on a ship vary, the required number of cameras is calculated using the camera's field of view and the compartment's dimensions. Assuming the compartment area is A square meters and the camera's field of view is θ degrees, the number of cameras can be calculated using the following formula:

[0053]

[0054] Where w and h represent the width and height of the camera image, respectively. This represents the tangent of half the camera's field of view, and the ceil function rounds up. This calculation of the number of cameras takes into account the camera's field of view coverage and makes the most of the camera's field of view.

[0055] S3. Install cameras and configure them to work in tandem to track image changes within the cabin and capture panoramic information, including:

[0056] S31. The images captured by each camera are stitched and optimized to combine the local images extracted by multiple cameras into a panoramic view, providing more comprehensive cabin information.

[0057] Image stitching can be described as periodically acquiring images from cameras and merging them into a panoramic image using image processing techniques. Image stitching is the process of combining partial images of multiple cabins into a single, complete image to create a wider field of view or a panoramic view. For example... Figure 3 As shown, it specifically includes:

[0058] (1) Feature extraction: For each part of the image, feature extraction algorithms (such as SIFT, SURF, ORB, etc.) are used to detect and describe feature points in the image. These feature points can be significant local structures in the image, such as corners and edges.

[0059] (2) Feature matching: Matching feature points between different images to find corresponding feature point pairs in multiple images. Common feature matching algorithms include distance threshold-based matching and nearest neighbor-based matching.

[0060] (3) Geometric correction: Geometric correction is performed using feature point pairs to estimate the transformation relationships between different images, such as translation, rotation, scaling, etc. This can be achieved by calculating transformation matrices (such as affine matrices or perspective matrices).

[0061] (4) Image stitching: Based on the estimated transformation relationship, multiple partial images are aligned and merged to generate a stitched complete image. This can be achieved by transforming the pixel position of each image and fusing the pixel values ​​according to the corresponding weights or blending methods.

[0062] (5) Image optimization: Optimize and adjust the generated stitched image, such as removing artifacts at the stitching edges, adjusting brightness and contrast, etc., to obtain better visual effects and coherence.

[0063] Image stitching requires precise feature extraction and matching, as well as accurate geometric correction and pixel fusion. For complex image stitching tasks, more advanced algorithms and techniques may be needed, such as multi-scale stitching and image fusion optimization. Image stitching plays a crucial role in constructing panoramic visual information for enclosed compartments on ships, enabling the combination of multiple local images into a single panoramic view, providing more comprehensive compartment information.

[0064] S32. Merge the video information collected by each camera to generate a panoramic video.

[0065] After the cameras are installed, a multi-camera system needs to be configured to enable them to work in tandem, track activities within the cabin, and capture panoramic information. Specifically, when one camera detects activity, it can trigger other cameras to track and capture the entire activity. Video stitching is the process of merging multiple video files into a single long video. For example... Figure 4 As shown, it specifically includes:

[0066] (1) Data loading: First, the multiple video files to be spliced ​​need to be loaded. Each video file is usually regarded as a continuous video stream, consisting of a series of frames.

[0067] (2) Frame-level stitching: For each video file, read frame by frame starting from the first frame. During the stitching process, each frame of each video is added to the final stitched video. This preserves the original video order and achieves frame-level stitching.

[0068] (3) Audio processing: If the video contains audio, it also needs to be processed. Usually, the audio is extracted from the original video and spliced ​​with the corresponding time segments of the video to be spliced.

[0069] (4) Merging into a long video: Once all video files have been spliced ​​at the frame level, they can be merged into a single long video. This can be achieved by creating a new video stream and adding all the spliced ​​frames into it.

[0070] (5) Save as a new video: Finally, save the merged video stream as a new video file, which will give you a long video containing all the spliced ​​videos.

[0071] Next, this information needs to be stored. Storing panoramic image data is a crucial component of multi-camera ship cabin systems. The merged panoramic image data is stored on accessible storage devices. Proper data storage and management will facilitate the effective use of this data for future retrieval and analysis. Furthermore, the panoramic images are monitored in real time, and image analysis and pattern recognition technologies are used to detect anomalies in the ship cabins and trigger alarms.

[0072] Furthermore, the method also includes:

[0073] S4. Monitor the panoramic video in real time, detect abnormal situations in the ship's cabins through image analysis and pattern recognition technology, and trigger an alarm.

[0074] This invention also discloses a multi-camera-based panoramic visual information construction system for ship cabins, comprising:

[0075] A vision installation system is used to acquire the size and shape data of the cabin and determine the field of view according to usage requirements; a camera placement unit is used to determine the placement method and number of cameras based on the size and shape data of the cabin and the field of view.

[0076] A visual panoramic information construction system is used to configure various cameras to work in tandem, thereby tracking image changes inside the cabin and capturing panoramic information. Specifically, it includes:

[0077] The image stitching module is used to stitch and optimize images captured by various cameras, combining local images extracted from multiple cameras into a panoramic view, providing more comprehensive cabin information.

[0078] The video merging module is used to merge video information collected by various cameras to generate a panoramic video.

[0079] The description of the multi-camera-based panoramic visual information construction system for ship cabins of the present invention is relatively simple because it corresponds to the multi-camera-based panoramic visual information construction method for ship cabins in the above embodiment. For related similarities, please refer to the description of the multi-camera-based panoramic visual information construction method for ship cabins in the above embodiment, which will not be described in detail here.

[0080] This application discloses a method and system for constructing panoramic visual information of ship cabins based on computer vision technology, applicable to the status monitoring of ship system equipment. The invention first outlines the camera installation requirements under parameters such as the volume of the ship cabin, equipment layout, and camera characteristics; then, based on visual segmentation and stitching technology of multi-camera video information, a method for constructing panoramic images and panoramic video information of the ship cabin system equipment is established; finally, the panoramic images are monitored in real time, and abnormal conditions in the ship cabins are detected through image analysis and pattern recognition technology, triggering alarms.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing panoramic visual information of ship cabins based on multiple cameras, characterized in that, Includes the following steps: S1. Obtain the size and shape data of the cabin, and determine the field of view based on usage requirements; S2. Determine the arrangement and number of cameras based on the dimensions, shape data, and field of view of the cabin; S3. Install cameras and configure them to work in tandem to track image changes within the cabin and capture panoramic information, including: S31. The images captured by each camera are stitched and optimized to combine local images extracted from multiple cameras into a panoramic view, providing more comprehensive cabin information. S32. Merge the video information collected by each camera to generate a panoramic video.

2. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 1, characterized in that, Calculate the number of cameras using the following formula: Where w and h represent the width and height of the camera image, respectively. This represents the tangent of half the camera's field of view, and the ceil function rounds up.

3. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 1, characterized in that, Configure the cameras to work in tandem, including setting each camera to trigger the other cameras to track the image as soon as any one camera detects a change, thereby capturing the entire process of the change.

4. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 1, characterized in that, Combine local images extracted from multiple cameras into a single panoramic view, including: Feature extraction: Perform feature extraction processing on each local image to obtain image feature points; Feature matching: Matching feature points between different images to find corresponding feature point pairs in multiple images; Geometric correction: Geometric correction is performed using corresponding feature point pairs in multiple images to estimate the transformation relationship between different images; Image stitching: Based on the estimated transformation relationship, multiple partial images are aligned and merged to generate a stitched complete image.

5. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 4, characterized in that, Combining local images extracted from multiple cameras into a panoramic view also includes: Image optimization: The generated stitched image is optimized and adjusted, including removing artifacts at the stitching edges and adjusting brightness and contrast.

6. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 1, characterized in that, The video information collected by each camera is merged, including: Data loading: Loads multiple video files to be spliced. Each video file is usually treated as a continuous video stream, consisting of a series of image frames. Frame-level stitching: For each video file, it is read frame by frame starting from the first frame. During the stitching process, each frame of the video is added to the final stitched video one by one. Audio processing: Extract the audio from the original video and splice it with the corresponding time segments of the spliced ​​video; Merge long videos: All video files are spliced ​​and merged at the frame level to obtain a long video; Save as a new video: Saves the merged long video as a new video file.

7. The method for constructing panoramic visual information of ship cabins based on multiple cameras according to claim 1, characterized in that, The method further includes: S4. Monitor the panoramic video in real time, detect abnormal situations in the ship's cabins through image analysis and pattern recognition technology, and trigger an alarm.

8. A system for constructing panoramic visual information of ship cabins based on multiple cameras, characterized in that, include: The visual installation system is used to acquire data on the size and shape of the compartment and determine the field of view based on usage requirements. A camera arrangement unit is used to determine the arrangement method and number of cameras based on the size, shape data and field of view of the cabin. A visual panoramic information construction system is used to configure various cameras to work in tandem, thereby tracking image changes inside the cabin and capturing panoramic information. Specifically, it includes: The image stitching module is used to stitch and optimize images captured by various cameras, combining local images extracted from multiple cameras into a panoramic view, providing more comprehensive cabin information. The video merging module is used to merge video information collected by various cameras to generate a panoramic video.

Images