A Visual Detection and Early Warning Method and System for Shore Power Cable Disconnection of Ships

Abstract

The present invention proposes a shore power ship cable break visual detection and early warning method and system, comprising: collecting sample images containing cables; calculating the coordinate values in the camera coordinate system based on the coordinates in the pixel coordinate system of the collected sample images and the camera internal parameters, and obtaining point cloud data in the camera coordinate system; converting the point cloud data in the camera coordinate system into point cloud data in the world coordinate system; based on the point cloud data in the world coordinate system, extracting point cloud data that meets the characteristics of the cable, and intercepting part as reference template data; comparing the point cloud data of the cable collected in real time with the reference template data, and issuing an alarm when the error exceeds the allowable range.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a shore power ship cable break visual detection and early warning method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] After the ship docks at the port, when loading and unloading containers, it takes a long time and cannot be monitored by personnel in real time. When encountering bad weather, the ship's large wind-exposed area is prone to strong winds causing the moorings to stretch suddenly and break, or the ship may be violently swayed up and down and back and forth due to the influence of waves, causing the bow and stern moorings to wear and break frequently. In severe cases, a chain reaction will occur, and the moorings will break one after another, causing the ship to drift away from the berth, resulting in collisions with the dock, facilities in the waterway or other ships, seriously affecting the safety of water shipping.

[0004] At present, in order to ensure the safety of ship docking, manual inspection and fixed camera monitoring are generally used. Both methods require human participation and judgment. Due to the subjective factors of the personnel themselves, it is easy to cause missed inspections or untimely detection.

[0005] Existing technical solutions can also improve the safety of system operation by adding fixed camera monitoring points, but this will greatly increase the cost of the equipment; or by delaying staff inspection time to ensure the safety of equipment use, which requires more manpower costs. Summary of the Invention

[0006] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a shore power ship cable break visual detection and early warning method, which uses visual detection means to intelligently monitor the cable status in real time and provide timely early warning.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, a shore power ship cable break visual detection and early warning method is disclosed, comprising:

[0009] Acquire a sample image containing the cable;

[0010] According to the coordinates of the pixel coordinate system of the collected sample image and the camera internal parameters, the coordinate values in the camera coordinate system are calculated to obtain the point cloud data in the camera coordinate system;

[0011] Convert the point cloud data in the camera coordinate system to the point cloud data in the world coordinate system;

[0012] Based on the point cloud data in the world coordinate system, extract the point cloud data that meets the characteristics of the cable, and cut off part of it as the reference template data;

[0013] The point cloud data of the cable collected in real time is compared with the reference template data, and an alarm is issued when the error exceeds the allowable range.

[0014] The present invention can realize real-time monitoring and analysis of cable status in complex scenarios through feature analysis based on three-dimensional point clouds, which not only reduces the workload of personnel, but also improves the early warning capability of on-site emergencies.

[0015] A further technical solution is to calibrate the mask area of the image detection area for the collected sample image containing the cable.

[0016] In the second aspect, a shore power ship cable break visual detection and warning system is disclosed, comprising:

[0017] An image acquisition module is configured to: acquire a sample image containing the cable;

[0018] The coordinate conversion module is configured to: calculate the coordinate value in the camera coordinate system according to the coordinates in the pixel coordinate system of the collected sample image and the camera intrinsic parameters, and obtain the point cloud data in the camera coordinate system;

[0019] Convert the point cloud data in the camera coordinate system to the point cloud data in the world coordinate system;

[0020] The feature extraction module is configured to: extract point cloud data that meets the cable characteristics based on the point cloud data in the world coordinate system, and intercept a portion as reference template data;

[0021] The recognition module is configured to compare the point cloud data of the cable collected in real time with the reference template data, and issue an alarm when the error exceeds the allowable range.

[0022] One or more of the above technical solutions have the following beneficial effects:

[0023] The present invention combines visual algorithms with depth camera point cloud data analysis to achieve real-time intelligent analysis of the status of ship cables. When the cable status is risky, early warning information can be issued in a timely manner, thereby effectively improving the port safety level and reducing manual labor intensity and operating costs.

[0024] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0026] Figure 1 This is a schematic diagram of camera placement according to an embodiment of the present invention;

[0027] Figure 2 Flowchart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0030] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0031] Example 1

[0032] The mooring detection object of the present invention is the ship's fixed cable, and the local cable point cloud data is collected by a fixed depth camera. The cable is identified by three-dimensional geometric feature recognition. Specifically, this embodiment discloses a shore power ship cable break visual detection and early warning method, including:

[0033] A camera is placed behind the bollard according to the field of view requirements. The camera is mounted on a fixed bracket and the cable data is collected and analyzed in real time through the microcomputer and depth camera on the equipment side. Figure 1 For the specific process, please refer to the attached Figure 2 As shown;

[0034] Data collection and camera calibration: The depth camera is used to collect sample images containing cables and calibrate the mask area of the image detection area. The main purpose is to select the detection area range in the current scene.

[0035] The camera depth image transforms the point cloud data in the camera coordinate system. According to the x, y coordinates in the pixel coordinate system provided by the RGB image (i.e. u, v in the formula) and the camera intrinsic parameters, the X, Y coordinate values in the camera coordinate system can be calculated. At the same time, the depth map directly provides the Z coordinate value in the camera coordinate system, and then the coordinates in the camera coordinate system are obtained. The coordinates of the obstacle points in the camera coordinate system are the point cloud sensor data, that is, the point cloud data in the camera coordinate system. The point cloud data can be used to calculate the three-dimensional coordinate position of the cable.

[0036] The above-mentioned depth map stores an image with three-dimensional coordinate information. In order to obtain the corresponding three-dimensional coordinates of the corresponding points on the RGB image, it is necessary to obtain them through the coordinate relationship formula between the RGB image and the depth map.

[0037] The relationship formula between the coordinates of the camera coordinate system P and the coordinates of the point in the pixel coordinate system Puv is:

[0038]

[0039] The specific solution formula after sorting out the above formula is as follows:

[0040] X=Z(uc x ) / f x

[0041] Y=Z(vc y ) / f y

[0042] Z=d

[0043] The Z value of the point in the camera coordinate system is the depth value d measured by the camera, which is the distance from the real point to the camera plane. If it is not, it can be multiplied to compensate. Adding color information to the corresponding point positions creates a color point cloud. The color point cloud forms a 3D image with color texture, providing a three-dimensional visualization of the data.

[0044] The point cloud image in the camera coordinate system is converted into point cloud data in the world coordinate system, which is used to obtain the conversion relationship between the world coordinate system of the cable and the image coordinate system, so as to know the spatial information of the cable under the camera.

[0045] According to the coordinate conversion formula from the world coordinate system to the pixel coordinate system:

[0046]

[0047]

[0048] The above describes the coordinate relationship between the world coordinate system Pw and the pixel coordinate system Puv. Therefore, using the above formula, after inputting the pixel coordinates [u, v] and the depth Z, we can calculate the coordinates of the point in the world coordinate system Pw. The coordinates of all obstacle points in the world coordinate system are the point cloud map data.

[0049] The depth map can directly provide the Z coordinate value of a point in the camera coordinate system, which is used as the Z value in the formula. The RGB map provides the coordinates of the point p = [u, v] in the pixel coordinate system.

[0050] According to the internal parameter formula, the three-dimensional coordinates P = [X, Y, Z] of the point in the camera coordinate system can be calculated. Then, according to the homogeneous transformation matrix T or the rotation matrix and translation vector R, t of the camera, the three-dimensional coordinates P of the point in the world coordinate system can be obtained. w =[X w , Y w, Z w ]. w It is a point cloud calibrated in the world coordinate system, also known as a point cloud map.

[0051] The present invention extracts point cloud data that matches the cable's characteristics from the point cloud data. Based on this extracted cable's characteristics, the system calculates its orientation and posture. This intercepted portion serves as a reference template, and subsequent point cloud data collected in real time is compared against this data. If the error exceeds the allowable range, the system issues an alarm.

[0052] To address the problem that when a ship swings greatly due to strong winds or surges, the fixed cable generates a large tension between the mooring post and the ship, which may cause damage or breakage of the cable in extreme cases. The present invention uses visual detection means to intelligently monitor the cable status in real time and provide timely warnings of possible dangers.

[0053] The present invention first uses a fixed depth camera to capture background 3D point cloud data when the cable is not attached. This data is then segmented from the 3D point cloud data of the current cable, resulting in 3D point cloud data containing the cable and a small amount of interfering data. The 3D point cloud data of the cable is then segmented using the cable's 3D geometric features as a template. The collected 3D data of the cable segment can be used to calculate the physical position, orientation, and curvature of the current cable segment as posture data. Compared to deep learning methods, the current visual algorithm avoids the extensive sample collection and labeling required. For different cable types, stable recognition rates can be achieved by simply adjusting the corresponding parameters.

[0054] The present invention performs feature processing on the point cloud data acquired by a 3D camera, removes interfering background, removes noise interference through high-pass filtering of the point cloud, establishes a 3D point cloud template sample, analyzes the difference between the 3D data of the cable to be tested and the template data, performs data fitting, and judges whether the cable is abnormal based on the data fitting results.

[0055] Example 2

[0056] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0057] Example 3

[0058] The purpose of this embodiment is to provide a computer-readable storage medium.

[0059] A computer-readable storage medium stores a computer program, which, when executed by a processor, performs the steps of the above method.

[0060] Example 4

[0061] The purpose of this embodiment is to provide a shore power ship cable break visual detection and warning system, including:

[0062] An image acquisition module is configured to: acquire a sample image containing the cable;

[0063] The coordinate conversion module is configured to: calculate the coordinate value in the camera coordinate system according to the coordinates in the pixel coordinate system of the collected sample image and the camera intrinsic parameters, and obtain the point cloud data in the camera coordinate system;

[0064] Convert the point cloud data in the camera coordinate system to the point cloud data in the world coordinate system;

[0065] The feature extraction module is configured to: extract point cloud data that meets the cable characteristics based on the point cloud data in the world coordinate system, and intercept a portion as reference template data;

[0066] The recognition module is configured to compare the point cloud data of the cable collected in real time with the reference template data, and issue an alarm when the error exceeds the allowable range.

[0067] Specifically, in the feature extraction module, point cloud data that meets the cable characteristics is extracted, and a portion is intercepted as reference template data, specifically:

[0068] The background data is segmented between the 3D point cloud data of the untied cable and the 3D point cloud data of the fixed cable, and the 3D point cloud data containing the cable and a small amount of interference data is obtained. Then, the 3D point cloud data of the cable is segmented according to the 3D geometric features of the cable as the reference template data.

[0069] Specifically, in the recognition module, the point cloud data of the cable collected in real time is compared with the reference template data, the difference between the three-dimensional data of the cable to be detected and the template data is analyzed, and data fitting is performed. Based on the data fitting results, it is determined whether the cable is abnormal.

[0070] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0071] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0072] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A shore power ship cable break visual detection and early warning method, characterized by: include: Collect sample images, specifically including: background 3D point cloud data and 3D point cloud data of the fixed cable when the cable is not tied; The coordinate values in the camera coordinate system are calculated based on the pixel coordinate system of the collected sample image and the camera internal parameters, and the point cloud data in the camera coordinate system is obtained; Convert the point cloud data in the camera coordinate system to the point cloud data in the world coordinate system; Specifically: Camera coordinate system P With pixel coordinate system Puv The relationship formula of the coordinates of the points below: The specific solution formula after sorting out the above formula is as follows: The midpoint in the camera coordinate system Z The value is the depth value measured by the camera d , that is, the distance from the real point to the camera plane; if it is not, it is doubled for compensation; color information is added to the position of the corresponding point to form a color point cloud, and the color point cloud forms a three-dimensional image with color texture, providing a three-dimensional visualization display of data; According to the coordinate conversion formula from the world coordinate system to the pixel coordinate system: Using the above formula, input pixel coordinates [ u ,v ] and depth Z After that, the coordinates of the point in the world coordinate system are obtained Pw , the coordinates of all obstacle points in the world coordinate system are point cloud map data; Based on the point cloud data in the world coordinate system, extract the point cloud data that meets the characteristics of the cable, and intercept part of it as the reference template data, specifically: The background 3D point cloud data of the untied cable is segmented from the 3D point cloud data of the currently fixed cable to obtain 3D point cloud data containing the cable and a small amount of interference data. The 3D point cloud data of the cable is then segmented based on the 3D geometric features of the cable as reference template data. The point cloud data of the cable collected in real time is compared with the reference template data, and an alarm is issued when the error exceeds the allowable range.

2. The shore power ship cable break visual detection and early warning method according to claim 1, characterized in that: For the collected sample image, the mask area of the image detection area is calibrated.

3. The shore power ship cable break visual detection and early warning method according to claim 1, characterized in that: The physical position, direction and curvature of the current cable segment can be calculated as posture data by collecting the three-dimensional point cloud data of the fixed cable.

4. The shore power ship cable break visual detection and early warning method according to claim 1, characterized in that: When acquiring sample images containing cables, the camera was positioned behind the bollard.

5. The shore power ship cable break visual detection and early warning method according to claim 1, characterized in that: The point cloud data of the cable collected in real time is compared with the reference template data, the difference between the three-dimensional data of the cable to be tested and the template data is analyzed, and data fitting is performed. Based on the data fitting results, it is determined whether the cable is abnormal.

6. A shore power ship cable break visual detection and warning system, characterized by: include: An image acquisition module is configured to: acquire a sample image containing the cable; The coordinate conversion module is configured to calculate the coordinate value in the camera coordinate system according to the coordinates in the pixel coordinate system of the collected sample image and the camera intrinsic parameters, thereby obtaining the point cloud data in the camera coordinate system; Convert the point cloud data in the camera coordinate system to the point cloud data in the world coordinate system; Specifically: Camera coordinate system P With pixel coordinate system Puv The relationship formula of the coordinates of the points below: The specific solution formula after sorting out the above formula is as follows: The midpoint in the camera coordinate system Z The value is the depth value measured by the camera d , that is, the distance from the real point to the camera plane; if it is not, it is doubled for compensation; color information is added to the position of the corresponding point to form a color point cloud, and the color point cloud forms a three-dimensional image with color texture, providing a three-dimensional visualization display of data; According to the coordinate conversion formula from the world coordinate system to the pixel coordinate system: Using the above formula, input pixel coordinates [ u ,v ] and depth Z After that, the coordinates of the point in the world coordinate system are obtained Pw , the coordinates of all obstacle points in the world coordinate system are point cloud map data; The feature extraction module is configured to extract the point cloud data that meets the cable characteristics based on the point cloud data in the world coordinate system, and intercept a portion as the reference template data, specifically: The background 3D point cloud data of the untied cable is segmented from the 3D point cloud data of the currently fixed cable to obtain 3D point cloud data containing the cable and a small amount of interference data. The 3D point cloud data of the cable is then segmented based on the 3D geometric features of the cable as reference template data. The recognition module is configured to compare the point cloud data of the fixed cable collected in real time with the reference template data, and issue an alarm when the error exceeds the allowable range.

7. The shore power ship cable break visual detection and warning system according to claim 6, characterized in that: In the recognition module, the point cloud data of the cable collected in real time is compared with the reference template data, the difference between the three-dimensional data of the cable to be detected and the template data is analyzed, and data fitting is performed. Based on the data fitting results, it is determined whether the cable is abnormal.

8. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method described in any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 5 are executed.

Images