Ship navigation attitude monitoring method and device, medium and product

Abstract

The invention provides a ship navigation attitude monitoring method and device, a medium and a product, and relates to the technical field of shipping safety monitoring, and the method comprises the steps: deploying a fixed monitoring camera at the shore of a navigation channel, and continuously collecting continuous frame real-time video images of a monitored water area in a non-contact manner; utilizing a preset target detection algorithm to accurately identify a ship target in the video image and position and size information of the ship target, and determining a ship motion track and a navigation direction based on time sequence change of continuous frames; by calculating the deviation angle between the actual course of the ship and the preset channel direction, early warning is automatically triggered when the deviation angle exceeds a threshold value. According to the method, a complex sensor does not need to be installed on the ship, equipment maintenance cost and navigation interference are avoided, meanwhile, the problems that manual monitoring is low in efficiency and prone to misjudgment are solved, the ship navigation attitude can be accurately monitored in real time, accidents such as stranding and collision caused by ship yawing can be effectively prevented, and channel navigation safety and management efficiency are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of shipping safety monitoring, in particular to a ship navigation posture monitoring method and device, medium and product. BACKGROUND

[0002] As an important part of China's comprehensive transportation system, inland shipping undertakes the key task of transporting bulk goods, which is of great significance to economic development. However, ship navigation safety has always been the focus of the industry, and the accuracy and timeliness of ship posture monitoring are directly related to the prevention and control of navigation risks.

[0003] Currently, there are limitations in the ship posture monitoring technology of inland shipping. Traditional manual screen monitoring relies on the naked eye observation of crew members or shore-based personnel, which is easily disturbed by weather conditions and human factors, cannot achieve quantitative measurement, and has slow response; inertial measurement units can monitor ship posture, but are costly and require modification of the ship, making them difficult to be widely applied; simple video monitoring can only record pictures and lacks intelligent analysis and early warning functions. These defects of existing technologies make it difficult for ship posture monitoring to meet the safety needs of inland shipping. They cannot accurately measure the ship posture angle and position deviation in real time, and lack the ability to predict ship behavior in complex environments; abnormal ship postures such as "horizontal" occur frequently, which not only increases the channel occupancy rate and collision risk, but also may cause grounding, and even trigger a chain of accidents in narrow channels, seriously threatening the safety of ship navigation and the safety of life and property. SUMMARY

[0004] Embodiments of the present application provide a ship navigation posture monitoring method, device, medium and product, aiming to solve the problem that existing technologies cannot measure the ship posture angle deviation in real time through non-contact continuous frame video images, making it difficult to monitor ship navigation posture abnormalities in a timely manner.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present application provides a ship navigation posture monitoring method, comprising the following steps: Obtaining continuous frame real-time video images of a monitored water area; Detecting a ship target in each frame of video image and the position and size of the ship target in the video image through a preset target detection algorithm; According to the time sequence changes of the position and size of the ship target in the continuous frame video images, obtaining the motion trajectory of the ship target, and determining the navigation direction of the ship target according to the motion trajectory; According to the navigation direction and the predetermined channel direction, calculating the heading deviation angle of the ship target.

[0006] Further, the target detection algorithm tracks the ship target by outputting a bounding box or a segmentation contour; when multiple ship targets appear, multi-target detection is performed and each target is recorded.

[0007] Further, the size is obtained by one or more of the following ways: When the ship target is not recognized by OCR and / or AIS as a ship type in the database, a minimum circumscribed rectangle of the ship target is fitted in consecutive frames to obtain the length of the long axis of the rectangle in pixels, and the median filter is used to remove outliers caused by occlusion or jitter, and the stable pixel length obtained is taken as the size of the ship target; When the ship target is recognized by OCR and / or AIS as a preset ship type in the preset database, the ship size of the corresponding ship type is obtained by querying the database; The real-time video image is obtained by a camera installed in advance on the shore of the channel, when the camera has been calibrated, the ship target is projected to the horizontal plane by using the perspective matrix to obtain the actual size of the ship target, and the actual size is dynamically updated in consecutive frames to form a real-time actual size sequence.

[0008] Further, the position and size of the ship target in consecutive frames are tracked by using a multi-target tracking method combining Kalman filtering and Hungarian algorithm or a DeepSort visual tracking algorithm to obtain the motion trajectory of the ship target in the image; when the long axis of the rectangle is detected, the direction of the long axis of the rectangle is taken as the sailing direction; when the long axis of the rectangle is not detected, the sailing direction of the ship target is determined according to the tangent direction of the motion trajectory.

[0009] Further, the predetermined channel direction is obtained by one or more of the following ways: When the channel centerline marker is detected in the video image, the channel centerline is taken as the channel direction; When the channel centerline marker is not detected, the channel direction is fitted by using the linear trend of the ship motion trajectory in consecutive frames; When the channel boundary on both sides is detected in the video image, the channel direction is determined by extracting the extension direction of the shoreline or channel marker.

[0010] Further, the heading deviation angle is obtained by one or more of the following ways: When the actual length of the ship target is known, the heading deviation angle is calculated according to the geometric relationship between the actual length of the ship target and its projection in the channel direction, and the calculation formula is as follows: cos θ = d / L, In the formula, θis a heading deviation angle; d is a projection length of the ship target in a channel direction; L is an actual length of the ship target; When the actual length of the ship target is unknown, the heading deviation angle is calculated by using a vector dot product formula, and the calculation formula is: , In the formula, Vs is a unit sailing direction vector; and Vc is a unit channel direction vector.

[0011] Further, when the heading deviation angle exceeds a preset threshold value, it is determined that the ship sailing posture is abnormal, and a warning is issued.

[0012] In a second aspect, the present application provides a ship sailing posture monitoring device, comprising a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to realize the ship sailing posture monitoring method as described above.

[0013] In a third aspect, the present application provides a computer readable storage medium, the storage medium stores at least one program, and the at least one program is executed by the processor to realize the ship sailing posture monitoring method as described above.

[0014] In a fourth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by the processor to realize the steps of the ship sailing posture monitoring method as described above.

[0015] The above technical solution has the following technical effects: By deploying a fixed monitoring camera on the channel shore, continuous frame real-time video images of the monitored water area are continuously collected in a non-contact manner; by using a preset target detection algorithm, the ship target and its position and size information in the video image are accurately identified, and the ship motion trajectory and sailing direction are determined based on the time sequence change of the continuous frames; by calculating the deviation angle of the actual heading of the ship and the predetermined channel direction, the warning is automatically triggered when the threshold value is exceeded. The present application does not need to install complex sensors on the ship, avoids the equipment maintenance cost and sailing interference, overcomes the problems of low efficiency and easy misjudgment of manual monitoring, can monitor the ship sailing posture in real time and accurately, effectively prevents the accidents such as grounding and collision caused by ship deviation, and significantly improves the channel sailing safety and management efficiency.

[0016] In further embodiments, the predetermined channel direction is obtained by multiple strategies, which significantly improves the flexibility, accuracy and environmental adaptability of channel direction determination. When there is a center line mark of the channel in the video image, the center line is directly used as the channel direction, which is suitable for scenes with clear channel markings. If no center line mark is detected, the linear trend of the ship motion trajectory of the continuous frames is fitted, and the channel direction is calculated based on the sailing rules of the passing ships, which solves the problem of direction determination in old channels or scenes with missing markings. When the boundaries of the channel on both sides are detected, the extension direction of the shoreline or channel marker can be extracted to determine the direction in complex water areas.

[0017] In further embodiments, when the actual length of the ship is known, a calculation method based on the projection geometry relationship can be used to quantify the heading deviation angle using the ship's own size information, which is suitable for scenes with known ship parameters and improves the accuracy of angle calculation. When the actual length of the ship is unknown, a vector dot product formula is used to calculate the vector relationship between the ship's sailing direction and the channel direction, avoiding the calculation obstacles caused by the lack of ship size data, and effectively solving the monitoring problem of unknown ships or temporary navigation ships. Through the two complementary heading deviation angle calculation methods, the method significantly enhances the versatility and accuracy of the monitoring system. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 FIG. 1 is a flowchart of a ship navigation posture monitoring method according to an embodiment of the present application.

[0019] Figure 2 FIG. 3 is a schematic diagram of calculating the heading deviation angle according to the geometric relationship between the actual length of the ship and its projection on the channel direction in an embodiment of the present application.

[0020] Figure 3 FIG. 4 is a structural schematic diagram of a ship navigation posture monitoring device in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To further illustrate the embodiments, the present application provides accompanying drawings. These drawings are part of the disclosure of the present application, which mainly serve to illustrate the embodiments and can be used to explain the operating principle of the embodiments in conjunction with the related description. With reference to these contents, those skilled in the art should be able to understand other possible implementations and the advantages of the present application. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0022] The present application will be further described in conjunction with the accompanying drawings and specific embodiments.

[0023] Embodiment one: Figure 1 FIG. 1 is a flowchart of a ship navigation posture monitoring method according to an embodiment of the present application.Figure 1 The method of the embodiment includes the following steps: Obtaining real-time video images of continuous frames of a monitored water area; In one specific implementation, a real-time video stream of a target channel is obtained by a monitoring camera installed on the shore. The camera continuously collects video frames and inputs image data to a subsequent processing unit. To ensure detection accuracy, the camera should be installed at an appropriate height overlooking the channel to obtain a good viewing angle and imaging clarity.

[0024] Detecting a ship target in each frame of video images and the position and size of the ship target in the video images by a preset target detection algorithm; In one specific implementation, each frame of video images transmitted by the image acquisition device is processed to automatically identify the ship target in the image. A mature target detection algorithm is used to obtain the position bounding box or segmentation contour of the ship and output the position and size information of the ship target in the image. Through continuous frame detection, the appearance position and shape orientation of the ship target can be preliminarily obtained. In one specific implementation, the target detection algorithm tracks the ship target by outputting the bounding box or segmentation contour; when multiple ship targets appear, multi-target detection is performed and each target is recorded.

[0025] In one specific implementation, the size of the ship target is obtained by one or more of the following methods: When the ship target is not recognized by OCR and / or AIS as a ship type in the database, a minimum circumscribed rotating rectangle of the ship target is fitted in continuous frames, the pixel length of the long axis of the rectangle is recorded, and the median filter is used to remove outliers caused by occlusion or jitter to obtain a stable pixel length as the ship size; When the ship target is recognized by OCR and / or AIS as a ship type in the database, the ship size of the corresponding ship type is obtained by querying the database; When the camera has been calibrated, the actual size of the ship target is obtained by projecting the ship target to the horizontal plane using the perspective matrix, and the actual size is dynamically updated in continuous frames to form a real-time actual size sequence.

[0026] According to the time sequence changes of the position and size of the ship target in the continuous frame video images, the motion trajectory of the ship target is obtained, and the sailing direction of the ship target is determined according to the motion trajectory; In one specific implementation, a multi-target tracking method using Kalman filtering combined with the Hungarian algorithm or a DeepSort visual tracking algorithm is used to track the positions and sizes of ship targets in consecutive frames to obtain the motion trajectory of the ship target in the image. Based on the ship target image information provided by the ship detection, the principal axis direction of the ship heading is extracted. Specifically, the module can perform image analysis on the detected ship target area to determine the bow-tail orientation of the ship. For example, the long side direction of the minimum circumscribed rectangle calculated using the segmentation contour of the ship target is considered as the bow direction, or the motion direction obtained by target tracking is used as a reference for the ship heading.

[0027] In one specific implementation, the principal axis direction extraction outputs the heading angle data of the ship relative to the image coordinate system.

[0028] In one specific implementation, when the rectangular long axis pixels are detected, the rectangular long axis direction is taken as the heading direction; when the rectangular long axis pixels are not detected, the heading direction of the ship target is determined according to the tangent direction of the motion trajectory.

[0029] According to the heading direction and the predetermined channel direction, the heading deviation angle is calculated.

[0030] In one specific implementation, the predetermined channel direction is obtained in one or more of the following ways: When the channel centerline marker is detected in the video image, the channel centerline is used as the channel direction; When the channel centerline marker is not detected, the channel direction is fitted using the linear trend of the ship motion trajectory in consecutive frames; When the channel boundaries on both sides are detected in the video image, the channel direction is determined by extracting the extension direction of the shoreline or channel marker. In one specific implementation, the channel direction estimation outputs the reference direction data of the channel direction relative to the image coordinate system.

[0031] In one specific implementation, the main direction of the channel is calculated according to the ship motion trajectory obtained by target tracking. Generally, a ship sailing normally will move along the channel direction, so its motion trajectory approximately reflects the trend of the channel in the image plane. Linear regression fitting is performed on the changes in the ship position within a certain time window to obtain an approximate straight line as the channel axis direction reference. If multiple ships are moving in the channel, the trajectory data of each target can also be integrated to improve the robustness of the estimation. When the boundaries on both sides of the channel can be clearly identified in the image, the extension direction of the shoreline or channel marker can also be extracted to determine the channel direction. The channel direction data output by this step is denoted as the reference axis direction vector.

[0032] In one specific implementation, the heading deviation angle is obtained in one or more of the following ways: When the actual length of the ship is known, for example, obtained by ship identification or reference database, the heading deviation angle is calculated according to the geometric relationship between the actual length of the ship and its projection in the channel direction, and the calculation formula is as follows: cos θ = d / L, In the formula, θ is the heading deviation angle; d is the projection length of the ship in the channel direction; L is the actual length of the ship; In a specific implementation, Figure 2 is a schematic diagram for calculating the heading deviation angle according to the geometric relationship between the actual length of the ship and its projection in the channel direction in an embodiment of the present application, in which a red cuboid represents the ship, and the length of the ship is L; A and B represent the head and tail positions of the ship on the water surface, respectively, and AB is the actual length of the ship L; the dashed arrow line represents the channel direction horizontal line, and the vertical projection horizontal length of the ship in the direction is d; the ship heading and the channel direction form an angle θ, and according to the geometric relationship, cos θ = d / L.

[0033] When the actual length of the ship is unknown, the heading deviation angle is calculated by using the vector dot product formula, and the calculation formula is: , In the formula, Vs is a unit sailing direction vector; and Vc is a unit channel direction vector.

[0034] In a specific implementation, the calculation module outputs the obtained heading deviation for alarm judgment; when the heading deviation angle exceeds a preset threshold, it is determined that the ship sailing posture is abnormal and a warning is issued.

[0035] The technical scheme of the embodiment of the present application can realize automatic analysis of the ship heading deviation degree in the video monitoring picture. In the specific implementation process of the embodiment of the present application, the algorithms of each module can be optimized according to needs, for example, a more advanced neural network is applied to improve the accuracy of ship detection, a light flow method is introduced to improve the robustness of target tracking in the case of occlusion, or the channel direction is corrected in combination with a GIS waterway electronic map. These deformation schemes are within the protection scope of the inventive concept.

[0036] Embodiment two: Figure 3 is a structural schematic diagram of the ship sailing posture monitoring device in an embodiment of the present application, like Figure 3As shown, the apparatus includes a processor 301, a memory 302, a bus 303, and a computer program stored in the memory 302 and executable on the processor 301, the processor 301 includes one or more processing cores, the memory 302 is connected to the processor 301 through the bus 303, the memory 302 is used for storing program instructions, and the processor implements the steps in the above method embodiments of the embodiment one of the application when executing the computer program.

[0037] Further, as an executable solution, the ship navigation posture monitoring device can be a computer unit, which can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The computer unit can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned constituent structure of the computer unit is only an example of the computer unit, and does not constitute a limitation on the computer unit, and can include more or fewer components than the above, or combine certain components, or different components. For example, the computer unit can also include an input / output device, a network access device, a bus, and the like, and the embodiments of the application do not limit this.

[0038] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like. The processor is the control center of the computer unit, and connects all parts of the computer unit through various interfaces and lines.

[0039] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer unit by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the mobile phone and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0040] Embodiment three: The application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize the steps of the method in the above-mentioned embodiments of the application.

[0041] The modules / units integrated by the computer unit, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. The computer program is executed by a processor to realize the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, an executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM) and a software distribution medium, etc. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction.

[0042] Embodiment four: The application further provides a computer program product, which includes a computer program. The computer program is executed by a processor to realize the steps of the method for monitoring the sailing posture of a ship.

[0043] Although the present application has been particularly shown and described with respect to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details can be made therein without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A method for monitoring a ship's navigation posture, characterized in that: The following steps are involved: Obtain continuous frame real-time video images of the monitored water area; Detecting the ship target in each frame of video image and the position and size of the ship target in the video image by using a preset target detection algorithm; Obtaining a motion trajectory of the ship target based on temporal changes in the position and size of the ship target in consecutive frames of video images, and determining the navigation direction of the ship target based on the motion trajectory; The course deviation angle of the ship target is calculated according to the sailing direction and the predetermined channel direction.

2. A method for monitoring a ship's navigation posture according to claim 1, characterized in that: The target detection algorithm tracks the ship target by outputting a bounding box or a segmentation outline; wherein, when multiple ship targets appear, multi-target detection is performed and recorded separately.

3. The method for monitoring the navigation posture of a ship according to claim 1, characterized in that: Obtain the size using one or more of the following methods: When the OCR and / or AIS fail to identify the ship target as a ship type in the database, the minimum circumscribed rotated rectangle of the ship target is fitted in consecutive frames to obtain the pixel length of the rectangle's major axis, and outliers caused by occlusion or jitter are removed using a median filter. The obtained steady-state pixel length is used as the size of the ship target; When the OCR and / or AIS is used to identify the ship target as a preset ship type in the preset database, the ship size of the corresponding ship type is obtained by querying the database; The real-time video image is acquired by a camera pre-installed on the shore of the waterway. When the camera has been calibrated, the ship target is projected onto the horizontal plane using a perspective matrix to obtain the actual size of the ship target, and the actual size is dynamically updated in consecutive frames to form a real-time actual size sequence.

4. A method for monitoring a ship's navigation posture according to claim 3, characterized in that: The position and size of the ship target in consecutive frames are tracked using a multi-target tracking method combining Kalman filtering with the Hungarian algorithm or a DeepSort visual tracking algorithm to obtain the motion trajectory of the ship target in the image. When a rectangular long-axis pixel is detected, the direction of the rectangular long axis is used as the navigation direction. When no rectangular long-axis pixel is detected, the navigation direction of the ship target is determined based on the tangent direction of the motion trajectory.

5. The method for monitoring the navigation posture of a ship according to claim 1, characterized in that: The predetermined course direction is obtained by one or more of the following methods: When a channel centerline mark is detected in the video image, the channel centerline is used as the channel direction; When there is no channel centerline mark, the channel direction is fitted using the linear direction of the ship's motion trajectory in continuous frames; When the video image detects the boundaries on both sides of the channel, the channel direction is determined by extracting the extension direction of the coastline or channel markers.

6. The method for monitoring the navigation posture of a ship according to claim 1, characterized in that: The heading deviation angle is obtained by one or more of the following methods: When the actual length of the ship target is known, the heading deviation angle is calculated according to the geometric relationship between the actual length of the ship target and its projection in the channel direction. The calculation formula is as follows: cos θ = d / L, Where, θ is the heading deviation angle; d is the projected length of the ship target in the channel direction; L is the actual length of the ship target; When the actual length of the ship target is unknown, the heading deviation angle is calculated using the vector dot product formula, which is: ， Where Vs is the unit sailing direction vector; Vc is the unit channel direction vector.

7. The method for monitoring the navigation posture of a ship according to claim 1, characterized in that: When the heading deviation angle exceeds a preset threshold, it is determined that the ship's navigation posture is abnormal and an early warning is issued.

8. A device for monitoring a ship's navigation posture, characterized in that: The method comprises a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the method for monitoring the navigation posture of a ship as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one program, and the at least one program is executed by a processor to implement the method for monitoring the navigation posture of a ship as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for monitoring the navigation posture of a ship as described in any one of claims 1 to 7 are implemented.

Images