An automatic monitoring and early warning method for bridge water area ships

Abstract

The application provides an automatic monitoring and early warning method for a bridge water area ship, and the method comprises the following steps: collecting ship image data of the bridge water area, and obtaining ship contour data and ship position data; obtaining preprocessed AIS ship position data to obtain target ship position parameter data; performing camera assumed world coordinate system conversion, and matching the ship position data converted to the camera assumed world coordinate system and the target ship position parameter data; obtaining dynamic and static information of each ship, constructing a three-dimensional safety distance model of the bridge water area, calculating a longitudinal safety distance and a transverse safety distance, and when the ship has an overheight condition and a ship-bridge collision dangerous behavior, highlighting the dangerous ship and outputting early warning information. The application realizes accurate monitoring of ship parameters by fusing and displaying the ship image data and the AIS data, outputs early warning information when the ship has an overheight condition and a collision danger, and improves the safety of the bridge water area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship monitoring, in particular to an automatic monitoring and early warning method for ships in bridge water area. BACKGROUND

[0002] Bridge water area is an important intersection area for transportation on both sides of the river, with complex environment and frequent accidents. With the development of water transportation, bridge collision incidents occur repeatedly, causing great losses to the traffic department in charge. The safety of the bridge foundation is one of the main themes of the bridge. At present, the maintenance and management method of urban bridges mainly arranges specific staff for daily inspection and repair. This not only has high cost of manpower and material resources, but also has insufficient efficiency, cannot be supervised for a long time, and is easy to have loopholes.

[0003] The ship monitoring and early warning method currently applied to bridge water area mainly includes AIS system (Automatic Identification System) and video monitoring system. Through the AIS system, the dynamic information such as ship position, heading, speed of the ship can be obtained, and the static information such as ship name, ship size and ship type can also be obtained. Using these information can monitor the ships in bridge water area, but the AIS system information broadcast has a time delay, and does not have the continuity and intuitiveness of the video monitoring system. The video monitoring system has functions such as video data acquisition and playback, and can monitor the key water area through video, has the advantages of intuitiveness and continuity, but cannot obtain the dynamic and static information of the ship, and needs to rely on manual monitoring and identification of dangerous behaviors of the ship. Therefore, how to accurately monitor the ships in bridge water area to eliminate the safety hazards of the ships in bridge water area has become a problem that the workers in the technical field need to solve urgently. SUMMARY

[0004] Therefore, it is necessary to provide an automatic monitoring and early warning method for ships in bridge water area to accurately monitor the ships in bridge water area and eliminate the safety hazards of the ships in bridge water area.

[0005] In order to achieve the above purpose, the present application provides an automatic monitoring and early warning method for ships in bridge water area, comprising:

[0006] Collecting ship image data of bridge water area, and establishing a ship target monitoring model based on the ship image data;

[0007] Obtaining ship contour data and ship position data by using the ship target monitoring model;

[0008] Obtaining AIS ship position data, and preprocessing the AIS ship position data to obtain target ship position parameter data;

[0009] The ship position data and the target ship position parameter data are converted to a camera assumed world coordinate system, distance and direction matching is performed on the ship position data and the target ship position parameter data converted to the camera assumed world coordinate system, and data information of successful matching is stored in a fusion data set;

[0010] According to the fusion data set, each ship dynamic and static information is obtained, a three-dimensional safety distance model of a bridge area water area is constructed, and a longitudinal safety distance and a transverse safety distance are calculated;

[0011] When a ship exists in an overheight situation and a ship-bridge collision dangerous behavior, the dangerous ship is highlighted and warning information is output in combination with the three-dimensional safety distance model, the ship target contour data, the ship position data and the target ship position parameter data.

[0012] In a possible implementation, the ship image data of the bridge area water area is collected, and a ship target monitoring model is established based on the ship image data, including:

[0013] The ship image data of the bridge area water area under different environmental conditions is collected by using a camera assembly;

[0014] The ship target monitoring model is constructed based on the improved YOLOv5 target detection algorithm.

[0015] The loss function of the improved YOLOv5 target detection algorithm is EIoU Loss, and the non-maximum suppression algorithm is Soft-NMS.

[0016] In a possible implementation, the ship target monitoring model is used to obtain ship contour data and ship position data, including:

[0017] The ship image data of the bridge area water area is detected by using the ship target detection model to obtain initial ship contour data and ship position data;

[0018] The ship contour data is obtained based on a Canny algorithm.

[0019] The ship contour data includes ship edge contour data and ship height data, and the ship position data includes coordinates of a center point of a ship bottom edge contour in a pixel coordinate system.

[0020] In a possible implementation, the AIS ship position data is obtained, and the AIS ship position data is preprocessed, including:

[0021] The AIS ship position data within a preset period of an image timestamp is collected, and the AIS ship position data is preprocessed;

[0022] The initial ship position parameter data is calculated according to the preprocessed AIS ship position data.

[0023] Screening the ship position parameter data outside the monitoring range of the bridge area channel to obtain target ship position parameter data;

[0024] The ship target contour data and the target ship position parameter data are aligned.

[0025] In a possible implementation, the ship position data and the target ship position parameter data are converted to a camera assumed world coordinate system, the ship position data and the target ship position parameter data converted to the camera assumed world coordinate system are matched in distance and direction, and the matched data information is stored in a fusion data set, including:

[0026] Obtaining the calibrated camera assembly parameter data;

[0027] The ship position data in the pixel coordinates are converted to the camera assumed world coordinate system, and the first position data of the target ship in the ship position data is calculated for the camera assembly;

[0028] The target ship position parameter data are converted to the camera assumed world coordinate system, and the second position data of the ship in the target ship position parameter data is calculated for the camera assembly;

[0029] The first position data and the second position data are fused to obtain fusion data, and the fusion data is stored in the fusion data set.

[0030] In a possible implementation, the camera assembly parameter data include an elevation angle, a focal length, a zero azimuth angle, a geographic position coordinate, an internal parameter, and an external parameter, the first position data include a first distance, a first azimuth, and a height of the target ship, and a calculation formula of the first position data is:

[0031]

[0032]

[0033]

[0034] H SP =y cq -y cp ;

[0035]

[0036] wherein d v represents the first distance, β v represents the first azimuth, γ represents the zero azimuth angle of the camera assembly, α p represents the elevation angle of the camera assembly, H SP represents the height of the target ship projection, H S represents the height of the target ship, (x w , y w) represents the coordinates of the target ship in the camera assumed world coordinate system, and a represents the H S angle corresponding to the side where the target ship is located, and a sp represents the H SP angle corresponding to the side where the target ship is located, and y cq represents the vertical coordinate of the top pixel information of the target ship profile converted to the world coordinate system, and y cp represents the vertical coordinate of the bottom pixel information of the target ship profile converted to the world coordinate system, and v p represents the top pixel information of the target ship image profile, I represents the image pixel height.

[0037] In a possible implementation, the target ship position parameter data includes AIS ship position data, the second position data includes a second distance and a second azimuth, and the calculation formula of the calculation of the camera assembly and the second position data of the ship in the target ship position parameter data is:

[0038]

[0039]

[0040]

[0041]

[0042]

[0043] wherein (x wa , y wa , z wa ) represents the AIS ship position data, T a represents the AIS information timestamp, represents the geographic position coordinates of the camera assembly, d a represents the second distance, and β a represents the second azimuth, R represents the radius of the earth, and (X, Y, 0) represents the ship coordinates of the AIS target ship in the camera assembly assumed world coordinate system with the north direction as the zero azimuth.

[0044] In a possible implementation, the fusion of the first position data and the second position data includes:

[0045] setting the first position data and the second position data with the difference within the preset range as the associated data;

[0046] determining the target number of AIS ships in the associated data;

[0047] When the target number of the AIS ship is greater than 1, then the error average of the first position data and the second position data is calculated, and the first position data and the second position data with the minimum error average are screened out for fusion;

[0048] The calculation formula of the fusion of the first position data and the second position data is:

[0049]

[0050] Wherein, k o represents the distance weight coefficient, k d represents the azimuth weight coefficient, R o represents the distance correlation degree, R d represents the azimuth correlation degree.

[0051] In a possible implementation, the calculation formula of the longitudinal safety distance and the transverse safety distance is:

[0052] S T = S H × 0.6 (B S + L sin β) + B L ;

[0053] S L = D1+D2+D3+D4+L S +L B ;

[0054]

[0055]

[0056]

[0057] Wherein, S T represents the transverse safety distance, S L represents the longitudinal safety distance, S H represents the vertical safety distance, B S represents the width of the ship, β represents the drift angle of the ship, B L represents the width of the bridge pier, D1 represents the ship's reversing brake deceleration distance, D2 represents the distance of the ship's towing anchor, D3 represents the wind flow increment, D4 represents the safety margin, L S represents the length of the ship, L B represents the length of the bridge pier, v0 represents the speed of the ship starting to reverse the brake, v represents the speed of the ship after ending the reverse brake, m represents the mass of the ship, P represents the power of the ship, k1 represents the wind load coefficient, k1 represents the flow load coefficient, v w represents the wind speed, v w represents the flow velocity, Δ represents the displacement of the ship, V represents the speed of the ship when throwing the anchor, λa Cp represents the grip coefficient of the anchor, W a K represents the uneven wind pressure reduction coefficient, B w B represents the wind area above the waterline of the ship, V d B represents the wind area below the waterline of the ship, V s V represents the speed of the ship, V w V represents the speed of the wind, T w T represents the time of the wind acting on the ship, V f V represents the speed of the flow, T f T represents the time of the flow acting on the ship, V

[0058] In one possible implementation, the formula for judging whether the ship has the risk of super-elevation and bridge collision is:

[0059]

[0060]

[0061] I H represents the super-elevation risk monitoring index, I C represents the collision risk monitoring index, A represents the ship position coordinates at the camera timestamp t, A v represents the left upstream pier dangerous area, A a represents the right upstream pier dangerous area, H B H represents the bridge clearance height, H S represents the ship clearance height.

[0062] If the bridge clearance height is greater than the ship clearance height, the ship has no super-elevation risk, and if the bridge clearance height is lower than the ship clearance height, the ship has a super-elevation risk; when the AIS predicts that the ship is located in the bridge dangerous area, it is judged that the ship has a bridge collision risk, and when the AIS predicts that the ship is located outside the bridge dangerous area, it is judged that the ship has no bridge collision risk.

[0063] The beneficial effects of the above embodiment are that ship image data of the bridge water area is collected, and ship profile data and ship position data are obtained; preprocessed AIS ship position data is obtained, and target ship position parameter data is obtained; camera assumed world coordinate system conversion is performed, and the ship position data converted to the camera assumed world coordinate system and the target ship position parameter data are matched; dynamic and static information of each ship is obtained, a three-dimensional safety distance model of the bridge water area is constructed, longitudinal safety distance and transverse safety distance are calculated, when a ship exists in an overheight condition and a ship-bridge collision dangerous behavior, the dangerous ship is highlighted and warning information is output. The present application realizes accurate monitoring of ship parameters by fusing and displaying ship image data and AIS data, outputs warning information when a ship exists in an overheight condition and a collision danger, and improves the safety of the bridge water area. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0065] Figure 1 A flowchart of an embodiment of the automatic monitoring and early warning method of the bridge water area ship provided by the present application;

[0066] Figure 2 An effect diagram of single-ship monitoring of an embodiment of the automatic monitoring and early warning method of the bridge water area ship provided by the present application;

[0067] Figure 3 An effect diagram of multi-ship monitoring of an embodiment of the automatic monitoring and early warning method of the bridge water area ship provided by the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0069] Some block diagrams shown in the drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.

[0070] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in one embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is expressly understood that the embodiments described herein are merely example embodiments of the application and that a substantial number of specific structures, features, configurations, materials, and components other than those described herein are also intended to be within the scope of the application.

[0071] Figure 1 A flowchart of an embodiment of a bridge water area ship automatic monitoring and early warning method provided by the application.

[0072] With reference to Figure 1 The application provides a bridge water area ship automatic monitoring and early warning method, comprising:

[0073] S101, collecting ship image data of a bridge water area, and establishing a ship target monitoring model based on the ship image data;

[0074] S102, obtaining ship contour data and ship position data by using the ship target monitoring model;

[0075] S103, obtaining AIS ship position data, and preprocessing the AIS ship position data to obtain target ship position parameter data;

[0076] S104, converting the ship position data and the target ship position parameter data to a camera assumed world coordinate system, matching the ship position data and the target ship position parameter data converted to the camera assumed world coordinate system in distance and direction, and storing the matched data information to a fusion data set;

[0077] S105, obtaining each ship dynamic and static information according to the fusion data set, constructing a bridge water area three-dimensional safety distance model, and calculating longitudinal safety distance and transverse safety distance;

[0078] S106, combining the three-dimensional safety distance model, the ship target contour data, the ship position data, and the target ship position parameter data, highlighting a dangerous ship and outputting early warning information when the ship has an overheight condition and a ship-bridge collision dangerous behavior.

[0079] The beneficial effects of the above embodiment are that ship image data of the bridge water area is collected, and ship profile data and ship position data are obtained; preprocessed AIS ship position data is obtained, and target ship position parameter data is obtained; camera assumed world coordinate system conversion is performed, and the ship position data converted to the camera assumed world coordinate system and the target ship position parameter data are matched; each piece of ship dynamic and static information is obtained, a three-dimensional safety distance model of the bridge water area is constructed, and the longitudinal safety distance and the transverse safety distance are calculated; when the ship has an overheight condition and a collision danger behavior, the dangerous ship is highlighted and warning information is output. The ship image data and the AIS data are fused and displayed, the ship parameters are accurately monitored, the warning information is output when the ship has an overheight condition and a collision danger, and the safety of the bridge water area is improved.

[0080] Needless to say, S101 and S102 can quickly identify ship targets and ship positions from the bridge water area by collecting ship image data of the bridge water area and obtaining ship profile data by using a ship target monitoring model, but cannot obtain the identity of the ship; S103 can confirm the ship position and identity information by obtaining target ship position parameter data through the AIS system, but there will be recognition errors when the bridge ship targets overlap and there is a ship name information shielding condition; S104 fuses the ship target profile and the target ship position parameter data to achieve accurate identification of the ship; S105 judges the overheight and collision danger of the ship by constructing a three-dimensional safety distance model of the bridge water area, provides a basis for the safe driving of the ship in the bridge water area, and thus improves the safety of the ship passing through the bridge water area.

[0081] In one embodiment, step S101 comprises:

[0082] Ship image data under different environmental conditions of the bridge water area is collected by using a camera assembly;

[0083] A ship target monitoring model is constructed based on an improved YOLOv5 target detection algorithm.

[0084] The loss function of the improved YOLOv5 target detection algorithm is EIoU Loss, and the non-maximum suppression algorithm is Soft-NMS.

[0085] Different environmental conditions refer to different postures, water area sizes, and light conditions of the bridge water area, and EIoU Loss and Soft-NMS are used to improve the loss function and the non-maximum suppression value of the YOLOv5 target detection algorithm, respectively, which can improve the monitoring capability of the ship target monitoring model for small ships and overlapping ships in the bridge area.

[0086] Further, step S102 comprises:

[0087] The ship target detection model is used to detect the ship image data of the bridge area water, and initial ship contour data and ship position data are obtained;

[0088] The ship contour data is obtained based on the Canny algorithm;

[0089] The ship contour data includes ship edge contour data and ship height data, and the ship position data includes the coordinates of the center point of the ship bottom edge contour in the pixel coordinate system. By extracting the ship edge contour data and the ship height data, the position information and the height information of the ship are determined.

[0090] The AIS system is a ship automatic identification system, which can realize the intercommunication of static information such as ship position, heading, speed, turning speed, distance between adjacent ships, ship name, call number, ship type, length and width of the ship between ships.

[0091] In one embodiment, step S103 includes:

[0092] The AIS ship position data within a preset period of the image timestamp is collected, and the AIS ship position data is preprocessed;

[0093] The initial ship position parameter data is calculated according to the preprocessed AIS ship position data;

[0094] The ship position parameter data outside the monitoring range of the bridge area waterway is screened out, and the target ship position parameter data is obtained;

[0095] The ship target contour data and the target ship position parameter data are aligned.

[0096] The preset period can be flexibly set according to actual needs, for example, the preset period is set to 30 seconds, and the preprocessing of the AIS ship position data includes cleaning the AIS ship position data to delete incomplete and chaotic AIS ship position data.

[0097] Further, the target ship position parameter data includes the initial position of the target ship, the heading ψ0, and the speed v0 information at the image timestamp T v . The calculation formula of the target ship position parameter data is:

[0098]

[0099] Wherein, T a represents the AIS information timestamp, p1 represents the initial time meridian elliptic curvature, and p2 represents the initial time latitude and longitude circle curvature.

[0100] In an embodiment, the calculation formula for the alignment processing of the ship target contour data and the target ship position parameter data is:

[0101]

[0102] wherein I a represents the bridge area monitoring range judgment index, represents the ship position at the time stamp t of the camera assembly, A v represents the detection range of the camera assembly, A a represents the channel monitoring range of the camera assembly; when the ship position is within the bridge area water monitoring range, i.e. I a = 1, the target ship information is retained, and when the ship position is outside the bridge area water monitoring range, I a = 0, the target information is deleted.

[0103] In an embodiment, the step S104 comprises:

[0104] obtaining the calibrated camera assembly parameter data;

[0105] converting the ship position data in the pixel coordinates to the camera assumed world coordinate system, and calculating the first position data of the target ship in the ship position data by the camera assembly;

[0106] converting the target ship position parameter data to the camera assumed world coordinate system, and calculating the second position data of the ship in the target ship position parameter data by the camera assembly;

[0107] fusing the first position data and the second position data to obtain fused data, and storing the fused data into the fused data set.

[0108] Further, the camera assembly parameter data comprises the pitch angle a p , the focal length f, the zero azimuth angle γ, the geographic position coordinates the internal parameter M1 and the external parameter M2, the first position data comprises the first distance, the first azimuth and the height of the target ship, and the calculation formula of the first position data is:

[0109]

[0110]

[0111]

[0112] H SP = y cq -y cp

[0113]

[0114] ​wherein d v represents the first distance, β v represents the first azimuth, γ represents the zero azimuth angle of the camera assembly, α p represents the pitch angle of the camera assembly, H SP represents the height of the target ship projection, H S represents the height of the target ship, (x w , y w ) represents the coordinates of the target ship in the camera- assumed world coordinate system, α represents the inner angle corresponding to the side where H S is located, α sp represents the inner angle corresponding to the side where H SP is located, y cq represents the vertical coordinate of the target ship profile top pixel information converted to the world coordinate system, y cp represents the vertical coordinate of the target ship profile bottom pixel information converted to the world coordinate system, v p represents the target ship image profile top pixel information, and I represents the image pixel height.

[0115] The target ship position parameter data includes AIS ship position data, the second position data includes a second distance and a second azimuth, and the calculation formula of the camera assembly and the second position data of the ship in the target ship position parameter data is:

[0116]

[0117]

[0118]

[0119]

[0120]

[0121] wherein (x wa , y wa , z wa ) represents the AIS ship position data, T a represents the AIS information timestamp, represents the geographic position coordinates of the camera assembly, d a represents the second distance, β a represents the second azimuth, R represents the earth radius, and (X, Y, 0) represents the ship coordinates of the AIS target ship in the camera assembly- assumed world coordinate system with the north direction as the zero azimuth.

[0122] Further, the fusion of the first position data and the second position data includes:

[0123] The first position data and the second position data with a difference value in a preset range are associated data;

[0124] The target number of AIS ships in the associated data is determined;

[0125] When the target number of AIS ships is greater than 1, the average error of the first position data and the second position data is calculated, and the first position data and the second position data with the smallest average error are screened out for fusion;

[0126] The calculation formula of the fusion of the first position data and the second position data is:

[0127]

[0128] Wherein, k o represents the distance weight coefficient, k d represents the azimuth weight coefficient, R o represents the distance correlation, R d represents the azimuth correlation.

[0129] Figure 2 An effect diagram of single-ship monitoring of an embodiment of the automatic monitoring and early warning method for bridge water area ships provided by the application, Figure 3 An effect diagram of multi-ship monitoring of an embodiment of the automatic monitoring and early warning method for bridge water area ships provided by the application.

[0130] Referring to Figure 2 and Figure 3 , the ship information can be accurately obtained through the fusion of the first position data and the second position data, the recognition error caused by the overlapping of multiple ships is avoided, and the accuracy of ship monitoring is improved.

[0131] In an embodiment, the calculation formula of the longitudinal safety distance and the transverse safety distance is:

[0132] S T = S H × 0.6 (B s + L sin β) + B L ;

[0133] S L = D1+D2+D3+D4+L S +L B ;

[0134]

[0135]

[0136]

[0137] Wherein, S T represents the lateral safety distance, S L represents the longitudinal safety distance, S H represents the vertical safety distance, B S represents the width of the ship, β represents the drift angle of the ship, B L represents the bridge pier width, D1 represents the ship's reversing brake deceleration distance, D2 represents the distance of the ship's towing anchor, D3 represents the wind flow increment, D4 represents the safety margin, L S represents the length of the ship, L B represents the length of the bridge pier, v0 represents the speed of the ship starting to reverse the brake, v represents the speed of the ship after ending the reverse brake, m represents the mass of the ship, P represents the power of the ship, k1 represents the wind load coefficient, k1 represents the flow load coefficient, v w represents the wind speed, v w represents the flow velocity, Δ represents the displacement of the ship, V represents the speed of the ship when anchoring, λ a represents the holding force coefficient of the anchor, W a represents the weight of the anchor in the air, K represents the uneven reduction coefficient of wind pressure, B w represents the wind area above the waterline of the ship, B d represents the wind area below the waterline of the ship, V s represents the speed of the ship, V w represents the speed of the wind, T w represents the time of the wind acting on the ship, α represents the wind pressure difference angle, V f represents the flow velocity, T f represents the time of the flow acting on the ship, β represents the flow pressure difference angle.

[0138] Need to explain, the vertical safety distance S H can refer to the value of 2 meters in “Inland Navigation Standard”, or can be flexibly set according to the actual situation of the water area.

[0139] In one embodiment, the formula for determining whether the ship has an overheight and ship-bridge collision danger behavior is:

[0140]

[0141]

[0142] Wherein, I H represents the overheight danger monitoring index, I c represents the collision danger monitoring index, represents the ship position coordinates at camera timestamp t, A v represents the left upstream pier dangerous area, A a represents the right upstream pier dangerous area, H B represents the bridge clearance height, HS The ship clearance height is represented.

[0143] According to the ship clearance height in the fusion data and the AIS ship position data, the three-dimensional safety distance model is constructed, the liquid radar is used to detect the bridge passing clearance height, so that whether the target ship has the super high and the ship bridge collision danger is realized.

[0144] Further, for the detected ship C =1, that is, the super high dangerous ship with the ship bridge collision danger, the ship is highlighted in the video monitoring of the monitoring personnel, and the detailed information of the ship is displayed near the ship, and the warning information is sent to the super high dangerous ship with the ship bridge collision danger.

[0145] Specifically, the warning information can be sent to the super high dangerous ship with the ship bridge collision danger through the very high frequency radio wave.

[0146] The beneficial effects of the above embodiment are that the ship image data of the bridge water area is collected, and the ship contour data is obtained; the preprocessed AIS ship position data is obtained, and the target ship position parameter data is obtained; the camera assumed world coordinate system conversion is performed, the ship contour data converted to the camera assumed world coordinate system and the target ship position parameter data are matched; the dynamic and static information of each ship is obtained, the three-dimensional safety distance model of the bridge water area is constructed, the longitudinal safety distance and the transverse safety distance are calculated, and whether the ship has the super high and the ship bridge collision danger behavior is judged. The ship image data and the AIS data are fused and displayed, the precision monitoring of the ship parameters is realized, the super high and the collision danger of the ship are judged by using the fusion data, and the safety of the bridge water area is improved.

[0147] The above describes in detail the bridge water area ship automatic monitoring and early warning method provided by the application, and the principles and implementation modes of the application are described by applying specific examples; the above embodiment is only used to help understand the method and the core idea of the application; meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation mode and the application range will be changed, and the above description should not be understood as the limitation of the application.

Claims

1. A method for automatic monitoring and early warning of a bridge water area ship, characterized in that, The method comprises the following steps: Collecting ship image data of the bridge water area, and establishing a ship target monitoring model based on the ship image data; Obtaining ship contour data and ship position data by using the ship target monitoring model; Obtaining AIS ship position data, and preprocessing the AIS ship position data to obtain target ship position parameter data; Obtaining calibrated camera component parameter data; Converting the ship position data in pixel coordinates to a camera assumed world coordinate system, calculating the first position data of the target ship in the ship position data and the camera component; Converting the target ship position parameter data to a camera assumed world coordinate system, calculating the second position data of the ship in the target ship position parameter data and the camera component; fusing the first position data and the second position data to obtain fusion data, and storing the fusion data in a fusion data set, wherein the camera component parameter data includes pitch, focal length, zero azimuth, geographic position coordinates, intrinsic parameters and extrinsic parameters, the first position data includes first distance and first azimuth, and the second position data includes second distance and second azimuth; According to the fusion data set, obtain each ship dynamic and static information, construct a three-dimensional safety distance model of the bridge water area, and calculate the longitudinal safety distance and the transverse safety distance; When the ship has an overheight condition and a ship-bridge collision danger behavior, highlight the dangerous ship and output warning information in combination with the three-dimensional safety distance model, the ship target contour data, the ship position data and the target ship position parameter data; The fusion of the first position data and the second position data comprises: Setting the first position data and the second position data with a difference within a preset range as associated data; Determining the target number of AIS ships in the associated data; When the target number of AIS ships is greater than 1, calculating the average error of the first position data and the second position data, and selecting the first position data and the second position data with the smallest average error for fusion; The calculation formula of the fusion of the first position data and the second position data is: ； wherein, denotes a distance weight coefficient, denotes an azimuth weight coefficient, denotes a distance correlation degree, represents the degree of orientation correlation; wherein , , , , denotes a first distance, denotes a first bearing, denotes a second distance, denotes a second bearing, denotes the coordinates of the target vessel in the camera assumed world coordinate system, denotes the AIS vessel position data, denotes the zero bearing of the camera assembly.

2. The method for automatic monitoring and early warning of bridge water area vessels according to claim 1, characterized in that, The method comprises the following steps: Collecting ship image data of the bridge water area, and establishing a ship target monitoring model based on the ship image data; Collecting ship image data of the bridge water area under different environmental conditions by using a camera component; Constructing a ship target monitoring model based on an improved YOLOv5 target detection algorithm; 3. The method for automatic monitoring and early warning of bridge water area vessels according to claim 1, characterized in that, The loss function of the improved YOLOv5 target detection algorithm is EIoU Loss, and the non-maximum suppression algorithm is Soft-NMS. The method comprises the following steps: Detecting the ship image data of the bridge water area by using the ship target detection model to obtain initial ship contour data and ship position data; Obtaining ship contour data based on a Canny algorithm; 4. The method for automatic monitoring and early warning of bridge water area vessels according to claim 1, characterized in that, The ship contour data includes ship edge contour data and ship height data, and the ship position data includes the coordinates of the center point of the ship bottom edge contour in the pixel coordinate system. The method comprises the following steps: Collect AIS ship position data in a preset period of image timestamp, and preprocess the AIS ship position data; Calculate initial ship position parameter data according to the preprocessed AIS ship position data; Screen out the ship position parameter data outside the bridge area channel monitoring range to obtain target ship position parameter data; Align the ship target contour data and the target ship position parameter data.

5. The method for automatic monitoring and early warning of bridge water area vessels according to claim 1, characterized in that, The first position data further includes the height of the target ship, and the calculation formula of the first position data is: ； ； ； wherein, represents the pitch angle of the camera assembly, represents the height of the target ship projection, represents the height of the target ship, represents the inner angle corresponding to the side where the target ship is located, represents the inner angle corresponding to the side where the target ship is located, represents the vertical coordinate of the top pixel information of the target ship profile converted to the world coordinate system, represents the vertical coordinate of the bottom pixel information of the target ship profile converted to the world coordinate system, represents the top pixel information of the target ship image profile, represents the image pixel height, f represents the focal length of the camera assembly, represents the zero azimuth angle of the camera assembly.

6. The method for automatic monitoring and early warning of the bridge water area ship according to claim 5, characterized in that, The target ship position parameter data includes AIS ship position data, and the calculation formula of the second position data of the ship in the calculation camera assembly and the target ship position parameter data is: ； ； ； wherein, represents an AIS information timestamp, represents the geographical position coordinates of the camera assembly, R represents the earth radius, represents the ship coordinates of the AIS target ship in the camera assembly assumed world coordinate system with the direction of true north as zero azimuth.

7. The method for automatic monitoring and early warning of the bridge water area ship according to claim 1, characterized in that, The calculation formula of the longitudinal safety distance and the transverse safety distance is: ； ； ； ； ; wherein, represents a lateral safety distance, represents a longitudinal safety distance, represents a vertical safety distance, represents a width of the ship, represents a leeway of the ship, represents a width of the bridge pier, represents a ship's reversing braking deceleration distance, represents a ship's towing anchor sailing distance, represents a wind flow increment, represents a safety margin, represents a length of the ship, represents a length of the bridge pier, represents a speed at which the ship starts reversing braking, represents a speed at which the ship ends reversing braking, represents a mass of the ship, represents a power of the ship, represents a wind load coefficient, represents a flow load coefficient, represents a wind speed, represents a flow speed, represents a displacement of the ship, represents a speed of the ship when anchoring, represents a holding power coefficient of the anchor, represents a weight of the anchor in air, represents a wind pressure uneven reduction coefficient, represents a wind area above the waterline of the ship, represents a wind area below the waterline of the ship, represents a speed of the ship, represents a speed of the wind, represents a time of wind acting on the ship, represents a wind pressure difference angle, represents a flow speed, represents a time of flow acting on the ship, represents a flow pressure difference angle.

8. The method for automatic monitoring and early warning of the bridge water area ship according to claim 7, characterized in that, The formula for judging whether the ship has the behavior of super-high and bridge collision danger is: ； ； wherein, represents an ultra-high risk monitoring index, represents a collision risk monitoring index, represents a ship position coordinate at a camera timestamp t, represents a left upstream pier dangerous area, represents a right upstream pier dangerous area, represents a bridge clearance height, represents a ship clearance height, represents a vertical safety distance.

Images