Intelligent Measurement Method for Ship Dimensions for Active Early Warning of Bridge Anti-Ship Collision

Abstract

The present invention discloses an intelligent method for measuring the size of ships for active early warning of ship collision prevention for bridges. The steps are as follows: establish a digital twin scenario for bridge navigation and construct a virtual image dataset of ships in the scenario; identify the ships in the virtual image dataset; establish a mapping relationship between the local coordinate system of the digital twin scenario and the two-dimensional image pixel coordinate system to match the ships in the image and the ships in the scenario; transform the ship size measurement into the solution of a regression problem and predict the ship size through a regression model; extract the optimal weights of the regression model and transfer them to the real scenario as a basic physical model, train and optimize the basic physical model to obtain an optimal measurement model with the highest measurement accuracy for ship height and width; and give early warnings for ships with excessive height or width based on the ship size measurement results. This method is not affected by the limitation of the data volume, can achieve long-distance ship size measurement, and provides reliable support for active anti-collision early warning of bridges.

Description

Technical Field

[0001] The present invention belongs to the technical field of active bridge anti-collision, and particularly relates to an intelligent measurement method for ship dimensions for active early warning of ship-bridge collision prevention for bridges. Background Art

[0002] With the rapid development of water transportation, the navigation density has increased sharply, and the tonnage of ships has also increased significantly. The risk of ship-bridge collision is increasing day by day. Therefore, the research on bridge anti-collision is very important.

[0003] Bridge anti-collision methods are mainly divided into two types: passive anti-collision and active anti-collision. As for passive anti-collision, bridge management units usually set up protective structures around bridge piers and superstructures to reduce the impact force of ship-bridge collisions. These protective devices usually have high stiffness and strength. Once a collision occurs, they may cause serious damage to ships. In addition, the anti-collision performance of the superstructure of the bridge is much smaller than that of the bridge pier. Once a ship with a height exceeding the navigable clearance collides with the main girder, it may cause damage or even collapse of the superstructure of the bridge, seriously threatening the safety of vehicles and personnel on the bridge.

[0004] As an active means, remotely identifying and warning ships at risk of hitting the bridge can greatly reduce the probability of ship-bridge collisions. Measuring the dimensions of ships is an effective way to achieve the above purpose. Current ship dimension measurement methods rely on technologies such as lidar and computer vision. However, the detection range of lidar is limited, and even if an over-limit ship is detected, it cannot reserve enough warning time for bridge management departments. The measurement accuracy of computer vision technology is greatly affected by the shooting distance, and ship images are usually taken at a long distance, resulting in a significant reduction in measurement accuracy. In addition, the acquisition of ship image datasets is also a major problem. The emergence of digital twin technology provides a new idea for solving the above problems. By establishing a digital twin model, a large amount of simulation data that is not affected by the environment can be generated. However, there is a gap in the research of digital twin methods in the field of ship-bridge collision prevention for bridges. Therefore, it is urgent to study ship dimension measurement methods from the perspective of digital twins to improve the active ship-bridge collision prevention ability of bridges. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent measurement method for ship dimensions for active early warning of ship-bridge collision prevention for bridges, which is not affected by the limitation of the amount of data, can achieve long-distance ship dimension measurement, and provides reliable support for active bridge anti-collision early warning.

[0006] To achieve the above object, the solution of the present invention is:

[0007] An intelligent measurement method for ship dimensions for active early warning of ship-bridge collision prevention for bridges includes the following steps:

[0008] Step 1: Create a digital twin scene of the bridge and construct a virtual image dataset of the ship in the scene. The three-dimensional coordinates P (X, Y, Z) and height H and width W of the ship are all known data.

[0009] Step 2: Use the single-stage target detection algorithm to identify the ship in the virtual image dataset and obtain the pixel height h and pixel width w of the ship; the coordinates of the pixel representative point of the identified ship are P o (u o ,v o );

[0010] Step 3: Establish a mapping relationship between the local coordinate system of the digital twin scene and the pixel coordinates of the virtual image, and convert the three-dimensional coordinates P (X, Y, Z) of the ship in the scene into the two-dimensional pixel coordinates P of the image plane. * (u * ,v * ); then according to P o With P * The Euclidean distance between the matching ship in the image and the ship in the scene;

[0011] Step 4: Based on the multi-source data that integrates the ship's 3D spatial coordinates and dimensions and 2D pixel coordinates and dimensions, a digital twin dataset is established. The ship's dimension measurement is converted into a regression problem, and the ship's dimensions are predicted using a regression model.

[0012] Step 5: Extract the optimal weights of the regression model and transfer them to the real scene as the basic physical model; collect real ship images, establish a physical dataset based on the digital twin dataset format, train and optimize the basic physical model, and obtain the optimal measurement model with the highest accuracy for ship height and width measurement; finally, issue an early warning for over-height or over-width ships based on the ship size measurement results.

[0013] In step 1 above, several virtual cameras are deployed at specific locations on the bridge in the digital twin scene to simulate capturing ships in the scene, thereby constructing a virtual image dataset. The number, deployment locations, and parameters of the virtual cameras are determined as follows:

[0014] In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths of the virtual image in the x and y directions are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi, the principal point coordinates are (c xi , c yi );

[0015] Define the coverage area function of the virtual camera The larger it is, the wider the shooting range of the virtual camera;

[0016] Define the image clarity function of the virtual camera The larger it is, the higher the clarity of the picture output by the virtual camera;

[0017] Define the multi-objective optimization problem:

[0018]

[0019] In the formula are the lower and upper limits of the parameters respectively, is the objective function;

[0020] By solving all unknowns, the number, layout positions and parameters of the virtual cameras are obtained.

[0021] In step 1 above, the ships in the digital twin scenario are distributed in a grid within a rectangle with length q and width p. Among them, the rectangle is divided into r grids along the direction perpendicular to the waterway, and the length of each grid is l j , and the following relational expressions are satisfied:

[0022]

[0023] In the formula, L is the length of the representative ship type for bridge navigation defense, and S is the clear width of the bridge navigation hole;

[0024] Ships are randomly distributed within each grid, and the three-dimensional dimensions of the ships vary randomly according to the scale factor α, where 0 < α < 1.

[0025] In step 3 above, the three-dimensional coordinates P(X, Y, Z) of the ships in the scenario are transformed into the two-dimensional pixel coordinates P * (u * , v * ) of the image plane, including the conversion relational expressions as follows:

[0026]

[0027] Among them, s is the scaling factor;

[0028] Define the local world coordinate system O-XYZ in the digital twin scenario. For k virtual cameras, i = 1, 2,..., k, the layout position coordinates are V(x vi , yvi , z vi ), the included angle between the central axis and the YOZ plane is θ i ; the focal lengths in the x and y directions on the image plane are f xi , f yi , and the principal point coordinates are (c xi , c yi ).

[0029] In step 3 above, match the ship in the image and the ship in the scene according to the minimum Euclidean distance between P o and P * . The calculation formula is as follows:

[0030]

[0031] In the formula, m and n represent the numbers of ships in the scene, and m ≠ n.

[0032] In step 4 above, the digital twin dataset is composed as follows:

[0033] The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the pixel coordinates of the center of the prediction box (u0, v0) when identifying the ship, the pixel coordinates of the upper left corner of the prediction box (u1, v1), the pixel coordinates of the lower right corner of the prediction box (u4, v4), and the coordinates of several randomly selected pixel points in the prediction box;

[0034] The output labels are: the height H and width W of the ship, 2 labels.

[0035] In step 5 above, the early warning judgment mechanism for ships with excessive height or width is as follows:

[0036] If the following formula is satisfied, it is determined that the ship is too high:

[0037] H l ≤ H p + H w

[0038] Among them, H l is the navigable clearance of the bridge, H p is the height of the ship output by the optimal measurement model, and H w is the real-time water level height in the navigable hole area;

[0039] If the following formula is satisfied, it is determined that the ship is too wide:

[0040] W l ≤ 1.25W p

[0041] Among them, W l is the navigable width of the bridge, and W p is the width of the ship output by the optimal measurement model.

[0042] After adopting the above solution, the beneficial effects of the present invention are as follows:

[0043] (1) By establishing a digital twin model, the present invention can generate a large amount of multi-source data such as virtual images, ship sizes, and ship positions, effectively solving the problem of insufficient measured data;

[0044] (2) The present invention transforms the ship size measurement into the solution of a regression problem, and the measurement accuracy of the ship size is not affected by the camera shooting distance;

[0045] (3) The present invention uses transfer learning technology to transfer the model weights with the best performance in the digital twin scenario to the real scenario, greatly improving the ship size measurement efficiency;

[0046] (3) The present invention can achieve long-distance ship size measurement, enabling the bridge management department to have sufficient time to issue early warning instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is the flowchart of the present invention;

[0048] Figure 2 is the schematic diagram of the digital twin model;

[0049] Figure 3 is the schematic diagram of the ship grid distribution;

[0050] Figure 4 is the schematic diagram of target matching;

[0051] Figure 5 is the schematic diagram of the digital twin data set. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] The technical solutions and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] As Figure 1 shown, the present invention provides an intelligent ship size measurement method for active ship collision prevention warning for bridges, including the following steps:

[0054] Step 1: Virtual image acquisition. Establish a digital twin model of bridge navigation, which can cooperate with Figure 2 , deploy virtual cameras at specific positions of the bridge to simulate the ships in the shooting scene, and then construct a two-dimensional virtual image data set. To obtain a large number of ship images, the ships in the digital twin scene are distributed in a grid pattern, and the three-dimensional coordinates P(X, Y, Z) and height H and width W of the ships are known;

[0055] Step 2: Ship target detection. Use a single-stage target detection algorithm to identify ships in the virtual image dataset, obtain the predicted bounding box information, and get the pixel height h and pixel width w of the ships. Denote the coordinates of the representative points of the detected ship pixels as P o (u o ,v o ).

[0056] Step 3: Ship target matching. Establish the mapping relationship between the local coordinate system of the 3D digital twin scene and the 2D image pixel coordinate system, and transform the 3D coordinates P(X, Y, Z) of the ship in space into the 2D pixel coordinates P * (u * ,v * ) in the image plane. Match the ships in the image and the ships in the scene according to the Euclidean distance between P o and P * ;

[0057] Step 4: Ship size measurement. After target matching, the 3D spatial coordinates, sizes of the ships and the 2D pixel coordinates, sizes can be fused. Based on the fused multi-source data, establish a digital twin dataset, transform the ship size measurement into the solution of a regression problem, and predict the ship size through a regression model;

[0058] Step 5: Real scene migration. Extract the optimal weights of the regression model and migrate them to the real scene as the basic physical model. Collect real ship images, establish a physical dataset according to the format of the digital twin dataset, train and optimize the basic physical model, and obtain the optimal measurement model with the highest measurement accuracy of ship height and width. Finally, give early warnings for ships with excessive height or width based on the ship size measurement results.

[0059] Furthermore, the determination methods for the number, layout positions and parameters of the virtual cameras in Step 1 are as follows:

[0060] Define a local world coordinate system O-XYZ in the virtual scene. For k virtual cameras (i = 1, 2,..., k), the installation position coordinates are V(x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths in the x and y directions on the image plane are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi , and the principal point coordinates are (c xi ,c yi ).

[0061] Define the coverage area function of the virtual camera shooting The larger it is, the wider the shooting range of the virtual camera is;

[0062] Define the image sharpness function of the virtual camera The larger it is, the higher the sharpness of the picture output by the virtual camera is;

[0063] Define the multi-objective optimization problem:

[0064]

[0065] In the formula are the lower and upper limits of the parameters respectively, is the objective function. Then solve all unknowns to obtain the number, position and parameters of the virtual camera.

[0066] Such as Figure 3 As shown, the grid distribution method of ships in step one is:

[0067] Distribute the ships in a grid within a rectangle with length q and width p. Divide the rectangle into r grids along the vertical channel direction, and the length of each grid is l j . The grid division needs to satisfy the following relational formula:

[0068]

[0069] In the formula, L is the length of the representative ship type for bridge navigation defense, and S is the clear width of the bridge navigation hole.

[0070] Randomly distribute the ships within each grid, and the three-dimensional size of the ships changes randomly according to a scale factor α (0 < α < 1), so as to make the ships fill the entire grid as much as possible.

[0071] Furthermore, in step three, the conversion relational formula between P and P * is as follows:

[0072]

[0073] In the formula, s is a scaling factor.

[0074] Such as Figure 4 As shown, in step three, the ships in the image and the ships in the scene are matched by finding the minimum Euclidean distance between P o and P * , and the calculation formula is as follows:

[0075]

[0076] In the formula, m and n represent the numbers of ship samples in the scene, and m ≠ n.

[0077] As Figure 5 shown, in step four, the digital twin data set is composed as follows:

[0078] The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the center pixel coordinates (u0, v0) of the prediction box, the upper-left pixel coordinates (u1, v1) of the prediction box, the lower-right pixel coordinates (u4, v4) of the prediction box, and the coordinates of 100 randomly selected pixel points in the prediction box. There are a total of 211 feature parameters.

[0079] The output labels are: the height H and width W of the ship, 2 labels.

[0080] In step five, the early warning judgment mechanism for ships with excessive height or width is as follows:

[0081] The height of the ship output by the optimal measurement model is H p , and the width of the ship is W p ; the navigable clearance height of the bridge is H l , and the navigable clearance width is W l ; the real-time water level height in the navigable hole area is H w .

[0082] If the following formula is satisfied, it is determined that the ship is too high:

[0083] H l ≤H p +H w

[0084] If the following formula is satisfied, it is determined that the ship is too wide:

[0085] W l ≤1.25W p

[0086] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented in various computer languages, for example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.

[0087] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device generate means for implementing the specified functions in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the specified functions in multiple blocks.

[0088] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the specified functions in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the specified functions in multiple blocks.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the specified functions in multiple blocks.

[0090] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0091] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent ship size measurement method for active warning of bridge ship collision prevention, characterized by The steps include: Step 1: Create a digital twin scene of the bridge and construct a virtual image dataset of the ship in the scene. The three-dimensional coordinates P (X, Y, Z) and height H and width W of the ship are all known data. Step 2: Use the single-stage target detection algorithm to identify the ship in the virtual image dataset and obtain the pixel height h and pixel width w of the ship; the coordinates of the pixel representative point of the identified ship are P o (u o ,v o ); Step 3: Establish a mapping relationship between the local coordinate system of the digital twin scene and the pixel coordinates of the virtual image, and convert the three-dimensional coordinates P (X, Y, Z) of the ship in the scene into the two-dimensional pixel coordinates P of the image plane. * (u * ,v * ); then according to P o With P * The Euclidean distance between the matching ship in the image and the ship in the scene; Step 4: Based on the multi-source data that integrates the ship's 3D spatial coordinates and dimensions and 2D pixel coordinates and dimensions, a digital twin dataset is established. The ship's dimension measurement is converted into a regression problem, and the ship's dimensions are predicted using a regression model. Step 5: Extract the optimal weights of the regression model and transfer them to the real scene as the basic physical model. Collect real ship images, build a physical dataset based on the digital twin dataset format, train and optimize the basic physical model, and obtain the optimal measurement model with the highest accuracy for ship height and width measurement. Finally, based on the ship size measurement results, issue early warnings for over-height or over-width ships. In step 1, several virtual cameras are deployed at specific locations of the bridge in the digital twin scene to simulate the shooting of ships in the scene, thereby constructing a virtual image dataset; wherein, the number, deployment locations, and parameters of the virtual cameras are determined as follows: In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths of the virtual image in the x and y directions are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi , the principal point coordinates are (c xi ,c yi ); Define the coverage area function captured by the virtual camera The larger it is, the wider the shooting range of the virtual camera is; Define the image clarity function taken by the virtual camera The larger the value, the higher the clarity of the image output by the virtual camera; Define the multi-objective optimization problem: In the formula are the lower and upper limits of the parameter, respectively. is the objective function; By solving all unknown quantities, the number, layout positions and parameters of virtual cameras are obtained; In step 1, the ships in the digital twin scene are distributed in a grid pattern within a rectangular range with a length of q and a width of p respectively, wherein the rectangle is divided into r grids along the direction perpendicular to the channel, and each grid has a length of l j , and satisfy the following relationship: Where, L is the length of the representative ship type of the bridge navigation defense, and S is the net width of the bridge navigation hole; Ships are randomly distributed in each grid, and the three-dimensional size of the ships varies randomly according to the scale factor α, 0<α<1.

2. The method according to claim 1, wherein: In step 3, the three-dimensional coordinates P (X, Y, Z) of the ship in the scene are converted into the two-dimensional pixel coordinates P of the image plane. * (u * ,v * ), Including, the conversion relationship is as follows: Where s is the scaling factor; In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i ; The focal lengths in the x and y directions on the image plane are f xi 、f yi , the principal point coordinates are (c xi ,c yi ).

3. The method according to claim 1, wherein: In step 3, according to P o With P * The minimum Euclidean distance between the matching ship in the image and the ship in the scene is calculated as follows: Where m and n represent the ship numbers in the scene, and m≠n.

4. The method according to claim 1, wherein: In step 4, the digital twin dataset is composed as follows: The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the coordinates of the center pixel of the prediction box when identifying the ship (u0, v0), the coordinates of the upper left corner pixel of the prediction box (u1, v1), the coordinates of the lower right corner pixel of the prediction box (u4, v4), and the coordinates of several randomly selected pixels in the prediction box; The output labels are: the height H and width W of the ship, 2 labels.

5. The method according to claim 1, wherein: In step 5, the early warning judgment mechanism for over-height or over-width ships is as follows: The ship is considered to be overheight if the following formula is met: H l ≤H p +H w Among them, H l For the bridge navigation clearance, H p is the ship height output by the optimal measurement model, H w The real-time water level height of the navigation hole area; A ship is considered over-width if the following formula is met: IN l ≤1.25W p Among them, W l W is the clear navigation width of the bridge, p The ship width is output by the optimal measurement model.

Images