Intelligent ship size measuring method for active early warning of ship collision prevention of bridge

Abstract

The invention discloses a ship size intelligent measurement method for active early warning of bridge ship collision prevention, and the method comprises the steps: building a digital twinning scene of bridge navigation, and constructing a virtual image data set of ships in the scene; identifying ships in the virtual image data set; establishing a mapping relation between a digital twin scene local coordinate system and a two-dimensional image pixel coordinate system, and matching ships in the image with ships in the scene; ship size measurement is converted into solving of a regression problem, and the ship size is predicted through a regression model; the optimal weight of the regression model is extracted and migrated to a real scene to serve as a basic physical model, the basic physical model is trained and optimized, and an optimal measurement model with the highest ship height and width measurement precision is obtained; and carrying out early warning on the ultrahigh or ultra-wide ship based on a ship size measurement result. The method is not influenced by data volume limitation, can realize long-distance ship size measurement, and provides reliable support for active anti-collision early warning of a bridge.

Description

Technical Field

[0001] The invention belongs to the technical field of active bridge collision prevention, and in particular relates to an intelligent ship size measurement method for active early warning of bridge ship collision prevention. Background Art

[0002] With the rapid development of water transportation, the navigation density has increased dramatically, the tonnage of ships has also increased significantly, and the risk of ship-bridge collision has increased day by day. Therefore, research on bridge collision prevention is very important.

[0003] Bridge collision avoidance methods are mainly divided into passive collision avoidance and active collision avoidance. As for passive collision avoidance, bridge management units usually set up protective structures around bridge piers and superstructures to reduce the impact of ship-bridge collisions. These protective devices usually have high rigidity and strength, and once a collision occurs, they may cause serious damage to the ship. In addition, the collision resistance of the bridge superstructure is much lower than that of the bridge piers. Once a ship with a height exceeding the navigation clearance collides with the main beam, it may cause damage to the bridge superstructure or even collapse, seriously threatening the safety of vehicles and personnel on the bridge.

[0004] As an active means, long-distance identification and early warning of ships at risk of collision can greatly reduce the probability of ship-bridge collision. Ship size measurement is an effective way to achieve the above purpose. The current ship size measurement method relies on technologies such as laser radar and computer vision. However, the detection range of laser radar is limited. Even if an over-limit ship is detected, it cannot reserve enough early warning time for the bridge management department. The measurement accuracy of computer vision technology is greatly affected by the shooting distance, and ship images are usually taken at a long distance, which greatly reduces the measurement accuracy. In addition, the acquisition of ship image data sets is also a major problem. The emergence of digital twin technology provides a new idea for solving the above problems. By establishing a digital twin model, a large amount of simulation data that is not affected by the environment can be generated. However, there is a gap in the research of digital twin methods in the field of bridge ship collision prevention. Therefore, it is urgent to study the ship size measurement method from the perspective of digital twins to improve the active ship collision prevention capability of bridges. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent ship size measurement method for active bridge anti-ship collision warning, which is not affected by data volume limitations, can realize long-distance ship size measurement, and provide reliable support for active bridge anti-collision warning.

[0006] In order to achieve the above object, the solution of the present invention is:

[0007] An intelligent ship size measurement method for active warning of bridge ship collision prevention includes the following steps:

[0008] Step 1: Establish a digital twin scene of bridge navigation and construct a virtual image dataset of the ship in the scene; the three-dimensional coordinates P (X, Y, Z) and height H and width W of the ship are all known data;

[0009] Step 2: Use a single-stage target detection algorithm to identify the ships in the virtual image dataset and obtain the pixel height h and pixel width w of the ship; the coordinates of the representative point of the identified ship pixel are P o (u o ,v o );

[0010] Step 3: Establish a mapping relationship between the local coordinate system of the digital twin scene and the pixel coordinates of the virtual image, and convert the three-dimensional coordinates P (X, Y, Z) of the ship in the scene into the two-dimensional pixel coordinates P of the image plane. * (u * ,v * );Then according to P o With P * The Euclidean distance between matches the ship in the image and the ship in the scene;

[0011] Step 4: Based on the multi-source data that integrates the three-dimensional spatial coordinates and dimensions of the ship and the two-dimensional pixel coordinates and dimensions, a digital twin dataset is established to transform the ship dimension measurement into the solution of a regression problem, and the ship dimension is predicted through a regression model;

[0012] Step 5: Extract the optimal weights of the regression model and transfer them to the real scene as the basic physical model; collect real ship images, establish a physical data set with reference to the digital twin data set format, train and optimize the basic physical model, and obtain the optimal measurement model with the highest measurement accuracy for ship height and width; finally, issue an early warning for over-height or over-width ships based on the ship size measurement results.

[0013] In the above step 1, several virtual cameras are deployed at specific locations of the bridge in the digital twin scene to simulate the ships in the shooting scene, thereby constructing a virtual image dataset; wherein the number, deployment locations and parameter determination methods of the virtual cameras are as follows:

[0014] In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths of the virtual image in the x and y directions are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi, the principal point coordinates are (c xi ,c yi );

[0015] Define the coverage area function captured by the virtual camera The larger it is, the wider the shooting range of the virtual camera is;

[0016] Define the image clarity function taken by the virtual camera The larger the value, the higher the clarity of the image output by the virtual camera;

[0017] Define the multi-objective optimization problem:

[0018]

[0019] In the formula are the lower and upper limits of the parameter, respectively. is the objective function;

[0020] By solving all unknown quantities, the number, layout positions and parameters of virtual cameras are obtained.

[0021] In step 1 above, the ships in the digital twin scene are distributed in a grid pattern within a rectangular range with a length and width of q and p respectively, where the rectangle is divided into r grids along the direction perpendicular to the channel, and the length of each grid is l j , and satisfy the following relationship:

[0022]

[0023] In the formula, L is the length of the representative ship type of the bridge navigation defense, and S is the net width of the bridge navigation hole;

[0024] Ships are randomly distributed in each grid, and the three-dimensional size of the ships varies randomly according to the scale factor α, 0<α<1.

[0025] In step 3 above, the three-dimensional coordinates P (X, Y, Z) of the ship in the scene are converted into two-dimensional pixel coordinates P on the image plane. * (u * ,v * ), including, the conversion relationship is as follows:

[0026]

[0027] Where s is the scaling factor;

[0028] In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,yvi ,z vi ), the angle between the central axis and the YOZ plane is θ i ; The focal lengths in the x and y directions on the image plane are f xi 、f yi , the principal point coordinates are (c xi ,c yi ).

[0029] In the above step 3, according to P o With P * The minimum Euclidean distance between matching ships in the image and ships in the scene is calculated as follows:

[0030]

[0031] Where m and n represent the numbers of the ships in the scene, and m≠n.

[0032] In step 4 above, the composition of the digital twin dataset is as follows:

[0033] The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the center pixel coordinates of the prediction box when identifying the ship (u0, v0), the upper left corner pixel coordinates of the prediction box (u1, v1), the lower right corner pixel coordinates of the prediction box (u4, v4), and the coordinates of several randomly selected pixels in the prediction box;

[0034] The output labels are: the height H and width W of the ship, 2 labels.

[0035] In step 5 above, the early warning judgment mechanism for over-height or over-width ships is as follows:

[0036] If the following formula is met, the ship is considered to be too high:

[0037] H l ≤H p +H w

[0038] Among them, H l H is the bridge navigation clearance. p is the ship height output by the optimal measurement model, H w The real-time water level height of the navigation hole area;

[0039] The ship is considered over-breadth if the following formula is met:

[0040] W l ≤1.25W p

[0041] Among them, W l W is the clear navigation width of the bridge, p Ship width output by the optimal measurement model.

[0042] After adopting the above scheme, the beneficial effects of the present invention are as follows:

[0043] (1) The present invention can generate a large amount of virtual images, ship dimensions, ship positions and other multi-source data by establishing a digital twin model, which can effectively solve the problem of insufficient measured data;

[0044] (2) The present invention transforms the ship size measurement into the solution of a regression problem, and the measurement accuracy of the ship size is not affected by the camera shooting distance;

[0045] (3) The present invention uses transfer learning technology to transfer the model weights with the best performance in the digital twin scene to the real scene, which greatly improves the efficiency of ship size measurement;

[0046] (3) The present invention can realize long-distance ship size measurement, and can reserve sufficient time for the bridge management department to issue early warning instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flow chart of the present invention;

[0048] Figure 2 This is a schematic diagram of the digital twin model;

[0049] Figure 3 This is a schematic diagram of the grid distribution of ships;

[0050] Figure 4 This is a schematic diagram of target matching;

[0051] Figure 5 Schematic diagram of the digital twin dataset. DETAILED DESCRIPTION

[0052] The technical solutions and beneficial effects of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] like Figure 1 As shown, the present invention provides a ship size intelligent measurement method for active warning of bridge ship collision prevention, comprising the following steps:

[0054] Step 1: Virtual image acquisition. Establish a digital twin model of the bridge navigation, which can be used in conjunction with Figure 2 , a virtual camera is deployed at a specific position of the bridge to simulate the ships in the shooting scene, and then a two-dimensional virtual image data set is constructed. In order to obtain a large number of ship images, the ships in the digital twin scene are distributed in a grid format, and the three-dimensional coordinates P (X, Y, Z) and height H and width W of the ship are known;

[0055] Step 2: Ship target detection. Use the single-stage target detection algorithm to identify the ships in the virtual image dataset, obtain the prediction box information, and obtain the pixel height h and pixel width w of the ship. The coordinates of the detected ship pixel representative point are recorded as P o (u o ,v o );

[0056] Step 3: Ship target matching. Establish a mapping relationship between the local coordinate system of the 3D digital twin scene and the 2D image pixel coordinate system, and convert the 3D coordinates P (X, Y, Z) of the ship in space into the 2D pixel coordinates P of the image plane. * (u * ,v * ). According to P o With P * The Euclidean distance between them is used to match the ship in the image with the ship in the scene;

[0057] Step 4: Ship size measurement. After target matching, the three-dimensional spatial coordinates, dimensions and two-dimensional pixel coordinates and dimensions of the ship can be fused. Based on the fused multi-source data, a digital twin dataset is established, the ship size measurement is converted into the solution of a regression problem, and the ship size is predicted through the regression model;

[0058] Step 5: Real scene migration. Extract the optimal weights of the regression model and migrate them to the real scene as the basic physical model. Collect real ship images, establish a physical data set based on the digital twin data set format, train and optimize the basic physical model, and obtain the optimal measurement model with the highest accuracy in measuring the height and width of the ship. Finally, based on the ship size measurement results, issue an early warning for over-height or over-width ships.

[0059] Furthermore, the method for determining the number, layout positions and parameters of virtual cameras in step 1 is as follows:

[0060] Define the local world coordinate system O-XYZ in the virtual scene. For k virtual cameras (i=1, 2, ..., k), the installation position coordinates are V(x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths in the x and y directions on the image plane are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi , the principal point coordinates are (c xi ,c yi ).

[0061] Define the coverage area function captured by the virtual camera The larger the value, the wider the shooting range of the virtual camera.

[0062] Define the image clarity function taken by the virtual camera The larger the value, the higher the clarity of the image output by the virtual camera;

[0063] Define the multi-objective optimization problem:

[0064]

[0065] In the formula are the lower and upper limits of the parameter, respectively. is the objective function. Then all unknown quantities are solved to obtain the number, position and parameters of the virtual cameras.

[0066] like Figure 3 As shown, the grid distribution of ships in step 1 is:

[0067] The ships are distributed in a grid pattern within a rectangular range with a length of q and a width of p. The rectangle is divided into r grids along the direction perpendicular to the channel, and each grid is l long. j The mesh division must satisfy the following relationship:

[0068]

[0069] Where L is the length of the representative ship type of the bridge navigation defense, and S is the net width of the bridge navigation hole.

[0070] Ships are randomly distributed in each grid, and the three-dimensional size of the ships varies randomly according to a scale factor α (0<α<1), so that the ships can fill the entire grid as much as possible.

[0071] Furthermore, in step 3, P and P * The conversion relationship between them is as follows:

[0072]

[0073] Where s is a scaling factor.

[0074] like Figure 4 As shown, in step 3, the ships in the image and the ships in the scene are found by P o With P * The minimum Euclidean distance between them is used to match, and the calculation formula is as follows:

[0075]

[0076] Where m and n represent the numbers of the ship samples in the scene, and m≠n.

[0077] like Figure 5 As shown in the figure, in step 4, the composition of the digital twin dataset is as follows:

[0078] The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the coordinates of the center pixel of the prediction box (u0, v0), the coordinates of the upper left corner pixel of the prediction box (u1, v1), the coordinates of the lower right corner pixel of the prediction box (u4, v4), and the coordinates of 100 randomly selected pixels in the prediction box. There are 211 feature parameters in total.

[0079] The output labels are: the height H and width W of the ship, 2 labels.

[0080] In step 5, the early warning judgment mechanism for over-height or over-width ships is as follows:

[0081] The ship height output by the optimal measurement model is H p 、The width of the ship is W p ; The bridge navigation clearance is H l , net width is W l ; The real-time water level in the navigation hole area is H w .

[0082] If the following formula is met, the ship is considered to be too high:

[0083] H l ≤H p +H w

[0084] The ship is considered over-breadth if the following formula is met:

[0085] W l ≤1.25W p

[0086] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0087] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0088] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0090] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0091] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent ship size measurement method for active warning of bridge ship collision prevention, characterized in that The steps include: Step 1: Establish a digital twin scene of bridge navigation and construct a virtual image dataset of the ship in the scene; the three-dimensional coordinates P (X, Y, Z) and height H and width W of the ship are all known data; Step 2: Use a single-stage target detection algorithm to identify the ships in the virtual image dataset and obtain the pixel height h and pixel width w of the ship; the coordinates of the representative point of the identified ship pixel are P o (u o ,v o ); Step 3: Establish a mapping relationship between the local coordinate system of the digital twin scene and the pixel coordinates of the virtual image, and convert the three-dimensional coordinates P (X, Y, Z) of the ship in the scene into the two-dimensional pixel coordinates P of the image plane. * (u * ,v * );Then according to P o With P * The Euclidean distance between matches the ship in the image and the ship in the scene; Step 4: Based on the multi-source data that integrates the three-dimensional spatial coordinates and dimensions of the ship and the two-dimensional pixel coordinates and dimensions, a digital twin dataset is established to transform the ship dimension measurement into the solution of a regression problem, and the ship dimension is predicted through a regression model; Step 5: Extract the optimal weights of the regression model and transfer them to the real scene as the basic physical model; collect real ship images, establish a physical data set with reference to the digital twin data set format, train and optimize the basic physical model, and obtain the optimal measurement model with the highest measurement accuracy for ship height and width; finally, issue an early warning for over-height or over-width ships based on the ship size measurement results.

2. The method according to claim 1, characterized in that: In step 1, a number of virtual cameras are deployed at specific locations of the bridge in the digital twin scene to simulate the shooting of ships in the scene, thereby constructing a virtual image data set; wherein the number, deployment locations and parameter determination methods of the virtual cameras are as follows: In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i , the focal lengths of the virtual image in the x and y directions are f xi 、f yi , the horizontal field of view is v hi , the vertical field of view is v vi , the principal point coordinates are (c xi ,c yi ); Define the coverage area function captured by the virtual camera The larger the value, the wider the shooting range of the virtual camera. Define the image clarity function taken by the virtual camera The larger the value, the higher the clarity of the image output by the virtual camera; Define the multi-objective optimization problem: In the formula are the lower and upper limits of the parameter, respectively. is the objective function; By solving all unknown quantities, the number, layout positions and parameters of virtual cameras are obtained.

3. The method according to claim 1, characterized in that: In step 1, the ships in the digital twin scene are distributed in a grid pattern within a rectangular range with a length and width of q and p respectively, wherein the rectangle is divided into r grids along the direction perpendicular to the channel, and each grid has a length of l j , and satisfy the following relationship: In the formula, L is the length of the representative ship type of the bridge navigation defense, and S is the net width of the bridge navigation hole; Ships are randomly distributed in each grid, and the three-dimensional size of the ships varies randomly according to the scale factor α, 0<α<1.

4. The method according to claim 1, characterized in that: In step 3, the three-dimensional coordinates P (X, Y, Z) of the ship in the scene are converted into two-dimensional pixel coordinates P on the image plane. * (u * ,v * ), Including, the conversion relationship is as follows: Where s is the scaling factor; In the digital twin scene, the local world coordinate system O-XYZ is defined. For k virtual cameras, i = 1, 2, ..., k, the layout position coordinates are V (x vi ,y vi ,z vi ), the angle between the central axis and the YOZ plane is θ i ; The focal lengths in the x and y directions on the image plane are f xi 、f yi , the principal point coordinates are (c xi ,c yi ).

5. The method according to claim 1, characterized in that: In step 3, according to P o With P * The minimum Euclidean distance between matching ships in the image and ships in the scene is calculated as follows: Where m and n represent the numbers of the ships in the scene, and m≠n.

6. The method according to claim 1, characterized in that: In step 4, the digital twin dataset is composed as follows: The input features are: the three-dimensional coordinates of the ship (X, Y, Z), h, w, the center pixel coordinates of the prediction box when identifying the ship (u0, v0), the upper left corner pixel coordinates of the prediction box (u1, v1), the lower right corner pixel coordinates of the prediction box (u4, v4), and the coordinates of several randomly selected pixels in the prediction box; The output labels are: the height H and width W of the ship, 2 labels.

7. The method according to claim 1, characterized in that: In step 5, the early warning judgment mechanism for over-height or over-width ships is as follows: If the following formula is met, the ship is judged to be too high: H l ≤H p +H w Among them, H l H is the bridge navigation clearance. p is the ship height output by the optimal measurement model, H w The real-time water level height of the navigation hole area; The ship is considered over-breadth if the following formula is met: IN l ≤1.25W p Among them, W l W is the clear navigation width of the bridge, p Ship width output by the optimal measurement model.

Images