Method, device and computer equipment for measuring ship speed

Abstract

The present application relates to a method, device, and computer equipment for measuring the speed of a ship, and relates to the field of ship control technology. The method comprises: acquiring image data containing a ship; inputting the image data into a pre-set ship detection model to obtain the ship's position and category information; determining the ship's trajectory based on the position information and a Kalman filter algorithm; determining the ship's relative speed in the image coordinate system based on the trajectory; and performing a homography transformation on the relative speed based on the category information to obtain the ship's speed in the world coordinate system. Because this method can determine the ship's speed using image data obtained with a visible light imaging device, it eliminates the need for expensive measurement equipment such as laser probes, thereby reducing the cost of ship speed measurement.

Description

Technical Field

[0001] The present application relates to the field of ship control technology, and in particular to a method, device and computer equipment for measuring the navigation speed of a ship. Background Art

[0002] The speed of a ship refers to the distance the ship travels relative to the seabed per unit time. It affects the ship's transportation cycle, operating costs and shipping competitiveness, and is an important technical performance of transport ships.

[0003] At present, the main methods for measuring the speed of a ship include laser ranging ship speed detection method and ultrasonic ranging ship speed detection method. Both methods require the acquisition of laser data or ultrasonic data of the ship, which places high demands on the measurement equipment. For example, the laser ranging ship speed detection method requires the use of a laser probe that is complex to operate and expensive. Therefore, the existing methods for measuring the speed of a ship are relatively expensive. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device and computer equipment for measuring the ship's navigation speed with low measurement cost in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for measuring a ship's sailing speed. The method for measuring a ship's sailing speed includes:

[0006] Acquiring image data containing ships;

[0007] Input the image data into the pre-set ship detection model to obtain the ship's location and category information;

[0008] Obtain the ship's navigation trajectory based on position information and Kalman filter algorithm;

[0009] Determine the relative speed of the ship in the image coordinate system based on the navigation trajectory;

[0010] The relative speed is transformed homographically according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0011] In one embodiment, performing homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system includes:

[0012] Determine the actual length of the ship in the world coordinate system based on the category information;

[0013] Get the relative length of the ship in the image coordinate system;

[0014] The navigation speed is obtained according to the formula of relative length, relative speed, actual length and homography transformation.

[0015] In one embodiment, the formula for homography transformation includes:

[0016]

[0017] Among them, V0 is the navigation speed, W0 is the actual length, V1 is the relative speed, and W1 is the relative length.

[0018] In one embodiment, obtaining the navigation trajectory of a ship based on the position information and a Kalman filter algorithm includes:

[0019] Determine the observation vector and the system state vector according to the position information;

[0020] According to the observation vector, system state vector and Kalman filter algorithm, the ship is tracked to obtain the ship's navigation trajectory.

[0021] In one embodiment, tracking a ship based on an observation vector, a system state vector, and a Kalman filter algorithm to obtain a navigation trajectory of the ship includes:

[0022] Determine the state transfer matrix based on the system state vector and the pre-set uniform motion model;

[0023] The state transfer matrix, observation vector and system state vector are input into the Kalman filter algorithm to obtain the ship's position estimation information;

[0024] Generate the ship's navigation trajectory based on the position estimation information.

[0025] In one embodiment, acquiring image data containing a vessel includes:

[0026] Receive image data containing ships sent by visible light imaging equipment carried by drones during ship shipping patrol missions.

[0027] In one embodiment, before inputting the image data into a pre-set ship detection model, the measurement method further includes:

[0028] Obtain historical patrol image data;

[0029] Classify and mark the positions of ships in historical patrol image data to obtain training data;

[0030] Preprocessing the training data to obtain preprocessed training data;

[0031] The preprocessed training data is used to train the YOLO model to obtain a ship detection model.

[0032] In one embodiment, preprocessing the training data to obtain preprocessed training data includes:

[0033] Perform convolution operation on training data using mean template;

[0034] The data obtained after the convolution operation is logarithmically transformed to obtain the preprocessed training data.

[0035] In a second aspect, the present application further provides a device for measuring the speed of a ship. The device for measuring the speed of a ship comprises:

[0036] A data acquisition module, used to acquire image data containing ships;

[0037] The data prediction module is used to input the image data into a pre-set ship detection model to obtain the location and category information of the ship;

[0038] The trajectory determination module is used to obtain the navigation trajectory of the ship based on the position information and the Kalman filter algorithm;

[0039] A speed calculation module is used to determine the relative speed of the ship in the image coordinate system based on the navigation trajectory;

[0040] The speed transformation module is used to perform homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0041] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program and the processor executes the steps of the above-mentioned method for measuring the sailing speed of a ship.

[0042] The above-described method, device, and computer equipment for measuring ship speed obtain image data containing a ship; input the image data into a pre-set ship detection model to obtain the ship's position and category information; determine the ship's trajectory based on the position information and a Kalman filter algorithm; determine the ship's relative speed in the image coordinate system based on the trajectory; and perform a homography transformation on the relative speed based on the category information to obtain the ship's speed in the world coordinate system. Because this method can determine the ship's speed using image data obtained with a visible light imaging device, it eliminates the need for expensive measurement equipment such as laser probes, reducing the cost of ship speed measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 1 is a flow chart of a method for measuring a ship's sailing speed in one embodiment;

[0044] Figure 2 In one embodiment Figure 1 A schematic flow chart of step 110;

[0045] Figure 3In one embodiment Figure 1 Flowchart of step 106;

[0046] Figure 4 In one embodiment Figure 3 Flowchart of step 304;

[0047] Figure 5 A schematic flow chart of a method for determining a ship's navigation speed in another embodiment;

[0048] Figure 6 In one embodiment Figure 5 Flowchart of step 506;

[0049] Figure 7 A schematic flow chart of a method for determining a ship's navigation speed in another embodiment;

[0050] Figure 8 1 is a structural block diagram of a device for measuring the speed of a ship according to an embodiment;

[0051] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0053] In one embodiment, Figure 1 As shown, a method for measuring the speed of a ship is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understood that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method may include the following steps:

[0054] Step 102: Acquire image data containing a ship.

[0055] Among them, image data refers to the collection of grayscale values ​​of each pixel expressed by numerical values. It has the characteristics of non-contact, long-distance, rich information and low hardware cost, and is widely used in various industries.

[0056] Specifically, image data containing ships can be obtained in a variety of ways. In one example, an image acquisition device and an embedded device can be installed on a terminal (such as an aircraft). The image acquisition device captures image data containing ships along the aircraft's flight path and stores the image data in the embedded device. The embedded device receives the image data, obtains the image data containing ships, and then processes it. In another example, an image acquisition device and an image transmission device can be installed on a terminal (such as an aircraft). The image acquisition device captures image data containing ships along the aircraft's flight path, and the image transmission device transmits the image data to a server. The server receives the image data, obtains the image data containing ships, and then processes it.

[0057] It should be noted that because image acquisition devices cannot directly perceive and measure distances in the world coordinate system, they cannot directly observe the ship's speed, nor can they determine the motion status of ships sailing in the monitored sea area (for example, anchoring, speeding, or underspeed), and thus cannot provide early warnings for certain dangerous ship behaviors. Using the measurement method of this embodiment, a terminal or server can obtain the ship's speed in the world coordinate system by analyzing and converting image data containing the ship.

[0058] Step 104: input the image data into a pre-set ship detection model to obtain the location information and category information of the ship.

[0059] Classification information refers to information used to categorize ships. For example, ships can be categorized into 1,000-ton, 10,000-ton, 50,000-ton, and 100,000-ton classes based on their carrying capacity. Position information refers to information related to the ship's position in the image data, such as coordinates, width, and height.

[0060] Specifically, pre-defined ship detection models can be established using methods based on image feature extraction and sliding windows, or using deep neural network regression. The former algorithm requires a large amount of computation, has poorly effective region selection strategies, and has low adaptability to manually extracted features, resulting in high time and space complexity. Furthermore, object detection accuracy typically only reaches a maximum of 35.1%. The latter, on the other hand, can fit arbitrary functions by increasing the number of neural network nodes. Leveraging the powerful fitting capabilities of neural networks, it can automatically learn the characteristics of ships in an image. The features extracted by the neural network are then fed into a fully connected network for regression classification, typically achieving a recognition accuracy of 92%.

[0061] After the ship detection model is generated, it can be used directly on the server for prediction, or it can be burned into the embedded device of the terminal. In this way, the category information and location information of the ships in the current scene can be analyzed in real time during the inspection process.

[0062] Step 106: Obtain the navigation trajectory of the ship based on the position information and the Kalman filter algorithm.

[0063] The Kalman filter algorithm uses a linear system state equation and observation data from the system input and output to optimally estimate the system state. In this embodiment, observation data can be determined based on position information, and then input into the Kalman filter algorithm to obtain the optimal position estimate information output by the Kalman filter, thereby determining the vessel's navigation trajectory.

[0064] Step 108: Determine the relative speed of the ship in the image coordinate system according to the navigation trajectory.

[0065] In a specific implementation, the displacement of the ship can be determined based on the navigation trajectory, and then the relative speed of the ship in the image coordinate system can be calculated based on the displacement. In one example, the relative speed can be calculated according to the following formula:

[0066]

[0067] Among them, V1 is the relative velocity, S is the displacement, and T is the prediction time.

[0068] Step 110 , performing homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0069] It should be understood that homography is a transformation relationship from one plane to another, and can also be used to describe the position mapping relationship between an object in the world coordinate system and the image coordinate system. After calculating the relative speed of the ship in the image coordinate system, the relative speed in the image coordinate system can be converted into the navigation speed in the world coordinate system through homography.

[0070] This embodiment acquires image data containing a ship; inputs the image data into a pre-defined ship detection model to obtain the ship's position and classification information; derives the ship's trajectory based on the position information and a Kalman filter algorithm; determines the ship's relative speed in the image coordinate system based on the trajectory; and performs a homography transformation on the relative speed based on the classification information to obtain the ship's speed in the world coordinate system. Because this method can determine the ship's speed using image data obtained with a visible light imaging device, it eliminates the need for expensive measurement equipment such as laser probes, reducing the cost of ship speed measurement.

[0071] Figure 2 In one embodiment Figure 1 The flow chart of step 110 is as follows: Figure 2 As shown, in one embodiment, step 110 may include the following steps:

[0072] Step 202: Determine the actual length of the ship in the world coordinate system according to the category information.

[0073] Step 204: Obtain the relative length of the ship in the image coordinate system.

[0074] Step 206: Obtain the navigation speed according to the formula of relative length, relative speed, actual length and homography transformation.

[0075] It should be understood that there is a certain correspondence between the category and length of a ship. For example, the length of a 1,000-ton ship is usually about 40m, the length of a 10,000-ton ship is usually about 118m, the length of a 50,000-ton ship is usually about 247m, and the length of a 100,000-ton ship is usually about 290m. Therefore, the actual length of the ship in the world coordinate system can be determined through the ship's category information.

[0076] In one example, a mapping relationship table between ship categories and actual lengths may be pre-established, and then the mapping relationship table may be searched based on the category information predicted by the ship detection model to find the actual length corresponding to the category information.

[0077] In a specific implementation, the relative length of the ship in the image coordinate system can be determined based on the position information predicted by the ship detection model, or by measuring the length of the ship in the image data, which is not limited in this embodiment.

[0078] It should be understood that there is a certain mapping relationship between the relative length and relative speed of the ship in the image coordinate system and the actual length and sailing speed of the ship in the world coordinate system. The homography transformation can represent this mapping relationship. Therefore, based on the three parameters of relative length, relative speed and actual length, and the formula of the homography transformation, the sailing speed of the ship in the world coordinate system can be calculated.

[0079] In one example, the formula for the homography transformation in step 206 may include:

[0080]

[0081] Among them, V0 is the navigation speed, W0 is the actual length, V1 is the relative speed, and W1 is the relative length.

[0082] This embodiment performs a homography transformation based on the actual length of the ship in the world coordinate system, the relative length of the ship in the image coordinate system, and the relative speed, thereby converting the relative speed of the ship in the image coordinate system into the navigation speed in the world coordinate system, thereby achieving speed measurement of long-distance ships and improving measurement accuracy.

[0083] Figure 3 In one embodiment Figure 1 The flow chart of step 106 is as follows: Figure 3 As shown, in one embodiment, step 106 may include the following steps:

[0084] Step 302: Determine an observation vector and a system state vector based on the position information.

[0085] Step 304: Track the ship according to the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0086] It should be understood that the Kalman filter algorithm uses the observation vector to correct the system state vector, which changes over time, to obtain the optimal state vector. In a specific implementation, the terminal can determine the observation vector and the system state vector based on the coordinates, width, and height of the center point of the ship target area in the image in the position information.

[0087] Specifically, the observation vector can be defined as Z = [x, y, w, h], and the system state vector can be defined as: X = [x, y, w, h, vx, vy], where x and y are the coordinates of the center point of the ship target area in the image, w and h are the width and height of the ship target area in the image, respectively, and vx and vy are the ship's running speeds in the x and y directions.

[0088] After defining the observation vector and the system state vector, the state transfer matrix can be determined based on the system state vector, and then the state transfer matrix, the observation vector and the system state vector are input into the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0089] This embodiment determines the observation vector and the system state vector based on the position information, tracks the ship based on the observation vector, the system state vector and the Kalman filter algorithm, and obtains the navigation trajectory of the ship, thereby realizing the application of the Kalman filter algorithm in image target tracking.

[0090] Figure 4 In one embodiment Figure 3 The flow chart of step 304 is as follows: Figure 4 As shown, in one embodiment, step 304 may include the following steps:

[0091] Step 402: Determine a state transfer matrix based on the system state vector and a preset uniform motion model.

[0092] In step 404, the state transfer matrix, the observation vector and the system state vector are input into a Kalman filter algorithm to obtain the estimated position information of the ship.

[0093] Step 406: Generate the navigation track of the ship based on the position estimation information.

[0094] The state transfer matrix is ​​a function of time and initial time. Taking the system state vector as: X = [x, y, w, h, vx, vy] as an example, assuming that the uniform motion model of the ship is as shown below:

[0095] x'=x+vx*dt

[0096] y'=y+vy*dt;

[0097] Then the state transfer matrix A of the ship motion system is:

[0098]

[0099] The system state model in the Kalman filter algorithm is: X k+1 =A k X k +w k , where X is an n-dimensional column vector representing the system state vector; w is an n-dimensional column vector representing the system process noise, w is assumed to be non-zero mean Gaussian white noise; A is an n*n matrix representing the state transfer matrix, which represents the transition relationship between the system state from time t-1 to time t. Based on the state transfer matrix A and the system state model, the system state vector X at the next moment can be determined k+1 .

[0100] The observation model in the Kalman filter algorithm is: Z k =H k X k +v k , where Z is an m-dimensional column vector representing the observation vector; v is an m-dimensional column vector representing the measurement noise vector, where v is assumed to be non-zero-mean Gaussian white noise. H is an m*n matrix representing the measurement matrix. The measurement matrix H can be determined based on the observation vector z and the observation model.

[0101] The time update equation in the Kalman filter algorithm is:

[0102]

[0103]

[0104] The measurement update equation in the Kalman filter algorithm is:

[0105]

[0106] Where R is the measurement noise, which obeys Gaussian distribution; K is the Kalman gain. Substitute the state transfer matrix A, measurement matrix H, observation vector Z, and system state vector X into the above time update equation and measurement update equation to obtain the system state vector X at the next moment. K+1 Correction is performed to obtain the optimal state estimate, that is, the optimal position estimate information of the ship.

[0107] It should be understood that for the position information at a certain moment, the position estimation information of the ship at that moment can be determined based on the position information at that moment and the Kalman filter algorithm. During a period of observation time, the position estimation information of the ship at different moments can be determined based on the position information at multiple different moments, and the position estimation information at multiple different moments can form the navigation track of the ship.

[0108] This embodiment determines the state transfer matrix based on the system state vector and a pre-set uniform motion model; inputs the state transfer matrix, observation vector and system state vector into the Kalman filter algorithm to obtain the ship's position estimation information; and generates the ship's navigation trajectory based on the position estimation information, thereby improving the accuracy of the navigation trajectory.

[0109] Figure 5 FIG. 1 is a flow chart of a method for determining a ship's sailing speed in another embodiment. Figure 5 As shown, the method for measuring the ship's sailing speed may further include the following steps:

[0110] Step 502: Acquire historical patrol image data.

[0111] Step 504: Classify and mark the positions of the ships in the historical patrol image data to obtain training data.

[0112] Step 506: preprocess the training data to obtain preprocessed training data.

[0113] Step 508: Use the preprocessed training data to train the YOLO model to obtain a ship detection model.

[0114] The historical patrol image data refers to patrol image data collected by the image acquisition device before the image data containing the ship is obtained.

[0115] It should be understood that a large amount of patrol image data is generated during a ship shipping patrol mission. Some images are shot at a long distance and contain less effective information, making them unsuitable for later image recognition applications. Some images are shot at a close distance and contain more detailed effective information, which facilitates high-precision recognition and target tracking in the later stages. After obtaining the historical patrol image data sent by the drone's visible light imaging equipment during the patrol mission, the ships in the historical patrol image data can be divided into four types according to their tonnage: 1,000 tons, 10,000 tons, 50,000 tons, and 100,000 tons. The comparison table between tonnage and length is shown in Table 1 below:

[0116] Table 1

[0117] category Thousand-ton class 10,000-ton level 50,000-ton class 100,000-ton class Length (m) 40 118 247 290

[0118] Then, the ships in the historical patrol image data are classified and labeled to produce effective training data.

[0119] During the preprocessing of the training data, the training data may be subjected to preprocessing such as convolution scaling and nonlinear enhancement. For example, a convolution template is used to implement a convolution operation on the training data, and during the convolution operation, the image in the training data is scaled to a multiple of 32 resolution, such as a resolution of 416*416. In one example, the mathematical expression of the convolution operation is as follows:

[0120] I(x,y)=S(x,y)*T(x,y);

[0121] Where S is the input original image, I is the output result of the convolution operation, T is the convolution template, and x and y are the horizontal and vertical coordinates of the pixel point in the image coordinate system, respectively.

[0122] In this embodiment, to improve detection accuracy, the YOLO model can be used for training. The YOLO model is an object detection method that uses a single neural network to directly predict object boundaries and class probabilities, boasting fast detection speed and high accuracy. The preprocessed training data is fed into the YOLO model, which segments the input image into N*N grids. The YOLO model predicts multiple bounding boxes for each grid, along with the corresponding confidence values ​​for each box, and also predicts the probability of being similar to each class to be detected. During training, stochastic gradient descent can be used to find the optimal solution to the loss function within the training data, accelerating the convergence of the loss function and rapidly obtaining a ship detection model.

[0123] In this embodiment, historical patrol image data is obtained, the ships in the historical patrol image data are classified and marked, training data is obtained, the training data is preprocessed to obtain preprocessed training data, and the preprocessed training data is used to train the YOLO model to obtain a ship detection model. The ship detection model can then be used to determine the category information and location information of the ship, thereby improving the detection accuracy.

[0124] Figure 6 In one embodiment Figure 5 The flow chart of step 506 is as follows: Figure 6 As shown, step 506 may further include the following steps:

[0125] Step 602: Perform convolution operation on the training data using the mean template.

[0126] Step 604: Perform logarithmic transformation on the data obtained after the convolution operation to obtain preprocessed training data.

[0127] Among them, the mean template is a type of convolution template, which refers to averaging the pixel values ​​in the area covered by the template, that is, average pooling. By using the mean template for convolution operation, image downsampling can be achieved and the image can be made smoother.

[0128] By using logarithmic transformation, the low gray value areas in the image can be expanded and the high gray value areas can be compressed to emphasize the low gray value areas of the image, thereby achieving the purpose of enhancing the image. In an example, the formula for logarithmic transformation can be:

[0129]

[0130] Among them, r is the control coefficient, which controls the intensity of the change of the enhancement curve. The larger r is, the steeper the curve is, and the more obvious the stretching of the low gray value area is. c is the scaling coefficient, and v is the pixel value of the input image.

[0131] Of course, in a specific implementation, after performing logarithmic transformation on the data obtained after the convolution operation, the data with too high or too low gray levels can be corrected by gamma transformation to enhance the contrast of the data. In addition, piecewise linear transformation can be used to enhance the contrast of each part of the data.

[0132] This embodiment uses a mean template to perform a convolution operation on the training data and performs a logarithmic transformation on the data obtained after the convolution operation. This makes the preprocessed training data less noisy and smoother, reduces the impact of light changes on ship detection, and improves training efficiency.

[0133] In one embodiment, Figure 1 Step 102 may include the following steps:

[0134] Step 1021: Receive image data containing ships sent by the visible light imaging device carried by the UAV during the ship shipping inspection mission.

[0135] It should be understood that the application of computer vision technology in ship and shipping production inspection and management focuses on the following areas: 1. Using drones for investigation, evidence collection, and emergency response can ensure rapid response and timely investigations, enabling the identification and tracking of hit-and-run vessels to prevent their escape; 2. Using drone-mounted cameras to inspect ships for overloading; 3. Using drone-mounted thermal imaging to check crew numbers; 4. Using onboard video and photography equipment to record and preserve evidence for investigation and processing. 5. Using drones to organize and direct ship traffic. Because drone-mounted visible light imaging equipment cannot directly perceive or measure distance in a world coordinate system, drones cannot be used to observe ship speeds.

[0136] In this embodiment, the terminal can be a drone. By mounting a visible light imaging device on the drone's gimbal, real-time image data of the flight path is acquired, and the image data is analyzed and processed to observe the ship's speed. During the visible light imaging device's shooting process, the gimbal can also accurately control the shooting angle and speed.

[0137] In one example, a visible light imaging device and an embedded device can be mounted on a gimbal on a drone. The visible light imaging device captures image data containing ships along the drone's flight path and stores the image data in the embedded device. The embedded device receives the image data and processes the image data after obtaining the image data containing ships.

[0138] In another example, a drone can be equipped with a visible light imaging device and an image transmission device on a gimbal. The visible light imaging device captures image data containing ships along the drone's flight path, and the image transmission device transmits the image data to a server. The server receives the image data and processes the image data containing ships after obtaining it.

[0139] In this embodiment, the visible light imaging device carried by the drone is used to capture image data containing the ship, thereby avoiding the use of high-cost measurement equipment such as laser probes, thereby further reducing the measurement cost.

[0140] Figure 7 FIG. 1 is a flow chart of a method for determining a ship's sailing speed in another embodiment. Figure 7 As shown, the method for measuring the ship's sailing speed includes the following steps:

[0141] Step 502: Acquire historical patrol image data.

[0142] Step 504: Classify and mark the positions of the ships in the historical patrol image data to obtain training data.

[0143] Step 602: Perform convolution operation on the training data using the mean template.

[0144] Step 604: Perform logarithmic transformation on the data obtained after the convolution operation to obtain preprocessed training data.

[0145] Step 508: Use the preprocessed training data to train the YOLO model to obtain a ship detection model.

[0146] Step 1021: receiving image data containing ships sent by a visible light imaging device carried by a UAV during a ship shipping patrol mission.

[0147] Step 104: input the image data into a pre-set ship detection model to obtain the location information and category information of the ship.

[0148] Step 302: Determine an observation vector and a system state vector based on the position information.

[0149] Step 402: Determine a state transfer matrix based on the system state vector and a preset uniform motion model.

[0150] In step 404, the state transfer matrix, the observation vector and the system state vector are input into a Kalman filter algorithm to obtain the estimated position information of the ship.

[0151] Step 406: Generate the navigation track of the ship based on the position estimation information.

[0152] Step 108: Determine the relative speed of the ship in the image coordinate system according to the navigation trajectory.

[0153] Step 202: Determine the actual length of the ship in the world coordinate system according to the category information.

[0154] Step 204: Obtain the relative length of the ship in the image coordinate system.

[0155] Step 206: Obtain the navigation speed according to the formula of relative length, relative speed, actual length and homography transformation.

[0156] Through the above method, this embodiment uses a low-cost drone and visible light imaging equipment to obtain image data containing ships, uses a ship detection model to predict the image data, and combines the Kalman filter algorithm to achieve long-distance ship speed measurement, expanding the application scenarios of drones and reducing measurement costs.

[0157] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0158] Based on the same inventive concept, embodiments of the present application also provide a device for measuring ship speed for implementing the aforementioned method for measuring ship speed. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for measuring ship speed provided below can be found in the aforementioned definitions of the method for measuring ship speed, and will not be further elaborated here.

[0159] In one embodiment, Figure 8 As shown, a device for measuring the navigation speed of a ship is provided, including: a data acquisition module 802, a data prediction module 804, a trajectory determination module 806, a speed calculation module 808 and a speed transformation module 810, wherein: the data acquisition module 802 is used to acquire image data containing a ship; the data prediction module 804 is used to input the image data into a pre-set ship detection model to obtain the position information and category information of the ship; the trajectory determination module 806 is used to obtain the navigation trajectory of the ship according to the position information and the Kalman filter algorithm; the speed calculation module 808 is used to determine the relative speed of the ship in the image coordinate system according to the navigation trajectory; and the speed transformation module 810 is used to perform homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0160] In one embodiment, the speed transformation module 810 is further used to determine the actual length of the ship in the world coordinate system based on the category information; obtain the relative length of the ship in the image coordinate system; and obtain the navigation speed based on the formula of relative length, relative speed, actual length and homography transformation.

[0161] Among them, the formula of homography transformation can include:

[0162]

[0163] Among them, V0 is the navigation speed, W0 is the actual length, V1 is the relative speed, and W1 is the relative length.

[0164] In one embodiment, the trajectory determination module 806 is further configured to determine an observation vector and a system state vector based on the position information; and to track the ship based on the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0165] In one embodiment, the trajectory determination module 806 is further used to determine the state transfer matrix based on the system state vector and a pre-set uniform motion model; input the state transfer matrix, the observation vector and the system state vector into the Kalman filter algorithm to obtain the position estimation information of the ship; and generate the navigation trajectory of the ship based on the position estimation information.

[0166] In one embodiment, the data acquisition module 802 is further configured to receive image data containing ships sent by a visible light imaging device carried by a UAV during a ship shipping patrol mission.

[0167] In one embodiment, the device for measuring the ship's sailing speed also includes a model training module (not shown), which is used to obtain historical patrol image data; classify and mark the positions of ships in the historical patrol image data to obtain training data; preprocess the training data to obtain preprocessed training data; and use the preprocessed training data to train the YOLO model to obtain a ship detection model.

[0168] In one embodiment, the model training module is further configured to perform a convolution operation on the training data using a mean template; and perform a logarithmic transformation on the data obtained after the convolution operation to obtain preprocessed training data.

[0169] Each module in the aforementioned ship speed measurement device may be implemented in whole or in part via software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0170] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9As shown. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store image data containing ships. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a method for measuring the navigation speed of a ship.

[0171] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0172] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0173] Acquiring image data containing ships;

[0174] Input the image data into the pre-set ship detection model to obtain the ship's location and category information;

[0175] Obtain the ship's navigation trajectory based on position information and Kalman filter algorithm;

[0176] Determine the relative speed of the ship in the image coordinate system based on the navigation trajectory;

[0177] The relative speed is transformed homographically according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0178] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0179] The actual length of the ship in the world coordinate system is determined according to the category information; the relative length of the ship in the image coordinate system is obtained; and the navigation speed is obtained according to the formula of relative length, relative speed, actual length and homography transformation.

[0180] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0181] The observation vector and the system state vector are determined according to the position information; the ship is tracked according to the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0182] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0183] Based on the system state vector and the pre-set uniform motion model, the state transfer matrix is ​​determined; the state transfer matrix, observation vector and system state vector are input into the Kalman filter algorithm to obtain the ship's position estimation information; based on the position estimation information, the ship's navigation trajectory is generated.

[0184] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0185] Receive image data containing ships sent by visible light imaging equipment carried by drones during ship shipping patrol missions.

[0186] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0187] Historical patrol image data is obtained; ships in the historical patrol image data are classified and marked to obtain training data; the training data is preprocessed to obtain preprocessed training data; and a YOLO model is trained using the preprocessed training data to obtain a ship detection model.

[0188] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0189] The training data is convolved using the mean template; the data obtained after the convolution operation is logarithmically transformed to obtain the preprocessed training data.

[0190] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program performs the following steps:

[0191] Acquiring image data containing ships;

[0192] Input the image data into the pre-set ship detection model to obtain the ship's location and category information;

[0193] Obtain the ship's navigation trajectory based on position information and Kalman filter algorithm;

[0194] Determine the relative speed of the ship in the image coordinate system based on the navigation trajectory;

[0195] The relative speed is transformed homographically according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0196] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0197] The actual length of the ship in the world coordinate system is determined according to the category information; the relative length of the ship in the image coordinate system is obtained; and the navigation speed is obtained according to the formula of relative length, relative speed, actual length and homography transformation.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0199] The observation vector and the system state vector are determined according to the position information; the ship is tracked according to the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0201] Based on the system state vector and the pre-set uniform motion model, the state transfer matrix is ​​determined; the state transfer matrix, observation vector and system state vector are input into the Kalman filter algorithm to obtain the ship's position estimation information; based on the position estimation information, the ship's navigation trajectory is generated.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0203] Receive image data containing ships sent by visible light imaging equipment carried by drones during ship shipping patrol missions.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0205] Historical patrol image data is obtained; ships in the historical patrol image data are classified and marked to obtain training data; the training data is preprocessed to obtain preprocessed training data; and a YOLO model is trained using the preprocessed training data to obtain a ship detection model.

[0206] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0207] The training data is convolved using the mean template; the data obtained after the convolution operation is logarithmically transformed to obtain the preprocessed training data.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0209] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0210] Acquiring image data containing ships;

[0211] Input the image data into the pre-set ship detection model to obtain the ship's location and category information;

[0212] Obtain the ship's navigation trajectory based on position information and Kalman filter algorithm;

[0213] Determine the relative speed of the ship in the image coordinate system based on the navigation trajectory;

[0214] The relative speed is transformed homographically according to the category information to obtain the navigation speed of the ship in the world coordinate system.

[0215] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0216] The actual length of the ship in the world coordinate system is determined according to the category information; the relative length of the ship in the image coordinate system is obtained; and the navigation speed is obtained according to the formula of relative length, relative speed, actual length and homography transformation.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0218] The observation vector and the system state vector are determined according to the position information; the ship is tracked according to the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

[0219] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0220] Based on the system state vector and the pre-set uniform motion model, the state transfer matrix is ​​determined; the state transfer matrix, observation vector and system state vector are input into the Kalman filter algorithm to obtain the ship's position estimation information; based on the position estimation information, the ship's navigation trajectory is generated.

[0221] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0222] Receive image data containing ships sent by visible light imaging equipment carried by drones during ship shipping patrol missions.

[0223] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0224] Historical patrol image data is obtained; ships in the historical patrol image data are classified and marked to obtain training data; the training data is preprocessed to obtain preprocessed training data; and a YOLO model is trained using the preprocessed training data to obtain a ship detection model.

[0225] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0226] The training data is convolved using the mean template; the data obtained after the convolution operation is logarithmically transformed to obtain the preprocessed training data.

[0227] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties.

[0228] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0229] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0230] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for measuring the speed of a ship, characterized in that: The measuring method comprises: Acquiring image data containing ships; Inputting the image data into a pre-set ship detection model to obtain the location information and category information of the ship; Obtaining a navigation track of the ship according to the position information and a Kalman filter algorithm; determining a relative speed of the vessel in an image coordinate system according to the navigation trajectory; Performing homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system; The performing of a homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in the world coordinate system includes: determining an actual length of the ship in a world coordinate system according to the category information; Obtaining the relative length of the ship in the image coordinate system; Obtaining the navigation speed according to the relative length, the relative speed, the actual length, and a formula for homography transformation; The formula for the homography transformation includes: ； Among them, V0 is the navigation speed, W0 is the actual length, V1 is the relative speed, and W1 is the relative length.

2. The measuring method according to claim 1, wherein Obtaining the navigation track of the ship according to the position information and the Kalman filter algorithm includes: determining an observation vector and a system state vector according to the position information; The ship is tracked according to the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

3. The measuring method according to claim 2, characterized in that Tracking the ship according to the observation vector, the system state vector, and the Kalman filter algorithm to obtain the navigation trajectory of the ship includes: Determining a state transfer matrix according to the system state vector and a preset uniform motion model; Inputting the state transfer matrix, the observation vector and the system state vector into the Kalman filter algorithm to obtain the position estimation information of the ship; A navigation track of the ship is generated according to the position estimation information.

4. The measuring method according to claim 1, wherein The acquiring of image data containing a ship includes: Receive image data containing ships sent by visible light imaging equipment carried by drones during ship shipping patrol missions.

5. The measuring method according to claim 1, wherein: Before inputting the image data into a preset ship detection model, the measurement method further includes: Obtain historical patrol image data; Classifying and position-marking ships in the historical patrol image data to obtain training data; Preprocessing the training data to obtain preprocessed training data; The preprocessed training data is used to train the YOLO model to obtain the ship detection model.

6. The method according to claim 5, characterized in that The preprocessing of the training data to obtain preprocessed training data includes: Performing a convolution operation on the training data using a mean template; The data obtained after the convolution operation is logarithmically transformed to obtain the preprocessed training data.

7. A device for measuring the speed of a ship, characterized in that: The measuring device comprises: A data acquisition module, used to acquire image data containing ships; a data prediction module, configured to input the image data into a pre-set ship detection model to obtain the position information and category information of the ship; A trajectory determination module, configured to obtain the navigation trajectory of the ship based on the position information and a Kalman filter algorithm; A speed calculation module, configured to determine the relative speed of the ship in the image coordinate system according to the navigation trajectory; a speed transformation module, configured to perform a homography transformation on the relative speed according to the category information to obtain the navigation speed of the ship in a world coordinate system; The speed conversion module is further configured to determine the actual length of the ship in the world coordinate system based on the category information; obtain the relative length of the ship in the image coordinate system; and obtain the navigation speed based on the relative length, the relative speed, the actual length, and a formula for homography transformation. The formula for the homography transformation includes: ； Among them, V0 is the navigation speed, W0 is the actual length, V1 is the relative speed, and W1 is the relative length.

8. The device according to claim 7, characterized in that The trajectory determination module is further used to determine an observation vector and a system state vector based on the position information; and to track the ship based on the observation vector, the system state vector and the Kalman filter algorithm to obtain the navigation trajectory of the ship.

9. The device according to claim 8, characterized in that The trajectory determination module is further used to determine a state transfer matrix based on the system state vector and a pre-set uniform motion model; input the state transfer matrix, the observation vector and the system state vector into the Kalman filter algorithm to obtain the position estimation information of the ship; and generate the navigation trajectory of the ship based on the position estimation information.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images