A method for measuring the length of a marine vessel with an intelligent reconnaissance device

Abstract

The application relates to the technical field of ship length measurement, and discloses a ship length measurement method of intelligent reconnaissance equipment, which comprises the following steps: collecting a ship image, distance direction data and sensor posture data of a target ship on the sea through an image acquisition sensor and a laser ranging sensor on an optoelectronic pod device; inputting the ship image into a trained target detection model to obtain a ship recognition frame; generating a ship three-dimensional point cloud according to the distance direction data; fusing the ship image, the ship three-dimensional point cloud and the ship recognition frame based on a space transformation matrix to obtain a fusion image; and inputting the fusion image and the sensor posture data into a trained deep neural network to obtain a ship length measurement result. The application can improve the accuracy of ship length measurement, enhance the robustness and real-time performance of the model, and provide effective technical support for sea ship monitoring and management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship length measurement, and more particularly to a ship length measurement method of intelligent reconnaissance equipment on the sea. BACKGROUND

[0002] The photoelectric pod is an optical-electric detection device widely used in military reconnaissance, terrain detection, and patrol security fields. Currently, the research on the traditional photoelectric pod system for measuring the length and depth of the ship on the sea mainly focuses on target recognition and coordinate conversion of multi-camera, and lacks research on high-level semantic understanding of image and video stream, thus it is difficult to realize deep analysis of the target and scene. These limitations hinder the development of the photoelectric pod system in processing complex dynamic environment and realizing intelligent processing.

[0003] With the emergence of deep learning, the convolutional neural network (CNN) in the field of sea monitoring has made a major breakthrough in automatic image interpretation. In the photoelectric pod system, there are many challenges in measuring the length of the ship using deep learning technology. First, the ship is relatively small in the image, and the available target features are limited, especially in the case of long-distance shooting. In addition, the diversity of ships on the sea, the change of different angles and scales, and the complex background conditions all increase the difficulty of detection. At the same time, there are currently methods based on three-dimensional point cloud reconstruction and database matching for accurate measurement, but these methods have high computational cost and low efficiency in practical application, which cannot meet the real-time requirements of ship length measurement, especially in the scene that requires fast measurement, and its limitations are particularly obvious. SUMMARY

[0004] The present application provides a ship length measurement method of intelligent reconnaissance equipment on the sea to overcome the low efficiency and low accuracy of the existing real-time ship length measurement detection technology.

[0005] To solve the above technical problems, the technical solution of the present application is as follows:

[0006] A ship length measurement method of intelligent reconnaissance equipment on the sea, comprising the following steps:

[0007] Collecting the ship image, distance direction data and sensor attitude data of the target ship on the sea through the image acquisition sensor and laser ranging sensor on the photoelectric pod equipment;

[0008] Inputting the ship image into the trained target detection model to obtain the ship recognition frame;

[0009] Generating the three-dimensional point cloud of the ship according to the distance direction data;

[0010] fuse the ship image, the ship three-dimensional point cloud and the ship recognition frame based on a spatial transformation matrix to obtain a fusion image;

[0011] input the fusion image and the sensor attitude data into a trained deep neural network to obtain a ship length measurement result.

[0012] Further, the present application also proposes a sea ship length measurement system of intelligent reconnaissance equipment, which applies the sea ship length measurement method proposed by the present application.

[0013] The intelligent reconnaissance equipment includes a photoelectric pod device and a laser ranging sensor, and is used for collecting a ship image, distance direction data and sensor attitude data of a target ship on the sea.

[0014] The target detection module has a trained target detection model loaded thereon, and is used for obtaining a ship recognition frame according to the input ship image.

[0015] The point cloud generation module is used for generating a ship three-dimensional point cloud according to the distance direction data.

[0016] The fusion module is used for fusing the ship image, the ship three-dimensional point cloud and the ship recognition frame based on a spatial transformation matrix to obtain a fusion image.

[0017] The measurement module has a trained deep neural network loaded thereon, and is used for outputting a ship length measurement result according to the input fusion image and sensor attitude data.

[0018] Further, the present application also proposes a device including a memory and a processor, and the memory has computer readable instructions stored therein, wherein the computer readable instructions are executed by the processor to make the processor execute the steps of the anti-interference vehicle detection method proposed by the present application.

[0019] Further, the present application also proposes a storage medium having computer readable instructions stored thereon, wherein the computer readable instructions are executed by a processor to implement the steps of the anti-interference vehicle detection method proposed by the present application.

[0020] Compared with the prior art, the technical scheme of the present application has the beneficial effects that:

[0021] The present application identifies a ship target by target detection on a ship image, combines fusion data of a three-dimensional point cloud and a ship image, and simultaneously uses sensor attitude data to assist in estimating and measuring a ship length, so as to improve the accuracy of ship length measurement, enhance the robustness and real-time performance of the model, and provide effective technical support for sea ship monitoring and management. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 Flow chart of the method for measuring the length of a marine vessel according to an embodiment of the present application.

[0023] Figure 2 Flow chart of the method for measuring the length of a marine vessel according to an embodiment of the present application.

[0024] Figure 3 Architecture diagram of the mixed receptive field convolution according to an embodiment of the present application.

[0025] Figure 4 Architecture diagram of the coordinate attention module according to an embodiment of the present application.

[0026] Figure 5 Schematic diagram of the R-Tree data structure according to an embodiment of the present application.

[0027] Figure 6 Schematic diagram of the measurement result of the length of a marine vessel according to an embodiment of the present application.

[0028] Figure 7 Architecture diagram of the system for measuring the length of a marine vessel according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The exemplary embodiments will be described in detail herein below with reference to the drawings. In the following description, the same drawings reference numbers are used to denote like or similar elements. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the present application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0031] It should be understood that although the terms "first," "second," "third," etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0032] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] Example 1

[0034] This embodiment proposes a method for measuring the length of a ship at sea using intelligent reconnaissance equipment. Figure 1 、 2 FIG. 1 is a flow chart of the method for measuring the length of a marine vessel according to the present embodiment.

[0035] The method for measuring the length of a ship at sea using intelligent reconnaissance equipment proposed in this embodiment includes the following steps:

[0036] S1, collecting images, distance and direction data, and sensor attitude data of target ships at sea through image acquisition sensors and laser ranging sensors on optoelectronic pod equipment;

[0037] S2. Input the vessel image into a trained object detection model to obtain a vessel recognition frame;

[0038] S3. generating a three-dimensional point cloud of the vessel according to the distance and direction data;

[0039] S4, fusing the vessel image, the vessel 3D point cloud, and the vessel identification frame based on a spatial transformation matrix to obtain a fused image;

[0040] S5. Input the fused image and the sensor posture data into a trained deep neural network to obtain a ship length measurement result.

[0041] In this embodiment, target detection and identification of ship targets are performed on ship images, and the fusion data of three-dimensional point clouds and ship images are combined and fused. At the same time, sensor posture data is used to assist in the estimation and measurement of ship length. This improves the accuracy of ship length measurement, enhances the robustness and real-time performance of the model, and provides effective technical support for maritime ship monitoring and management.

[0042] Exemplarily, the embodiment takes an optoelectronic pod device as the intelligent reconnaissance equipment, which integrates image acquisition sensors and laser ranging sensors for collecting the running state of the optoelectronic pod device, the optoelectronic pod angle data, the laser radar image and the visible light image collected by the optoelectronic pod camera, etc. All steps or part of steps of the method are deployed in the processor or the intelligent chip in the intelligent reconnaissance equipment, or a communication module is deployed in the intelligent reconnaissance equipment to transmit the collected data to the terminal to execute all steps or part of steps of the method.

[0043] Exemplarily, the optoelectronic pod device further includes a stable platform, an inertial sensor and the like to ensure the stability of the device during data acquisition.

[0044] Exemplarily, the image acquisition sensor on the optoelectronic pod device can optionally use an MV-CH650-90XM camera to collect visible light images. The camera has multiple advanced functions such as camera-in-mode correction, multi-ROI, correlated double sampling, combined analog and digital gain control, etc., which can effectively reduce noise and provide high-quality images with high pixel uniformity.

[0045] Exemplarily, the laser ranging sensor on the optoelectronic pod device can optionally use an LF type laser ranging module. Its characteristics include: high precision, up to 1mm; wide range, from 0.05m to 80m; fast frequency, up to 20Hz; suitable for indoor industrial measurement. The laser ranging module uses the time difference of laser flight to measure the distance, adopts the method of phase laser ranging, and converts the distance by measuring the phase delay between the laser emission and the echo signal. It can also be combined with a survey camera to generate true color point cloud data.

[0046] In the embodiment, the image acquisition sensor and the laser ranging sensor collect data at the same time and store them in the memory.

[0047] In an optional embodiment, the target detection model includes a YOLOv5 network using mixed receptive field convolution.

[0048] In the embodiment, the expression of the tth group output of the mixed receptive field convolution is:

[0049]

[0050] In the formula, represents the feature map of the tth group output, represents the feature map of the tth group input; h, w and c respectively represent the height, width and channel of the input feature map; represents the convolution kernel of the tth group; k t represents the convolution kernel size of the tth group; m is the channel multiplication factor, c tindicates the number of channels.

[0051] Exemplarily, the convolution operation of the backbone network part in the target detection model is based on the idea of mixed receptive field convolution, which improves the original convolution kernel to a combination of 1x1 and 3x3 convolution kernels. As shown in Figure 3 , which is an architecture diagram of the mixed receptive field convolution of the embodiment. Among them, the channels of the input feature map will be divided into several groups, which will be calculated with several convolution kernels of different sizes, and then the output feature maps obtained will be concatenated to obtain the final output feature map.

[0052] In the embodiment, by combining the mixed receptive field convolution to construct the target detection model, the advantages of multiple convolution kernels are fully utilized, which can better extract features. As an improved scheme of ordinary convolution layers, it will improve the accuracy and efficiency of the target detection method. Because the use of multiple convolution kernels for convolution operation leads to inconsistent outputs, the target detection model adjusts them to the same size by padding zeros at the boundary, and finally merges them through the Concat operation, thereby improving the performance of the model and the accuracy of target detection.

[0053] Further, in an optional embodiment, the target detection model further comprises a coordinate attention module; the coordinate attention module comprises, in sequence, a residual network layer, an average pooling layer, a compression layer, a spatial information encoding layer, a convolution layer and a normalization layer.

[0054] Among them, after the input feature map is processed by the residual network layer, it enters the average pooling layer to perform average pooling in the horizontal and vertical directions respectively to obtain a first one-dimensional vector and a second one-dimensional vector; the first one-dimensional vector and the second one-dimensional vector enter the compression layer to perform Concat and Conv2d operations to compress the number of channels, and then use batch normalization and nonlinear method to encode the spatial information in the vertical and horizontal directions through the spatial information encoding layer, and divide the spatial dimension into a first tensor and a second tensor in the horizontal and vertical directions; the first tensor and the second tensor are adjusted to the same number of channels as the input feature map through the Conv2d operation of the convolution layer, and then output after normalization processing and weighting processing through the normalization layer.

[0055] The embodiment effectively introduces channel information into the attention mechanism by combining the coordinate attention module, thereby improving the feature expression capability of the convolutional neural network model.

[0056] Exemplarily, as shown in Figure 4 , which is an architecture diagram of the coordinate attention module of the embodiment. Among them, H, W and C respectively represent the height, width and channel number of the input feature map, and r represents the reduction rate.

[0057] The coordinate attention module in the embodiment is divided into two parts: information embedding and attention generation. Among them, the information is embedded into the structure through the global average pooling operation. Since the ordinary average pooling compresses the global information together, it cannot obtain the position information, so the global average pooling operation is used for information embedding in the embodiment, and by processing the features in the width and height directions, the attention module can more accurately locate the target of interest in the model. The coordinate attention mechanism is decomposed by the following formula:

[0058]

[0059] Wherein, Z c is the output related to the cth channel; H, W represent the height and width of the input feature map X c respectively. The convolution kernel with a dimension of (H, 1) or (1, W) is used in the embodiment to calculate all channels in the horizontal and vertical coordinates respectively.

[0060] Further, in an optional embodiment, the target detection model selects CIoULoss (Complete Intersection over Union Loss) as the loss function of the regression frame during its training process. Its expression is:

[0061]

[0062] CIoU Loss=1–CIoU

[0063] In the formula, IoU represents the intersection over union, which is used to evaluate the overlap between the predicted value and the true value of the output frame; ρ represents the Euclidean distance, b and b gt represent the center points of the predicted frame B and the real frame B gt , c represents the diagonal length of the minimum enclosing box covering the predicted frame B and the real frame B gt ; α is a positive weighting parameter; v represents the consistency degree of the aspect ratio; represents the aspect ratio of the real frame B gt , represents the aspect ratio of the predicted frame B.

[0064] CIoU Loss is selected as the loss function of the regression frame in the embodiment, and the center point is used for distance normalization. Among them, CIoU Loss is an improved version of GIoU Loss, which considers multiple factors such as position, shape and direction, so that the model can learn the features of the target frame more comprehensively, and it is more sensitive to the position prediction of the target frame, which helps to improve the positioning accuracy.

[0065] Further optionally, the target detection model is pre-trained using an existing dataset mainly composed of visible light monitoring images and laser ranging data of marine vessels.

[0066] In a specific implementation, a dataset containing 7000 images is selected, each image having a resolution of 1920x1080, containing images of vessels of the categories of ore carrier, container ship, bulk carrier, general cargo ship, fishing boat and passenger ship, and the dataset covers vessels of various sizes, brightness and angles. Further, the dataset is divided into a training set, a validation set and a test set in a ratio of 8:1:1, and the improved YOLO detection method is trained, and the training parameters are shown in Table 1.

[0067] Table 1 YOLO training parameters

[0068] Parameter Parameter value lr0 0.01 momentum 0.929 weight_decay 0.00005 epoch 100 batch_size 16 image_size 640

[0069] After training, the traditional YOLOv3, YOLOv5 and the improved YOLOv5 of the present embodiment are compared, and the specific comparison results are shown in Table 2. The target detection model described in the present embodiment has greater performance improvement.

[0070] Table 2 Model test accuracy

[0071] Model YOLOv3 YOLOv5 This embodiment Accuracy 0.667 0.718 0.892

[0072] Finally, the trained model is deployed in an optoelectronic pod to detect vessels and store the detected data in memory.

[0073] In an optional embodiment, the distance direction data includes laser beam position, laser beam direction, laser reflection distance and reflection intensity.

[0074] Then, in the S3 step, the step of generating a vessel three-dimensional point cloud according to the distance direction data includes:

[0075] S3.1, preprocessing the distance direction data, including removing noise, error correction and filtering;

[0076] S3.2, converting the preprocessed data into a point cloud form to obtain a vessel three-dimensional point cloud.

[0077] Illustratively, in the S3.1 step of the present embodiment, optionally, Gaussian filtering is used to remove noise in the original data; and, optionally, according to the calibration parameters of the laser scanning device, the data is error corrected to eliminate the errors of the scanning instrument; and, optionally, filtering out points that are too far or too close, as well as invalid points (such as points beyond the scanning range), and only keeping valid data points.

[0078] Exemplarily, in the step S3.2, a laser beam is emitted by a laser ranging sensor on the optoelectronic pod device, and a light signal returned by the target ship on the sea is received to measure the distance of the surface of the target ship, so as to generate high-precision three-dimensional point cloud data.

[0079] Exemplarily, in the step S3.2, the target ship on the sea can also be photographed from different angles by a binocular camera carried on the optoelectronic pod device, and a coordinate conversion is performed based on an external parameter calibration matrix of the binocular camera to obtain three-dimensional point cloud data of the target ship.

[0080] In an optional embodiment, in the step S4, the ship image, the ship three-dimensional point cloud and the ship recognition frame are fused based on the spatial transformation matrix, including the following steps:

[0081] S4.1, constructing a spatial transformation matrix according to parameters of an image acquisition sensor and the laser ranging sensor carried on the optoelectronic pod device;

[0082] S4.2, extracting feature points in the ship image and the three-dimensional point cloud based on a Canny algorithm and an improved Douglas-Peucker algorithm, and calculating a spatial straight line intersection point through a RANSAC algorithm, and converting to obtain a first fused image in combination with the spatial transformation matrix;

[0083] S4.3, matching the ship recognition frame with the first fused image by using an R-Tree spatial index algorithm to obtain a second fused image.

[0084] Exemplarily, in the step S4.1, the positions of the image acquisition sensor and the laser ranging sensor are calculated, a camera internal and external parameter matrix is calculated, and a spatial transformation matrix is obtained, which is used to map each frame of point cloud data to a corresponding frame of image.

[0085] The expression of the spatial transformation matrix is:

[0086]

[0087] In the formula, Z c is an axis parallel to the optical axis in the camera coordinate system; (u, v) is a two-dimensional image pixel coordinate point; f x , f y are horizontal and vertical focal lengths of the camera respectively, and (u0, v0) is an optical center, and the four parameters constitute a camera internal parameter matrix; [R|T] is a rotation and translation matrix, which constitutes an external parameter matrix; X ω , Y ω , and Z ω represent a world coordinate system, which is defined as a radar coordinate system in the embodiment.

[0088] For example, in step S4.2, the Canny algorithm and the improved Douglas-Peucker algorithm are used to extract feature points from the optical image. The RANSAC algorithm is used to segment the calibration plate plane in the scene and calculate the plane normal vector, extracting the boundary contour of the plane plate edge points. The RANSAC algorithm is then used to fit a spatial straight line to the plane calibration plate outline. Two points are randomly selected from the target point cloud to generate a straight line. The distance between the remaining point cloud and the straight line is calculated. Points less than a set threshold are considered inliers. This process is repeated and the least squares method is used to fit the point set with the largest number of inliers as a straight line. The equation of the spatial straight line is obtained. The feature points of the plane calibration plate are then obtained, and the intersection of the spatial straight lines is calculated. By substituting the camera internal and external parameter matrix parameters into the collinearity equation, the pixel coordinates of the corresponding points in the three-dimensional point cloud in the optical image can be obtained. The RGB information of the corresponding pixels is then assigned to the point cloud to achieve image and point cloud fusion, thus obtaining a first fused image.

[0089] The RANSAC-based spatial line fitting algorithm used in this embodiment has the characteristics of high stability and can eliminate outliers.

[0090] The R-Tree used in step S4.3 of this embodiment is a dynamically balanced tree that organizes data based on the spatial positional relationships of objects. It offers strong flexibility and adjustability. The basic concept behind this tree structure is to divide spatial objects into a series of smaller rectangular regions, with each node representing a rectangular region and leaf nodes containing the actual spatial objects. The rectangular regions within a node are typically represented by a minimum bounding rectangle (MBR), which encloses the minimum extent of all objects within the node.

[0091] This embodiment uses the R-Tree algorithm to establish a data structure to describe the positional relationship between the detection box and the point cloud. Further, optionally, a two-layer R-Tree data structure is designed, where the first layer of the R-Tree stores the object detection box and the second layer stores the point cloud data points.

[0092] like Figure 5 , which is a schematic diagram of the R-Tree data structure of this embodiment, wherein R1 is the detection box of the detected target ship, and R2, R3, R4, ..., R7 are point cloud data points.

[0093] Furthermore, in an optional embodiment, the deep neural network includes three sequentially connected and hidden neural network layers; each of the neural network layers includes at least 256 neurons, and the activation function of each of the neural network layers includes a rectified linear function; and the last neural network layer is superimposed with two neurons and one sigmoid function.

[0094] The input of the deep neural network in this embodiment includes fused images and sensor pose data, where the fused images provide the main and direct information of the ship images, and the sensor pose data provides the pose information of the ship.

[0095] Further optionally, the mean square error (MSE) is used as the loss function to measure the difference between the model output and the actual length, the deep neural network is trained, and the stochastic gradient descent (SGD) is used to adjust the model parameters to minimize the loss function. The number of hidden layers is determined by experiment to be 3, the number of neurons is 256, and the learning rate is 0.01.

[0096] The output of the model in this embodiment is converted to a value in the range of 0-1 by a sigmoid function, representing the estimated value of the length of the ship. This estimated length value can be directly used or further processed and interpreted according to specific application requirements. As shown in Figure 6 , it is a schematic diagram of the sea ship length measurement result of this embodiment.

[0097] Further optionally, the trained model is evaluated and verified using an independent test data set to test its performance on unseen data, and the model parameters are adjusted or the model structure is improved according to the evaluation results to further improve the performance and generalization ability of the model, and the trained model is deployed to the optoelectronic pod to realize real-time estimation of the length of the ship.

[0098] Embodiment 2

[0099] This embodiment applies the sea ship length measurement method proposed in embodiment 1, and proposes a sea ship length measurement system of an intelligent reconnaissance equipment, as shown in Figure 7 , it is an architecture diagram of the sea ship length measurement system of this embodiment.

[0100] The sea ship length measurement system of the intelligent reconnaissance equipment proposed in this embodiment includes:

[0101] The intelligent reconnaissance equipment includes an optoelectronic pod device and a laser ranging sensor, which is used to collect ship images, distance direction data and sensor pose data of the target ship on the sea;

[0102] The target detection module has a trained target detection model loaded thereon, which is used to obtain a ship recognition box according to the input ship image;

[0103] The point cloud generation module is used to generate a three-dimensional point cloud of the ship according to the distance direction data;

[0104] The fusion module is used to fuse the ship image, the three-dimensional point cloud of the ship and the ship recognition box based on a spatial transformation matrix to obtain a fused image;

[0105] A measurement module, on which a trained deep neural network is loaded, is configured to output a ship length measurement result according to the inputted fusion image and sensor pose data.

[0106] It can be understood that the system of the embodiment corresponds to the method of the above-mentioned embodiment 1, and the optional items in the above-mentioned embodiment 1 are also applicable to the embodiment, and thus are not repeated here.

[0107] Embodiment 3

[0108] The embodiment proposes a computer device, which comprises a memory and a processor, the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to make the processor execute the steps of the ship length measurement method at sea proposed in the embodiment 1.

[0109] Embodiment 4

[0110] The embodiment proposes a storage medium, which stores computer readable instructions, wherein the computer readable instructions are executed by a processor to realize the steps of the ship length measurement method at sea proposed in the embodiment 1.

[0111] Exemplarily, the storage medium includes but is not limited to a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0112] Exemplarily, the instructions, programs, code sets or instruction sets can be implemented in a conventional programming language.

[0113] Exemplarily, the processor includes but is not limited to a smart phone, a personal computer, a server, a network device, etc., and is configured to execute all or part of the steps of the ship length measurement method at sea described in the embodiment 1.

[0114] The terms in the drawings are only used for exemplary illustration, and should not be understood as a limitation on the present application;

[0115] Obviously, the above-mentioned embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the implementation modes of the present application. Any modification, equivalent replacement and improvement made on the basis of the above-mentioned description by those skilled in the art should be included in the protection scope of the claims of the present application.

Claims

1. A method for measuring the length of a marine vessel using intelligent reconnaissance equipment, characterized in that: The following steps are involved: The image acquisition sensor and laser ranging sensor on the optoelectronic pod device collect the ship image, distance direction data and sensor attitude data of the target ship at sea; Inputting the ship image into a trained object detection model to obtain a ship recognition frame; generating a three-dimensional point cloud of the vessel according to the distance and direction data; fusing the vessel image, the vessel three-dimensional point cloud, and the vessel identification frame based on a spatial transformation matrix to obtain a fused image; Inputting the fused image and the sensor posture data into a trained deep neural network to obtain a vessel length measurement result; The target detection model includes a YOLOv5 network using mixed receptive field convolution; wherein the expression of the t-th group output of the mixed receptive field convolution is: Where, represents the feature map of the t-th group output, Represents the feature map of the tth group input; h, w, c represent the height, width and channel of the input feature map respectively; represents the convolution kernel of group t; k t represents the convolution kernel size of the tth group; m is the multiplication factor of the channel, c t Indicates the number of channels; During the training process of the target detection model, CIoU Loss is selected as the loss function of the regression box; its expression is: CIoU Loss = 1-CIoU Where IoU represents the intersection over union ratio, which is used to evaluate the degree of overlap between the predicted value and the true value of the output box; ρ represents the Euclidean distance, b and b gt Represents the predicted box B and the real box B respectively gt The center point of the predicted box B and the real box B is represented by c. gt The diagonal length of the minimum bounding box; α is a positive trade-off parameter; v represents the consistency of the aspect ratio; Represents the true box B gt The aspect ratio, Indicates the aspect ratio of the prediction box B; The step of fusing the vessel image, the vessel three-dimensional point cloud, and the vessel identification frame based on the spatial transformation matrix comprises the following steps: Constructing a spatial transformation matrix according to parameters of the image acquisition sensor and the laser ranging sensor carried by the optoelectronic pod device; Extracting feature points from the vessel image and the three-dimensional point cloud based on the Canny algorithm and the improved Douglas-Peucker algorithm, calculating spatial straight line intersections using the RANSAC algorithm, and converting the spatial transformation matrix to obtain a first fused image; Matching the vessel identification frame with the first fused image using an R-Tree spatial index algorithm to obtain a second fused image; The deep neural network includes three sequentially connected and hidden neural network layers; each of the neural network layers includes at least 256 neurons, and the activation function of each of the neural network layers includes a rectified linear function; the last neural network layer is superimposed with two neurons and a sigmoid function.

2. The method for measuring the length of a marine vessel according to claim 1, wherein: The target detection model also includes a coordinate attention module; the coordinate attention module includes a residual network layer, an average pooling layer, a compression layer, a spatial information encoding layer, a convolution layer and a normalization layer connected in sequence; Among them, after the input feature map is processed by the residual network layer, it enters the average pooling layer for average pooling in the horizontal and vertical directions respectively to obtain a first one-dimensional vector and a second one-dimensional vector; the first one-dimensional vector and the second one-dimensional vector enter the compression layer for Concat and Conv2d operations to compress the number of channels, and then the spatial information encoding layer uses batch normalization and nonlinear methods to encode the spatial information in the vertical and horizontal directions, and is divided into a first tensor and a second tensor in the horizontal and vertical directions in the spatial dimension; the first tensor and the second tensor are respectively adjusted to the same number of channels as the input feature map by performing Conv2d operations through the convolution layer, and then output after normalization and weighting processing through the normalization layer.

3. The method for measuring the length of a marine vessel according to claim 1, wherein: The distance direction data includes the laser beam position, laser beam direction, laser reflection distance and reflection intensity; and the step of generating a three-dimensional point cloud of a ship according to the distance direction data includes: Preprocessing the distance and direction data, including noise removal, error correction, and filtering; The preprocessed data is converted into point cloud form to obtain the three-dimensional point cloud of the ship.

4. A marine vessel length measurement system for intelligent reconnaissance equipment, applying the marine vessel length measurement method according to any one of claims 1 to 3, characterized in that: include: Intelligent reconnaissance equipment, including optoelectronic pods and laser ranging sensors, is used to collect ship images, distance and direction data, and sensor attitude data of target ships at sea; The object detection module is equipped with a trained object detection model and is used to obtain a vessel recognition frame based on the input vessel image; a point cloud generation module, configured to generate a three-dimensional point cloud of the vessel based on the distance and direction data; a fusion module, configured to fuse the vessel image, the vessel three-dimensional point cloud, and the vessel identification frame based on a spatial transformation matrix to obtain a fused image; a measurement module equipped with a trained deep neural network for outputting a vessel length measurement result based on the input fused image and the sensor posture data; The target detection model includes a YOLOv5 network using mixed receptive field convolution; wherein the expression of the t-th group output of the mixed receptive field convolution is: Where, represents the feature map of the t-th group output, Represents the feature map of the tth group input; h, w, c represent the height, width and channel of the input feature map respectively; represents the convolution kernel of group t; k t represents the convolution kernel size of the tth group; m is the multiplication factor of the channel, c t Indicates the number of channels; During the training process of the target detection model, CIoU Loss is selected as the loss function of the regression box; its expression is: CIoU Loss = 1-CIoU Where IoU represents the intersection over union ratio, which is used to evaluate the degree of overlap between the predicted value and the true value of the output box; ρ represents the Euclidean distance, b and b gt Represents the predicted box B and the real box B respectively gt The center point of the predicted box B and the real box B is represented by c. gt The diagonal length of the minimum bounding box; α is a positive trade-off parameter; v represents the consistency of the aspect ratio; Represents the true box B gt The aspect ratio, Indicates the aspect ratio of the prediction box B; The step of fusing the vessel image, the vessel three-dimensional point cloud, and the vessel identification frame based on the spatial transformation matrix comprises the following steps: Constructing a spatial transformation matrix according to parameters of the image acquisition sensor and the laser ranging sensor carried by the optoelectronic pod device; Extracting feature points from the vessel image and the three-dimensional point cloud based on the Canny algorithm and the improved Douglas-Peucker algorithm, calculating spatial straight line intersections using the RANSAC algorithm, and converting the spatial transformation matrix to obtain a first fused image; Matching the vessel identification frame with the first fused image using an R-Tree spatial index algorithm to obtain a second fused image; The deep neural network includes three sequentially connected and hidden neural network layers; each of the neural network layers includes at least 256 neurons, and the activation function of each of the neural network layers includes a rectified linear function; the last neural network layer is superimposed with two neurons and a sigmoid function.

5. A device comprising a memory and a processor, wherein the memory stores computer-readable instructions, characterized in that: When the computer-readable instructions are executed by the processor, the processor is caused to perform the steps of the marine vessel length measurement method according to any one of claims 1 to 3.

6. A storage medium having computer-readable instructions stored thereon, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the method for measuring the length of a marine vessel according to any one of claims 1 to 3 are implemented.

Images