Ship detection method, device and system, path planning method and electronic equipment

Abstract

The present invention proposes a ship detection method, device and system, path planning method and electronic equipment, and relates to the field of navigation technology. The ship detection method includes: obtaining point cloud data of a target ship, and converting the point cloud data into corresponding point cloud pillars; extracting features of the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales; performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale; performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale; fusing the second feature information and the third feature information to obtain fused feature information; performing regression processing on the fused feature information to obtain the detection result of the target ship.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to a ship detection method, device and system, a path planning method and electronic equipment. Background Art

[0002] At present, deep learning methods based on 3D point clouds have received widespread attention in the autonomous driving industry. The point cloud data generated by the LiDAR sensor contains the spatial position information, reflection intensity information, orientation information, and color information of the object. However, in the process of detection based on point cloud data, the 3D point cloud algorithm is limited by the depth of the feature extraction network, and the 3D point cloud algorithm is insufficient in extracting point cloud features, thereby reducing the accuracy of ship detection. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0004] To this end, a first aspect of the present invention is to propose a ship detection method.

[0005] A second aspect of the present invention is to provide a ship detection device.

[0006] The third aspect of the present invention is to provide a ship detection system.

[0007] A fourth aspect of the present invention is to propose a path planning method.

[0008] A fifth aspect of the present invention is to provide an electronic device.

[0009] In view of this, according to one aspect of the present invention, a ship detection method is proposed, which includes: acquiring point cloud data of a target ship, and converting the point cloud data into corresponding point cloud pillars; performing feature extraction on the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales, wherein the structure of the feature extraction network is in a pyramid form, and feature extraction networks at different levels correspond to first feature information of different scales respectively; performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale; performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale; fusing the second feature information and the third feature information to obtain fused feature information; performing regression processing on the fused feature information to obtain a detection result of the target ship.

[0010] The execution subject of the technical solution of the ship detection method provided by the present invention can be an electronic device, or a ship detection device, and can also be determined according to actual use requirements, which is not specifically limited here. In order to more clearly describe the ship detection method provided by the present invention, the execution subject of the ship detection method is described below as a ship detection device.

[0011] Specifically, in the ship detection method provided by the present invention, in the process of detecting the target ship, the ship detection device obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the ship detection device inputs the obtained point cloud pillar data into the feature extraction network, so as to extract features of the point cloud pillars through the feature extraction network, and obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network is in the form of a pyramid, and the feature extraction networks at different levels correspond to the first feature information of different scales. Further, the ship detection device inputs the extracted first feature information of different scales into NAM, i.e., the normalized attention module, so as to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weight of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0012] Specifically, the normalized attention module includes a channel attention submodule and a spatial attention submodule, that is, the normalized attention mechanism includes a channel attention submodule and a spatial attention submodule, the channel attention submodule corresponds to the first feature information of the first scale, and the spatial attention submodule corresponds to the first feature information of the second scale. On this basis, the ship detection device performs weight penalty processing on the first feature information of the first scale based on the channel attention submechanism to obtain the second feature information corresponding to the first scale, and performs weight penalty processing on the first feature information of the second scale based on the spatial attention submechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale. Further, the ship detection device fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0013] The ship detection method according to the present invention may also have the following additional technical features:

[0014] In some technical solutions, optionally, a weight penalty processing is performed on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale, including: determining the channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale; and performing weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain the second feature information.

[0015] In this technical solution, in the process of the ship detection device performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain the second feature information corresponding to the first scale, specifically, the ship detection device inputs the first feature information of the first scale into the channel attention sub-module to determine the channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale, thereby screening out the important feature information, i.e., foreground information, and the unimportant feature information, i.e., background information, in the first feature information of the first scale. Further, the ship detection device performs weight penalty processing on the first feature information of the first scale according to the determined channel weight information to punish and suppress the unimportant feature information, i.e., background information, and obtain the corresponding second feature information. In this way, by punishing and suppressing the unimportant feature information based on the channel attention sub-mechanism, the weight of the less significant point cloud pillars can be reduced, thereby increasing the efficiency of weight calculation while maintaining performance, enhancing the feature extraction capability of the feature extraction network for 3D point clouds, and improving the accuracy of ship detection.

[0016] In some technical solutions, optionally, the relationship between the second feature information and the first feature information is:

[0017]

[0018] Among them, B out Indicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value.

[0019] In this technical solution, in the process where the ship detection device performs weight penalty processing on the first feature information of the first scale through the channel attention submodule to obtain the second feature information corresponding to the first scale, the channel attention submodule can specifically process the first feature information according to the following formula based on the scaling factor in the batch normalization layer:

[0020]

[0021] Among them, B out is the output feature of the channel attention submodule, i.e., the second feature information, B in represents the input feature of the batch normalization layer of the channel attention submodule, i.e., the first feature information, i.e., the output feature of the previous neural network layer, μ β represents the mean of the feature information of the input channel attention submodule, σ β represents the standard deviation of the feature information of the input channel attention submodule, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value.

[0022] In some technical solutions, optionally, based on the spatial attention sub-mechanism in the normalized attention mechanism, weight penalty processing is performed on the first feature information of the second scale to obtain third feature information corresponding to the second scale, including: reducing the spatial dimension of the point cloud pillar to obtain corresponding pseudo-channel data; based on the spatial attention sub-mechanism and the pseudo-channel data, determining the spatial weight information of the first feature information corresponding to the second scale; and performing weight penalty processing on the first feature information corresponding to the second scale according to the spatial weight information to obtain the third feature information.

[0023] In this technical solution, in the process of the ship detection device performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale, specifically, the ship detection device reduces the spatial dimension of the point cloud pillar, converts the point cloud data from a two-dimensional spatial dimension to a one-dimensional spatial dimension, that is, converts the two-dimensional feature map corresponding to the point cloud pillar into one-dimensional data, and inputs the one-dimensional data into the spatial attention sub-module as pseudo-channel data. Further, the ship detection device calculates the importance of the point cloud pillars at different spatial positions based on the spatial attention sub-mechanism and pseudo-channel data, thereby obtaining the spatial weight information of the first feature information of the second scale. Further, the ship detection device performs weight penalty processing on the first feature information of the second scale according to the determined spatial weight information to punish and suppress unimportant feature information, i.e., background information, to obtain the corresponding third feature information. In this way, the importance of point cloud pillars at different spatial positions is calculated based on the spatial attention sub-mechanism, thereby punishing and suppressing unimportant feature information, and reducing the weights of less significant point cloud pillars, thereby increasing the efficiency of weight calculation while maintaining performance, enhancing the feature extraction capability of the feature extraction network for 3D point clouds, and improving the accuracy of ship detection.

[0024] In some technical solutions, optionally, the channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

[0025] In this technical solution, the above-mentioned normalized attention module includes a channel attention submodule and a spatial attention submodule. Among them, the structure of the feature extraction network is in the form of a pyramid, the channel attention submodule corresponds to the feature extraction network in the middle and lower layers of the pyramid structure, and the spatial attention submodule corresponds to the feature extraction network in the upper layer of the pyramid structure. That is, the channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

[0026] It can be understood that for the feature extraction network in the form of a pyramid, feature extraction networks at different levels are used to obtain feature information of different scales. Among them, the feature information obtained in the upper layer of the pyramid has a larger resolution and richer spatial position features, so a spatial attention submodule can be placed in the upper layer of the pyramid. The feature information in the middle and lower layers of the pyramid has been extracted, and the feature information in the middle and lower layers of the pyramid is more significant. Therefore, a channel attention submodule can be placed in the middle and lower layers of the pyramid. By punishing the feature information again through the channel attention submodule, it is possible to better distinguish important features from unimportant features and improve the accuracy of ship detection.

[0027] In some technical schemes, optionally, regression processing is performed on the fused feature information to obtain the detection result of the target ship, including: regression processing of the two-dimensional detection frame of the fused feature information along the first direction and the second direction of the point cloud pillar to obtain the first size information of the fused feature information; regression processing of the size information of the fused feature information along the third direction of the point cloud pillar to obtain the second size information of the fused feature information; determining the detection result of the target ship based on the fused feature information, the first size information and the second size information; wherein the first direction is the horizontal axis direction of the coordinates of the point cloud pillar, the second direction is the vertical axis direction of the coordinates of the point cloud pillar, and the third direction is the vertical axis direction of the coordinates of the point cloud pillar.

[0028] In this technical solution, when the ship detection device performs regression processing on the obtained fused feature information to determine the detection result of the target ship, the ship detection device regresses the final point cloud feature information, i.e., the fused feature information, along the first direction and the second direction of the point cloud pillar, i.e., the prior frame, i.e., the two-dimensional detection frame, based on the target detection algorithm such as the SSD (Single Shot Multi-Box-Detector) algorithm, to obtain the first position information and the first size information of the fused feature information in the first direction and the second direction. The first direction is the horizontal axis direction of the coordinates of the point cloud pillar, i.e., the X-axis direction, and the second direction is the vertical axis direction of the coordinates of the point cloud pillar, i.e., the Y-axis direction. Further, the ship detection device regresses the size information of the fused feature information along the third direction of the point cloud pillar, to obtain the second position information and the second size information of the fused feature information in the third direction. The third direction is the vertical axis direction of the coordinates of the point cloud pillar, i.e., the Z-axis direction. Furthermore, the ship detection device splices the first position information, the second position information, the first size information and the second size information of the fused feature information to obtain a three-dimensional detection frame of the target ship, and processes the fused feature information according to the three-dimensional detection frame to obtain a detection result of the target ship. In this way, the position and size information of the fused feature information in different directions are regressed to obtain a three-dimensional detection frame of the target ship, and then the detection result of the target ship is obtained, thereby ensuring the accuracy of the obtained detection result.

[0029] According to a second aspect of the present invention, a ship detection device is proposed, which includes: a processing unit, which is used to obtain point cloud data of a target ship and convert the point cloud data into corresponding point cloud pillars; the processing unit is also used to extract features from the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales, wherein the structure of the feature extraction network is in a pyramid form, and feature extraction networks at different levels correspond to first feature information of different scales respectively; the processing unit is also used to perform weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale; the processing unit is also used to perform weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale; the processing unit is also used to fuse the second feature information and the third feature information to obtain fused feature information; the processing unit is also used to perform regression processing on the fused feature information to obtain a detection result of the target ship.

[0030] Specifically, the ship detection device provided by the present invention includes a processing unit. In the process of detecting the target ship, the processing unit obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the processing unit inputs the obtained point cloud pillar data into the feature extraction network to extract features of the point cloud pillars through the feature extraction network to obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network is in the form of a pyramid, and the feature extraction networks at different levels correspond to the first feature information of different scales. Further, the processing unit inputs the extracted first feature information of different scales into the NAM, i.e., the normalized attention module, to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weights of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0031] Specifically, the normalized attention module includes a channel attention submodule and a spatial attention submodule, that is, the normalized attention mechanism includes a channel attention submodule and a spatial attention submodule, the channel attention submodule corresponds to the first feature information of the first scale, and the spatial attention submodule corresponds to the first feature information of the second scale. On this basis, the processing unit performs weight penalty processing on the first feature information of the first scale based on the channel attention submechanism to obtain the second feature information corresponding to the first scale, and performs weight penalty processing on the first feature information of the second scale based on the spatial attention submechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale. Further, the processing unit fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0032] According to a third aspect of the present invention, a ship detection system is proposed, comprising: a processing module, used to obtain point cloud data of a target ship and convert the point cloud data into corresponding point cloud pillars; a feature extraction network, used to perform feature extraction on the point cloud pillars to obtain first feature information of the target ship at different scales, the structure of the feature extraction network is in a pyramid form, and feature extraction networks at different levels correspond to first feature information of different scales respectively; a normalized attention module, the normalized attention module comprising: a channel attention submodule, used to perform weight penalty processing on the first feature information of the first scale to obtain second feature information corresponding to the first scale; and a spatial attention submodule, used to perform weight penalty processing on the first feature information of the second scale to obtain third feature information corresponding to the second scale; the processing module is also used to fuse the second feature information and the third feature information to obtain fused feature information, and perform regression processing on the fused feature information to obtain a detection result of the target ship.

[0033] Specifically, the ship detection system provided by the present invention includes a processing module, a feature extraction network and a normalized attention module, and the normalized attention module includes a channel attention submodule and a spatial attention submodule. In the process of detecting the target ship, the processing module obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the processing module inputs the obtained point cloud pillar data into the feature extraction network, and the feature extraction network extracts features from the point cloud pillars to obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network is in the form of a pyramid, and the feature extraction networks at different levels correspond to the first feature information of different scales. Further, the feature extraction network inputs the extracted first feature information of different scales into NAM, i.e., the normalized attention module, so as to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weight of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0034] Specifically, the channel attention submodule performs weighted penalty processing on the first feature information of the first scale to obtain the second feature information corresponding to the first scale, and the spatial attention submodule performs weighted penalty processing on the first feature information of the second scale to obtain the third feature information corresponding to the second scale. Furthermore, the processing module fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0035] According to a fourth aspect of the present invention, a path planning method is proposed, the method comprising: detecting a target ship according to the ship detection method in any of the above technical solutions to obtain a detection result of the target ship; and determining a navigation path according to the detection result.

[0036] The path planning method provided by the present invention detects the target ship according to the ship detection method in any of the above technical solutions, obtains the detection result of the target ship, and then determines the navigation path according to the detection result. Therefore, the path planning method provided by the present invention has all the beneficial effects of the ship detection method in any of the technical solutions in the first aspect above, which will not be repeated here.

[0037] The detection result of the target ship may specifically include the location information and size information of the target ship, etc., which are not specifically limited here.

[0038] According to the fifth aspect of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the ship detection method in any of the above technical solutions are implemented. Therefore, the electronic device provided in the fifth aspect of the present invention has all the beneficial effects of the ship detection method in any of the technical solutions in the first aspect, which will not be described in detail here.

[0039] Additional aspects and advantages of the present invention will become apparent in the following description or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0041] Figure 1 One of the flow charts of the ship detection method according to the embodiment of the present invention is shown;

[0042] Figure 2 The second flowchart of the ship detection method according to the embodiment of the present invention is shown;

[0043] Figure 3 A schematic diagram of the network structure of the channel attention submodule of an embodiment of the present invention is shown;

[0044] Figure 4 A schematic diagram of the network structure of the spatial attention submodule of an embodiment of the present invention is shown;

[0045] Figure 5 A structural block diagram of a ship detection device according to an embodiment of the present invention is shown;

[0046] Figure 6A structural block diagram of a ship detection system according to an embodiment of the present invention is shown;

[0047] Figure 7 A schematic diagram showing a flow chart of a path planning method according to an embodiment of the present invention is shown;

[0048] Figure 8 A structural block diagram of a path planning device according to an embodiment of the present invention is shown;

[0049] Fig. 9 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0050] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0052] Combine the following Figures 1 to 9 , the ship detection method, device and system, path planning method and electronic device provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0053] In one embodiment of the present invention, Figure 1 As shown, the ship detection method may specifically include the following steps 102 to 112:

[0054] Step 102, obtaining point cloud data of the target ship, and converting the point cloud data into corresponding point cloud pillars;

[0055] Step 104, extracting features from the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales;

[0056] Step 106, performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale;

[0057] Step 108, performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale;

[0058] Step 110, fusing the second feature information and the third feature information to obtain fused feature information;

[0059] Step 112, performing regression processing on the fused feature information to obtain a detection result of the target ship;

[0060] Among them, the structure of the feature extraction network is in the form of a pyramid, and feature extraction networks at different levels correspond to first feature information of different scales.

[0061] The execution subject of the technical solution of the ship detection method provided by the present invention can be an electronic device, or a ship detection device, and can also be determined according to actual use requirements, which is not specifically limited here. In order to more clearly describe the ship detection method provided by the present invention, the execution subject of the ship detection method is described below as a ship detection device.

[0062] The ship detection method provided by the present invention introduces NAM (Normalization-based Attention Module) into the field of point cloud ship detection to process point cloud pillars of unstructured nature, and applies sparse weight penalty on the attention module to reduce the weights of less significant point cloud pillars, thereby increasing the efficiency of weight calculation while maintaining performance. It can enhance the feature extraction capability of the feature extraction network for 3D point clouds by suppressing insignificant features through penalty without increasing the number of layers and network parameters of the feature extraction network.

[0063] Specifically, in the ship detection method provided by the present invention, in the process of detecting the target ship, the ship detection device obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the ship detection device inputs the obtained point cloud pillar data into the feature extraction network, so as to extract features of the point cloud pillars through the feature extraction network, and obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network is in the form of a pyramid, and the feature extraction networks at different levels correspond to the first feature information of different scales. Further, the ship detection device inputs the extracted first feature information of different scales into NAM, i.e., the normalized attention module, so as to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weight of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0064] Specifically, the normalized attention module includes a channel attention submodule and a spatial attention submodule, that is, the normalized attention mechanism includes a channel attention submodule and a spatial attention submodule, the channel attention submodule corresponds to the first feature information of the first scale, and the spatial attention submodule corresponds to the first feature information of the second scale. On this basis, the ship detection device performs weight penalty processing on the first feature information of the first scale based on the channel attention submechanism to obtain the second feature information corresponding to the first scale, and performs weight penalty processing on the first feature information of the second scale based on the spatial attention submechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale. Further, the ship detection device fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0065] Among them, the point cloud data of the above-mentioned target ship can be specifically generated by laser sensors, radar sensors and other devices. The point cloud data of the target ship can reflect the spatial position information, reflection intensity information, orientation information and color information of the target ship, and can be used in navigation, automatic driving and other fields.

[0066] Furthermore, point cloud data is a kind of unstructured data, and the points in the point cloud data are unevenly distributed. Therefore, before extracting point cloud features through the feature extraction network, the point cloud data needs to be represented as a data form suitable for the feature extraction network. The current main point cloud representation methods include voxel-based point cloud representation methods and pillar-based point cloud representation methods.

[0067] Among them, the voxel-based point cloud representation method is to assign each point in the point cloud data to a specific volume element, and then let multiple points fall into the same volume element according to the roughness of the voxel grid, and finally extract local features according to the points in each voxel. Furthermore, the pillar-based point cloud representation method is an extension of the voxel-based point cloud representation method. Specifically, the pillar-based point cloud representation method is to stretch the height of each voxel to the same height range as the point cloud data on the basis of voxels, that is, the point cloud is no longer aggregated into cubic volume elements, but aggregated into vertical pillars rising from the ground, namely point cloud pillars.

[0068] Furthermore, it can be understood that for the feature extraction network, too deep a network layer will cause the number of parameters of the feature extraction network to be huge, thereby reducing the detection speed of ship detection, while too shallow a network layer will make the detection accuracy of ship detection insufficient, thereby reducing the feature extraction ability of the feature extraction network. Therefore, in order to balance the detection accuracy and detection speed, the feature extraction network needs to use an appropriate convolution layer for feature extraction.

[0069] On this basis, the ship detection method provided by the present invention can specifically use a 16-layer residual network as a feature extraction network. At the same time, in order to promote the flow of information between the various levels in the feature extraction network, the present invention builds the above residual network into the shape of a feature pyramid. Among them, each residual network is composed of a 3×3 convolutional layer, a BN (Batch Normalization) layer, a ReLU activation function, a 3×3 convolutional layer, and a BN layer in sequence. Furthermore, residual networks of different levels are used to extract first feature information of different scales.

[0070] Furthermore, in the actual application process, the above-mentioned NAM, i.e., the normalized attention module, is added after the second BN layer of each residual network, that is, the normalized attention module is added at the end of each residual network, and the input features of the normalized attention module are the output features of the corresponding residual network. In this way, when the first feature information is processed based on the normalized attention mechanism, the feature information can be avoided from being missed.

[0071] Among them, NAM, the normalized attention module, takes into account the contribution factor of the convolution kernel weight, which can further suppress insignificant features, thereby highlighting the foreground that needs attention, that is, important features. Among them, NAM uses the weight coefficient of the BN layer to indicate the importance of the weight.

[0072] Furthermore, the normalized attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule corresponds to the residual network of the middle and lower layers of the pyramid structure, and the spatial attention submodule corresponds to the residual network of the upper layer of the pyramid structure.

[0073] In some embodiments of the present invention, optionally, the above step 106 may specifically include the following steps 106a and 106b:

[0074] Step 106a, determining channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale;

[0075] Step 1046: Perform weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain second feature information.

[0076] In this embodiment, in the process of the ship detection device performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain the second feature information corresponding to the first scale, specifically, the ship detection device inputs the first feature information of the first scale into the channel attention sub-module to determine the channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale, thereby screening out the important feature information, i.e., foreground information, and the unimportant feature information, i.e., background information, in the first feature information of the first scale. Further, the ship detection device performs weight penalty processing on the first feature information of the first scale according to the determined channel weight information to punish and suppress the unimportant feature information, i.e., background information, and obtain the corresponding second feature information. In this way, by punishing and suppressing the unimportant feature information based on the channel attention sub-mechanism, the weight of the less significant point cloud pillars can be reduced, thereby increasing the efficiency of weight calculation while maintaining performance, enhancing the feature extraction capability of the feature extraction network for 3D point clouds, and improving the accuracy of ship detection.

[0077] Among them, in the field of two-dimensional images, the channel attention submodule is used to measure the importance of image pixels. In the ship detection method provided by the present invention, NAM is introduced into the field of point cloud ship detection. The channel attention submodule is essentially used to measure the importance between different point clouds. Therefore, the above-mentioned process of performing weight penalty processing on the first feature information based on the channel attention sub-mechanism can also be called a point cloud normalization process.

[0078] Furthermore, the network structure of the channel attention submodule is as follows Figure 3 As shown, the sigmoid function is a common S-shaped function, which is also called an S-shaped growth curve. The sigmoid function is used as an activation function of a neural network to map variables between 0 and 1. Further, the relationship between the weight ω and the radiation transformation parameter γ is:

[0079]

[0080] Among them, i and j represent the order, ω i That is, it represents the i-th weight, γ # That is, it represents the jth affine transformation parameter, γ i That is, it represents the i-th affine transformation parameter.

[0081] In some embodiments of the present invention, optionally, the relationship between the second feature information and the first feature information is:

[0082]

[0083] Among them, B outIndicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value.

[0084] In this embodiment, in the process of the ship detection device performing weight penalty processing on the first feature information of the first scale through the channel attention submodule to obtain the second feature information corresponding to the first scale, the channel attention submodule can specifically process the first feature information according to the following formula based on the scaling factor in the batch normalization layer:

[0085]

[0086] Among them, B out is the output feature of the channel attention submodule, i.e., the second feature information, B in represents the input feature of the batch normalization layer of the channel attention submodule, i.e., the first feature information, i.e., the output feature of the previous neural network layer, μ β represents the mean of the feature information of the input channel attention submodule, σ β represents the standard deviation of the feature information of the input channel attention submodule, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value.

[0087] In some embodiments of the present invention, optionally, the above step 108 may specifically include the following steps 108a to 108c:

[0088] Step 108a, reducing the spatial dimension of the point cloud pillar to obtain corresponding pseudo channel data;

[0089] Step 108b, determining spatial weight information of the first feature information corresponding to the second scale based on the spatial attention sub-mechanism and the pseudo-channel data;

[0090] Step 108c: Perform weight penalty processing on the first feature information corresponding to the second scale according to the spatial weight information to obtain third feature information.

[0091] In this embodiment, in the process of the ship detection device performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale, specifically, the ship detection device reduces the spatial dimension of the point cloud pillar, converts the point cloud data from a two-dimensional spatial dimension to a one-dimensional spatial dimension, that is, converts the two-dimensional feature map corresponding to the point cloud pillar into one-dimensional data, and inputs the one-dimensional data into the spatial attention sub-module as pseudo-channel data. Further, the ship detection device calculates the importance of the point cloud pillars at different spatial positions based on the spatial attention sub-mechanism and pseudo-channel data, thereby obtaining the spatial weight information of the first feature information of the second scale. Further, the ship detection device performs weight penalty processing on the first feature information of the second scale according to the determined spatial weight information to punish and suppress unimportant feature information, i.e., background information, to obtain the corresponding third feature information. In this way, the importance of point cloud pillars at different spatial positions is calculated based on the spatial attention sub-mechanism, thereby punishing and suppressing unimportant feature information, and reducing the weights of less significant point cloud pillars, thereby increasing the efficiency of weight calculation while maintaining performance, enhancing the feature extraction capability of the feature extraction network for 3D point clouds, and improving the accuracy of ship detection.

[0092] Among them, in the ship detection method provided by the present invention, the above-mentioned spatial attention submodule is used to calculate the importance of point cloud pillars at different spatial positions. However, due to the non-structural nature of the point cloud, the ship detection device cannot directly compare the importance of different point cloud pillars. Therefore, in the ship detection method provided by the present invention, before calculating the importance of point cloud pillars at different spatial positions, the ship detection device first reduces the spatial dimension of the point cloud pillar to convert the point cloud data from a two-dimensional spatial dimension to a one-dimensional spatial dimension, and converts the two-dimensional feature map corresponding to the point cloud pillar into one-dimensional data.

[0093] Specifically, since the point cloud data is in the form of a point cloud pillar, the Z axis of the point cloud pillar can be ignored, and the ship detection device flattens the point cloud pillar in the X-axis direction and the Y-axis direction, that is, converts the two-dimensional matrix corresponding to the two-dimensional feature map into a one-dimensional array to place all elements in the two-dimensional matrix in the same row. For example, for a two-dimensional matrix with 3 rows and 4 columns, it is converted into a row matrix with 1 row and 12 columns.

[0094] Furthermore, the network structure of the spatial attention submodule is as follows Figure 4 As shown. Among them, the relationship between the weight ω and the network parameter λ is:

[0095]

[0096] Among them, i and j represent the order, ω i That is, it represents the i-th weight, λ #That is, it represents the jth network parameter, λ i That is, it represents the i-th network parameter.

[0097] In some embodiments of the present invention, optionally, the channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

[0098] In this embodiment, the above-mentioned normalized attention module includes a channel attention submodule and a spatial attention submodule. Among them, the structure of the feature extraction network is in the form of a pyramid, the channel attention submodule corresponds to the feature extraction network of the middle and lower layers of the pyramid structure, and the spatial attention submodule corresponds to the feature extraction network of the upper layer of the pyramid structure. That is, the channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

[0099] It can be understood that for the feature extraction network in the form of a pyramid, feature extraction networks at different levels are used to obtain feature information of different scales. Among them, the feature information obtained in the upper layer of the pyramid has a larger resolution and richer spatial position features, so a spatial attention submodule can be placed in the upper layer of the pyramid. The feature information in the middle and lower layers of the pyramid has been extracted, and the feature information in the middle and lower layers of the pyramid is more significant. Therefore, a channel attention submodule can be placed in the middle and lower layers of the pyramid. By punishing the feature information again through the channel attention submodule, it is possible to better distinguish important features from unimportant features and improve the accuracy of ship detection.

[0100] In some embodiments of the present invention, optionally, the above step 112 may specifically include the following steps 112a to 112c:

[0101] Step 112a, performing regression processing on the two-dimensional detection frame of the fused feature information along the first direction and the second direction of the point cloud pillar to obtain first size information of the fused feature information;

[0102] Step 112b, performing regression processing on the size information of the fused feature information along the third direction of the point cloud pillar to obtain second size information of the fused feature information;

[0103] Step 112c, determining a detection result of the target ship according to the fused feature information, the first size information and the second size information;

[0104] Among them, the first direction is the horizontal axis direction of the coordinates of the point cloud pillar, the second direction is the vertical axis direction of the coordinates of the point cloud pillar, and the third direction is the vertical axis direction of the coordinates of the point cloud pillar.

[0105] In this embodiment, in the process of the ship detection device regressing the obtained fused feature information to determine the detection result of the target ship, the ship detection device regresses the final point cloud feature information, i.e., the fused feature information, along the first direction and the second direction of the point cloud pillar, i.e., the prior frame, i.e., the two-dimensional detection frame, based on the target detection algorithm such as the SSD (Single Shot Multi-Box-Detector) algorithm, to obtain the first position information and the first size information of the fused feature information in the first direction and the second direction. The first direction is the horizontal axis direction of the coordinates of the point cloud pillar, i.e., the X-axis direction, and the second direction is the vertical axis direction of the coordinates of the point cloud pillar, i.e., the Y-axis direction. Further, the ship detection device regresses the size information of the fused feature information along the third direction of the point cloud pillar, to obtain the second position information and the second size information of the fused feature information in the third direction. The third direction is the vertical axis direction of the coordinates of the point cloud pillar, i.e., the Z-axis direction. Furthermore, the ship detection device splices the first position information, the second position information, the first size information and the second size information of the fused feature information to obtain a three-dimensional detection frame of the target ship, and processes the fused feature information according to the three-dimensional detection frame to obtain a detection result of the target ship. In this way, the position and size information of the fused feature information in different directions are regressed to obtain a three-dimensional detection frame of the target ship, and then the detection result of the target ship is obtained, thereby ensuring the accuracy of the obtained detection result.

[0106] Among them, in the process of the ship detection device regressing the prior frame, i.e., the two-dimensional detection frame, of the fused feature information along the first direction and the second direction of the point cloud pillar, the ship detection device calculates the position offset of the prior frame, and determines the final position and size of the prior frame based on the position offset.

[0107] Furthermore, in the process of the ship detection device regressing the size information of the fused feature information along the third direction of the point cloud pillar, the ship detection device determines the true value and prior value of the size information of the fused feature information along the third direction of the point cloud pillar, and calculates the difference between the true value and the prior value, and then based on the calculated difference, determines the position and size of the fused feature information along the third direction of the point cloud pillar through the loss function and back propagation.

[0108] In summary, if Figure 2 As shown, the ship detection method provided by the present invention may specifically include the following steps 202 to 210:

[0109] Step 202, pillar representation of point cloud data;

[0110] Step 204, using a residual network with a NAM channel submodule to perform feature extraction;

[0111] Step 206, using a residual network with a NAM spatial submodule to perform feature extraction;

[0112] Step 208, fusing feature extraction information of different scales;

[0113] Step 210, using the SSD algorithm to perform prior box regression and regress the height information separately.

[0114] In one embodiment of the present invention, a ship detection device is also provided. Figure 5 As shown, Figure 5 The structural block diagram of the ship detection device 300 according to the embodiment of the present invention is shown. The ship detection device 300 may specifically include the following processing unit 302:

[0115] The processing unit 302 is used to obtain the point cloud data of the target ship and convert the point cloud data into corresponding point cloud pillars;

[0116] The processing unit 302 is further used to extract features from the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales, wherein the structure of the feature extraction network is in a pyramid form, and feature extraction networks at different levels correspond to first feature information at different scales;

[0117] The processing unit 302 is further configured to perform weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale;

[0118] The processing unit 302 is further configured to perform weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale;

[0119] The processing unit 302 is further used to fuse the second feature information and the third feature information to obtain fused feature information;

[0120] The processing unit 302 is further used to perform regression processing on the fused feature information to obtain a detection result of the target ship.

[0121] The ship detection device 300 provided by the embodiment of the present invention introduces NAM into the field of point cloud ship detection to process point cloud pillars of unstructured nature, and applies sparse weight penalty on the attention module to reduce the weights of less significant point cloud pillars, thereby increasing the efficiency of weight calculation while maintaining performance. It can enhance the feature extraction capability of the feature extraction network for 3D point clouds by penalizing and suppressing insignificant features without increasing the number of layers and network parameters of the feature extraction network.

[0122] Specifically, the ship detection device 300 provided by the present invention includes a processing unit 302. In the process of detecting the target ship, the processing unit 302 obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the processing unit 302 inputs the obtained point cloud pillar data into the feature extraction network, so as to extract features of the point cloud pillars through the feature extraction network, and obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network is in the form of a pyramid, and the feature extraction networks at different levels correspond to the first feature information of different scales. Further, the processing unit 302 inputs the extracted first feature information of different scales into the NAM, i.e., the normalized attention module, so as to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weight of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0123] Specifically, the normalized attention module includes a channel attention submodule and a spatial attention submodule, that is, the normalized attention mechanism includes a channel attention submodule and a spatial attention submodule, the channel attention submodule corresponds to the first feature information of the first scale, and the spatial attention submodule corresponds to the first feature information of the second scale. On this basis, the processing unit 302 performs weight penalty processing on the first feature information of the first scale based on the channel attention submechanism to obtain the second feature information corresponding to the first scale, and performs weight penalty processing on the first feature information of the second scale based on the spatial attention submechanism in the normalized attention mechanism to obtain the third feature information corresponding to the second scale. Further, the processing unit 302 fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0124] Among them, the point cloud data of the above-mentioned target ship can be specifically generated by laser sensors, radar sensors and other devices. The point cloud data of the target ship can reflect the spatial position information, reflection intensity information, orientation information and color information of the target ship, and can be used in navigation, automatic driving and other fields.

[0125] Furthermore, point cloud data is a kind of unstructured data, and the points in the point cloud data are unevenly distributed. Therefore, before extracting point cloud features through the feature extraction network, the point cloud data needs to be represented as a data form suitable for the feature extraction network. The current main point cloud representation methods include voxel-based point cloud representation methods and pillar-based point cloud representation methods.

[0126] Among them, the voxel-based point cloud representation method is to assign each point in the point cloud data to a specific volume element, and then let multiple points fall into the same volume element according to the roughness of the voxel grid, and finally extract local features according to the points in each voxel. Furthermore, the pillar-based point cloud representation method is an extension of the voxel-based point cloud representation method. Specifically, the pillar-based point cloud representation method is to stretch the height of each voxel to the same height range as the point cloud data on the basis of voxels, that is, the point cloud is no longer aggregated into cubic volume elements, but aggregated into vertical pillars rising from the ground, namely point cloud pillars.

[0127] Furthermore, it can be understood that for the feature extraction network, too deep a network layer will cause the number of parameters of the feature extraction network to be huge, thereby reducing the detection speed of ship detection, while too shallow a network layer will make the detection accuracy of ship detection insufficient, thereby reducing the feature extraction ability of the feature extraction network. Therefore, in order to balance the detection accuracy and detection speed, the feature extraction network needs to use an appropriate convolution layer for feature extraction.

[0128] On this basis, the ship detection device 300 provided by the present invention can specifically use a 16-layer residual network as a feature extraction network. At the same time, in order to promote the flow of information between the various levels in the feature extraction network, the present invention builds the above residual network into the shape of a feature pyramid. Among them, each residual network is composed of a 3×3 convolution layer, a BN layer, a ReLU activation function, a 3×3 convolution layer, and a BN layer in sequence. Furthermore, residual networks of different levels are used to extract first feature information of different scales.

[0129] Furthermore, in the actual application process, the above-mentioned NAM, i.e., the normalized attention module, is added after the second BN layer of each residual network, that is, the normalized attention module is added at the end of each residual network, and the input features of the normalized attention module are the output features of the corresponding residual network. In this way, when the first feature information is processed based on the normalized attention mechanism, the feature information can be avoided from being missed.

[0130] Among them, NAM, the normalized attention module, takes into account the contribution factor of the convolution kernel weight, which can further suppress insignificant features, thereby highlighting the foreground that needs attention, that is, important features. Among them, NAM uses the weight coefficient of the BN layer to indicate the importance of the weight.

[0131] Furthermore, the normalized attention module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule corresponds to the residual network of the middle and lower layers of the pyramid structure, and the spatial attention submodule corresponds to the residual network of the upper layer of the pyramid structure.

[0132] In some embodiments of the present invention, optionally, the processing unit 302 is specifically used to: determine the channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale; perform weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain second feature information.

[0133] In some embodiments of the present invention, optionally, the relationship between the second feature information and the first feature information is:

[0134]

[0135] Among them, B out Indicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value.

[0136] In some embodiments of the present invention, optionally, the processing unit 302 is specifically used to: reduce the spatial dimension of the point cloud pillar to obtain corresponding pseudo-channel data; determine the spatial weight information of the first feature information corresponding to the second scale based on the spatial attention sub-mechanism and the pseudo-channel data; perform weight penalty processing on the first feature information corresponding to the second scale according to the spatial weight information to obtain third feature information.

[0137] In some embodiments of the present invention, optionally, the channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

[0138] In some embodiments of the present invention, optionally, the processing unit 302 is specifically used to: perform regression processing on the two-dimensional detection frame of the fused feature information along the first direction and the second direction of the point cloud pillar to obtain the first size information of the fused feature information; perform regression processing on the size information of the fused feature information along the third direction of the point cloud pillar to obtain the second size information of the fused feature information; determine the detection result of the target ship based on the fused feature information, the first size information and the second size information; wherein the first direction is the horizontal axis direction of the coordinates of the point cloud pillar, the second direction is the vertical axis direction of the coordinates of the point cloud pillar, and the third direction is the vertical axis direction of the coordinates of the point cloud pillar.

[0139] In one embodiment of the present invention, a ship detection system is also provided. Figure 6 As shown, Figure 6The structural block diagram of the ship detection system 400 according to the embodiment of the present invention is shown. The ship detection system 400 may specifically include a processing module 402, a feature extraction network 404 and a normalized attention module 406, wherein the normalized attention module 406 includes a channel attention submodule 408 and a spatial attention submodule 410.

[0140] In the process of detecting the target ship, the processing module 402 obtains the point cloud data of the target ship to be detected, and converts the obtained point cloud data into corresponding point cloud pillars. Further, the processing module 402 inputs the obtained point cloud pillar data into the feature extraction network 404, and the feature extraction network 404 extracts features from the point cloud pillars to obtain the first feature information of the target ship at different scales. Among them, the structure of the feature extraction network 404 is in the form of a pyramid, and the feature extraction networks 404 at different levels correspond to the first feature information of different scales. Further, the feature extraction network 404 inputs the extracted first feature information of different scales into the NAM, i.e., the normalized attention module 406, so as to perform weight penalty processing on the above-mentioned first feature information of different scales based on the normalized attention mechanism, thereby reducing the weights of the less significant point cloud pillars and suppressing the unimportant first feature information.

[0141] Specifically, the channel attention submodule 408 performs weight penalty processing on the first feature information of the first scale to obtain the second feature information corresponding to the first scale, and the spatial attention submodule 410 performs weight penalty processing on the first feature information of the second scale to obtain the third feature information corresponding to the second scale. Further, the processing module 402 fuses the second feature information and the third feature information obtained above to obtain fused feature information, and then performs regression processing on the obtained fused feature information to obtain the detection result of the target ship. In this way, combining the feature extraction network and the normalized attention mechanism to extract point cloud features can enhance the feature extraction capability of the feature extraction network for 3D point clouds without increasing the number of layers and network parameters of the feature extraction network, thereby improving the accuracy of ship detection.

[0142] In one embodiment of the present invention, a path planning method is also proposed. Figure 7 As shown, the path planning method may specifically include the following steps 702 and 704:

[0143] Step 702, detecting the target ship according to the ship detection method to obtain the detection result of the target ship;

[0144] Step 704, determining a navigation path according to the detection result.

[0145] The path planning method provided by the present invention detects the target ship according to the ship detection method in any of the above embodiments, obtains the detection result of the target ship, and then determines the navigation path according to the detection result. Therefore, the path planning method provided by the present invention has all the beneficial effects of the ship detection method in any of the above embodiments of the first aspect, which will not be described in detail here.

[0146] The detection result of the target ship may specifically include the location information and size information of the target ship, etc., which are not specifically limited here.

[0147] In one embodiment of the present invention, a path planning device is also provided. Figure 8 As shown, Figure 8 FIG. 5 is a block diagram showing a structure of a path planning device 500 according to an embodiment of the present invention. Specifically, the path planning device 500 may include the following processing unit 502:

[0148] The processing unit 502 is used to detect the target ship according to the ship detection method in any of the above embodiments to obtain a detection result of the target ship;

[0149] The processing unit 502 is further configured to determine a navigation path according to the detection result.

[0150] The path planning device 500 provided by the present invention detects the target ship according to the ship detection method in any of the above embodiments, obtains the detection result of the target ship, and then determines the navigation path according to the detection result. Therefore, the path planning device 500 provided by the present invention has all the beneficial effects of the ship detection method in any of the above embodiments of the first aspect, which will not be described in detail here.

[0151] In one embodiment of the present invention, an electronic device is also provided. Fig. 9 As shown, Fig. 9 FIG. 6 is a block diagram of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 includes:

[0152] A memory 602, where programs or instructions are stored;

[0153] Processor 604, when executing the above program or instruction, implements the steps of the ship detection method in any of the above embodiments.

[0154] The electronic device 600 provided in this embodiment includes a memory 602 and a processor 604. When the program or instructions in the memory 602 are executed by the processor 604, the steps of the ship detection method in any of the above embodiments are implemented. Therefore, the electronic device 600 has all the beneficial effects of the ship detection method in any of the above embodiments, which will not be repeated here.

[0155] Specifically, the memory 602 and the processor 604 may be connected via a bus or other means. The processor 604 may include one or more processing units, and the processor 604 may be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and other chips.

[0156] In one embodiment of the present invention, a readable storage medium is also provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the ship detection method in any of the above embodiments are implemented.

[0157] The readable storage medium provided by the embodiment of the present invention can implement the steps of the ship detection method in any of the above embodiments when the program or instruction stored therein is executed by the processor. Therefore, the readable storage medium has all the beneficial effects of the ship detection method in any of the above embodiments, which will not be described in detail here.

[0158] Specifically, the above-mentioned readable storage medium may include any medium capable of storing or transmitting information. Examples of readable storage media include electronic circuits, semiconductor memory devices, read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), flash memory, erasable ROM (EROM), magnetic tape, floppy disk, optical disk, hard disk, optical fiber medium, radio frequency (RF) link, optical data storage device, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0159] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance unless otherwise clearly specified and limited; the terms "connection", "installation", "fixation" and the like should be understood in a broad sense, for example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0160] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0161] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0162] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A ship detection method, characterized in that: include: Acquire point cloud data of the target ship, and convert the point cloud data into corresponding point cloud pillars; Extracting features from the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales, wherein the structure of the feature extraction network is in a pyramid form, and the feature extraction networks at different levels correspond to the first feature information at different scales respectively; Performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale; Performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale; fusing the second feature information and the third feature information to obtain fused feature information; Performing regression processing on the fused feature information to obtain a detection result of the target ship; The step of performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale includes: Determining channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale; Performing weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain the second feature information; The relationship between the second feature information and the first feature information is: Among them, B out Indicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value; The step of performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale includes: Reducing the spatial dimension of the point cloud pillar to obtain corresponding pseudo channel data; Determining spatial weight information of the first feature information corresponding to the second scale based on the spatial attention sub-mechanism and the pseudo-channel data; A weight penalty process is performed on the first feature information corresponding to the second scale according to the spatial weight information to obtain the third feature information.

2. The ship detection method according to claim 1, characterized in that: The channel attention sub-mechanism corresponds to the lower and middle layers of the pyramid structure of the feature extraction network, and the spatial attention sub-mechanism corresponds to the upper layer of the pyramid structure of the feature extraction network.

3. The ship detection method according to claim 1 or 2, characterized in that: The performing regression processing on the fused feature information to obtain the detection result of the target ship includes: Performing regression processing on the two-dimensional detection frame of the fused feature information along the first direction and the second direction of the point cloud pillar to obtain first size information of the fused feature information; Performing regression processing on the size information of the fused feature information along the third direction of the point cloud pillar to obtain second size information of the fused feature information; Determining a detection result of the target ship according to the fused feature information, the first size information, and the second size information; Among them, the first direction is the horizontal axis direction of the coordinates of the point cloud pillar, the second direction is the vertical axis direction of the coordinates of the point cloud pillar, and the third direction is the vertical axis direction of the coordinates of the point cloud pillar.

4. A ship detection device, characterized in that: include: A processing unit, for acquiring point cloud data of a target ship and converting the point cloud data into corresponding point cloud pillars; The processing unit is further used to extract features from the point cloud pillars through a feature extraction network to obtain first feature information of the target ship at different scales, wherein the structure of the feature extraction network is in a pyramid form, and the feature extraction networks at different levels correspond to the first feature information at different scales respectively; The processing unit is further configured to perform weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale; The processing unit is further configured to perform weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale; The processing unit is further configured to fuse the second feature information and the third feature information to obtain fused feature information; The processing unit is further used to perform regression processing on the fused feature information to obtain a detection result of the target ship; The step of performing weight penalty processing on the first feature information of the first scale based on the channel attention sub-mechanism in the normalized attention mechanism to obtain second feature information corresponding to the first scale includes: Determining channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale; Performing weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain the second feature information; The relationship between the second feature information and the first feature information is: Among them, B out Indicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value; The step of performing weight penalty processing on the first feature information of the second scale based on the spatial attention sub-mechanism in the normalized attention mechanism to obtain third feature information corresponding to the second scale includes: Reducing the spatial dimension of the point cloud pillar to obtain corresponding pseudo channel data; Determining spatial weight information of the first feature information corresponding to the second scale based on the spatial attention sub-mechanism and the pseudo-channel data; A weight penalty process is performed on the first feature information corresponding to the second scale according to the spatial weight information to obtain the third feature information.

5. A ship detection system, characterized in that: include: A processing module, used for acquiring point cloud data of a target ship and converting the point cloud data into corresponding point cloud pillars; A feature extraction network is used to extract features from the point cloud pillar to obtain first feature information of the target ship at different scales. The structure of the feature extraction network is in a pyramid form, and the feature extraction networks at different levels correspond to the first feature information at different scales. A normalized attention module, the normalized attention module comprising: a channel attention submodule, configured to perform weight penalty processing on the first feature information at a first scale to obtain second feature information corresponding to the first scale; and A spatial attention submodule, configured to perform weight penalty processing on the first feature information at a second scale to obtain third feature information corresponding to the second scale; The processing module is further used to fuse the second feature information and the third feature information to obtain fused feature information, and perform regression processing on the fused feature information to obtain a detection result of the target ship; The performing weight penalty processing on the first feature information at the first scale to obtain second feature information corresponding to the first scale specifically includes: Determining channel weight information of the first feature information corresponding to the first scale based on the channel attention sub-mechanism and the weight ratio of the first feature information corresponding to the first scale; Performing weight penalty processing on the first feature information corresponding to the first scale according to the channel weight information to obtain the second feature information; The relationship between the second feature information and the first feature information is: Among them, B out Indicates the second feature information, B in Represents the first feature information, μ β represents the mean of the first feature information, σ β represents the standard deviation of the first feature information, γ and β are trainable radiation transformation parameters, BN represents the batch normalization layer, and ∈ represents the minimum value; The performing weight penalty processing on the first feature information at the second scale to obtain third feature information corresponding to the second scale specifically includes: Reducing the spatial dimension of the point cloud pillar to obtain corresponding pseudo channel data; Determining spatial weight information of the first feature information corresponding to the second scale based on the spatial attention sub-mechanism and the pseudo-channel data; A weight penalty process is performed on the first feature information corresponding to the second scale according to the spatial weight information to obtain the third feature information.

6. A path planning method, characterized in that: include: According to the ship detection method according to any one of claims 1 to 3, detecting a target ship to obtain a detection result of the target ship; A navigation path is determined according to the detection result.

7. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the ship detection method according to any one of claims 1 to 3 are implemented.

Images