Ship target detection method and system based on large-kernel multi-scale attention mechanism

Abstract

The invention discloses a ship target detection method and system based on a large-kernel multi-scale attention mechanism. The ship target detection method comprises the steps of obtaining a to-be-detected ship target image; performing target detection on the to-be-detected ship target image by using a target detection neural network based on a multi-scale fusion strategy; in the target detection neural network, a detection task is divided into bounding box prediction, category prediction and object prediction, a large-kernel multi-scale attention module is integrated, the large-kernel multi-scale attention module firstly adopts 5 * 5 depth separable convolution to integrate local features, and then a two-dimensional large convolution kernel (k, k) is decomposed into cascaded horizontal depth separable convolution (1, 2); the method comprises the following steps of: extracting multi-scale information (k, k) and vertical depth separable convolution (k, 1), and weighting channel local features extracted by Ghost Module by utilizing the extracted multi-scale information; and the weighted features are deeply fused through the Ghost Module again, so that the accuracy of ship detection is further improved.

Description

Background Art

[0002] In the field of ship detection, the working environment of ships is very complex. For example, when a ship is sailing, containers, tall buildings, cargo, gravel piles, etc. in the background are very similar to the outline of the ship itself and are often mistaken for ships. Secondly, since the sea is very wide and there are many types of ships, including yachts and small fishing boats, the size of ships varies greatly, which requires the neural network to have good multi-size processing capabilities and to effectively filter background information.

[0003] Using a trained neural network for target detection is a commonly used target detection method; however, the size of the ship itself varies greatly and the background information is complex. It is often difficult to take the background information into account when performing convolution, and it is impossible to use the background information to identify the target.

[0004] In response to the difficulty of ship target detection, various strategies have been proposed in the existing technology to solve the problem of ship target detection, including multi-scale fusion strategy, context information extraction strategy and other methods, which have greatly improved the model performance.

[0005] Among them, for the multi-scale fusion strategy, FPN (Feature Pyramid Networks) is one of the pioneering works, which proposed to construct a feature pyramid for feature fusion; for example, in the YOLO series of networks, from YOLOv3 to the current new version of YOLOv9, multi-scale fusion has appeared.

[0006] Figure 1 The structural diagram of the existing FPN is shown; Figure 1 As can be seen in the figure, the existing FPN realizes multi-scale feature fusion by first upsampling the low-level features and then adding them to the high-level features to form a residual structure, thereby realizing the fusion of high- and low-level features.

[0007] Regarding the context information extraction strategy, self-attention expands the receptive field to the global level, so that it can take into account the global background information and use the background information to identify the target.

[0008] However, although the above multi-scale fusion strategy and context information extraction strategy have performed well when facing ship targets, the performance in detection accuracy still needs to be further improved. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a ship target detection method and system based on a large-core multi-scale attention mechanism to address the deficiencies in the above-mentioned prior art, so as to solve the technical problem of low detection accuracy of the existing methods.

[0010] The present invention adopts the following technical solutions: The ship target detection method based on the large-core multi-scale attention mechanism includes the following steps: Acquire the image of the ship target to be detected; The target detection neural network based on multi-scale fusion strategy is used to detect the target image of the ship to be detected, so as to realize the ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

[0011] Preferably, in the target detection neural network, YOLOX or ResNet50 that implements multi-scale information fusion based on a large-core multi-scale attention module is used.

[0012] Preferably, in the target detection neural network, LKGDH is used as the detection head part.

[0013] Preferably, the local information extracted by input feature X for:

[0014] in, represents the input features, Represents a 5×5 depthwise separable convolution.

[0015] Preferably, deep integration through Ghost Module Specifically:

[0016] in, represents the input features, represents a 3×3 depthwise separable convolution, represents a 1×1 convolution, Represents the result of bitwise multiplication of local channel features and multi-scale features.

[0017] Preferably, the result of bitwise multiplication of the local channel feature and the multi-scale feature is for:

[0018] in, Represents the local and channel features extracted by Ghost Module. represents the extracted multi-scale information, and × represents the bitwise multiplication operation.

[0019] Preferably, the local and channel features extracted by Ghost Module for: .

[0021] Preferably, in the target detection neural network, multi-scale feature extraction is performed on the extracted spatial information to obtain multi-scale information as follows:

[0022] in, Represents the local information extracted from the input feature X, Respectively represent There are three branches, each consisting of a horizontal convolution kernel and a vertical convolution kernel, and the sizes of the convolution kernels are and , .

[0023] Preferably, the ship target image to be detected is an offshore ship image.

[0024] In a second aspect, an embodiment of the present invention provides a ship target detection system based on a large-core multi-scale attention mechanism, including: Data module, obtaining the image of the ship target to be detected; The detection module uses a target detection neural network based on a multi-scale fusion strategy to perform target detection on the image of the ship target to be detected, thereby realizing ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

[0025] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned ship target detection method based on the large-core multi-scale attention mechanism are implemented.

[0026] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned ship target detection method based on the large-core multi-scale attention mechanism.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects: A ship target detection method based on a large-core multi-scale attention mechanism, in which a large-core multi-scale attention module is used in a target detection neural network based on a multi-scale fusion strategy to realize the extraction of multi-scale information; the large-core multi-scale attention mechanism is divided into two branches, one branch extracts cross-channel features and local features through a Ghost Module, and the other branch uses a branch cascaded with multiple vertical large convolution kernels and horizontal large convolution kernels, and the extracted multi-scale information will be bit-wise multiplied with the local information; then, the feature will be further fused through the Ghost Module again; based on the above operations, ship target detection based on the target detection neural network can further improve the detection performance for ships.

[0028] Furthermore, YOLOX or ResNet50 based on the large-core multi-scale attention module to achieve multi-scale information fusion helps improve detection performance, enhance model robustness, reduce computational complexity, simplify network structure, and promote cross-scale feature fusion. These improvements help achieve more efficient and accurate target detection.

[0029] Furthermore, using LKGDH as the detection head helps improve detection performance, enhance model robustness, reduce computational complexity, simplify network structure, and promote cross-scale feature fusion. These improvements help achieve more efficient and accurate object detection, especially when facing objects with large size variations.

[0030] Furthermore, deep fusion settings through Ghost Module help improve feature representation capabilities, enhance model robustness, reduce computational complexity, simplify network structure, and promote cross-scale feature fusion. These improvements help achieve more efficient and accurate target detection.

[0031] Furthermore, bitwise multiplication of local channel features with multi-scale features helps achieve deeper and more comprehensive feature fusion, enhances feature expression capabilities, improves model robustness, simplifies network structure, and improves computational efficiency. These improvements help achieve more efficient and accurate object detection.

[0032] Furthermore, the local and channel features extracted by Ghost Module help improve feature representation capabilities, enhance model robustness, reduce computational complexity, simplify network structure, and promote cross-scale feature fusion. These improvements help achieve more efficient and accurate target detection.

[0033] Furthermore, the multi-scale feature extraction module in the target detection neural network aims to improve detection performance, enhance robustness, reduce computational complexity, simplify the network structure, and promote cross-scale feature fusion. These improvements help achieve more efficient and accurate target detection, especially when facing targets with large size variations.

[0034] Furthermore, offshore vessels usually have different characteristics from inland or ocean-going vessels, such as size, structure, and navigation environment. Target images designed specifically for offshore vessels can better capture these characteristics and improve the accuracy of detection; offshore vessels operate in offshore waters, and their environmental conditions (such as seawater depth, wave size, etc.) are different from those in inland or ocean environments. By setting offshore ships as target images, it can be ensured that the detection system can adapt to visual changes under these specific conditions. Clearly defining offshore ships as target images helps to limit the application scenarios of the detection system, making the development and optimization process more centralized and efficient. This helps to reduce unnecessary complexity and focus on improving the performance of offshore ship detection. The detection of offshore ships includes both optical detection problems under land backgrounds and the influence of marine environmental factors such as waves and tides. Target detection in this environment is both challenging and relatively controllable, which is conducive to the research and development of new detection technologies and algorithms. Offshore ships are widely used in many fields such as cargo transportation, offshore engineering, and fishery operations. Research and development on this target image can directly serve these industries and has high practical application value.

[0035] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0036] In summary, the present invention significantly improves the performance of ship target detection through an innovative neural network structure, and provides new ideas and directions for the development of the field of ship target detection.

[0037] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings to be used in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is a structural diagram of the existing ResNet50 network; Figure 2 It is a flow chart of the ship target detection method based on the large-core multi-scale attention mechanism of the present invention; Figure 3 The decoupled detection head based on the large-core multi-scale attention mechanism proposed in the present invention; Figure 4 for Figure 3 Schematic diagram of the structure of the large-core multi-scale attention module used in the detection head; Figure 5 In the embodiment of the present invention, Figure 1 The ResNet50 shown in the figure uses Figure 3 The structural schematic diagram of the decoupling detection head is shown; Figure 6 is the label distribution of the data set used in the simulation verification of the embodiment of the present invention; Figure 7 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 8 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0042] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0043] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0044] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0045] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0046] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0047] The present invention provides a ship target detection method based on a large-core multi-scale attention mechanism, firstly obtaining a ship target image to be detected; using a target detection neural network based on a multi-scale fusion strategy to detect the target image; wherein the detection head in the target detection neural network is divided into a detection head based on a large-core multi-scale attention mechanism; the detection head adopts a decoupled design, and the detection task is divided into bounding box prediction, category prediction and object prediction, wherein object prediction and bounding box prediction share a branch. Compared with the ordinary decoupled detection head, a large-core multi-scale attention module is integrated. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the ordinary two-dimensional large convolution kernel (k, k) into a cascade of horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). Three scales (k=7, 11, 21) are used in the implementation to reduce the number of parameters while obtaining multi-scale spatial information. The extracted multi-scale information will weight the channel local features extracted by the Ghost Module; the weighted features will be deeply fused again through the Ghost Module. The present invention further improves the accuracy of ship detection.

[0048] See also Figure 2 The present invention provides a ship target detection method based on a large-core multi-scale attention mechanism, comprising the following steps: S1. Acquire a target image to be detected by a ship; Here, there may be multiple specific types of target images, such as images taken by a drone at high altitude, or remote sensing images taken by a radar, etc.; in such target images, there may be various types of ships.

[0049] S2. Use the preset target detection neural network based on multi-scale fusion strategy to detect ships in the target image.

[0050] See also Figure 4 In the target detection neural network, a large-core multi-scale attention module is used to extract multi-scale spatial information. The large-core multi-scale attention mechanism is divided into two branches. One branch extracts cross-channel features and local features through the Ghost Module, and the other branch uses multiple vertical large convolution kernels and horizontal large convolution kernels cascaded. The extracted multi-scale information will be bit-wise multiplied with the local information. Then, the feature will be further fused through the Ghost Module again to get the final output.

[0051] Exemplarily, the target detection neural network based on the above LKG to implement large-core multi-scale attention may include: YOLOX or ResNet50, but is certainly not limited thereto.

[0052] In order to make the layout of the manual clear, the following example illustrates how to implement the fusion of channel features and spatial features based on the large-core multi-scale attention module in the target detection neural network: In the above large-core multi-scale attention module, the preprocessing method for extracting local information from input features includes:

[0053] in, represents the input features, represents a 5×5 depthwise separable convolution, Represents the local information extracted from the input feature X.

[0054] It can be understood that the depth-wise separable convolution used here performs convolution on the feature map of each channel, so the main features extracted here are spatial features.

[0055] Perform multi-scale feature extraction on the extracted spatial information, including:

[0056] in, represents the extracted multi-scale information, Represents the local information extracted in the previous step, Represent three branches respectively. The three branches are composed of a horizontal convolution kernel and a vertical convolution kernel. The sizes of the convolution kernels are and .

[0057] It is understandable that here a large convolution kernel is decomposed into a vertical convolution and a horizontal convolution and depth-separable convolution is used, which reduces the number of parameters while achieving the same effect and facilitates deployment. Different branches have different receptive fields, so the algorithm can adaptively select the required scale information according to the input image.

[0058] Here, in order to make full use of the cross-channel information of the input features without losing the rich local information, the embodiment of the present invention extracts the spatial features while another branch adopts the Ghost Module, specifically, including:

[0059] in, represents the input features, Represents the local and channel features extracted by Ghost Module. represents a 3×3 depthwise separable convolution, It means that a 1×1 convolution is used to reduce the number of channels and extract channel features.

[0060] It is understandable that Ghost Module first performs channel modeling and then spatial modeling. During spatial modeling, the similarity of adjacent feature maps is used to perform partial feature residual connections, effectively reducing the amount of parameters.

[0061] Next, the local channel features are bitwise multiplied with the multi-scale features, including:

[0062] in, Represents the local and channel features extracted by Ghost Module. Represents the extracted multi-scale information, where × represents a bitwise multiplication operation.

[0063] It is understandable that the features extracted by the two branches focus on different information. In order to ensure that the model can make full use of all information, a bitwise multiplication method is used here to fuse multi-scale spatial information and cross-channel local information.

[0064] Then, the fused features need to be further fused through the Ghost Module, including:

[0065] in, Represents the final output of the large-core multi-attention mechanism.

[0066] The following example illustrates how to implement multi-scale information extraction based on a large-core multi-scale attention module in a target detection neural network.

[0067] For example, Figure 5 A schematic diagram of the structure of ResNet50 based on a large-core multi-scale attention module for multi-scale information extraction is shown in Figure 2. Figure 1 It can be seen from the structure of the existing ResNet50 shown in the figure that the large-core multi-scale attention module in the embodiment of the present invention further strengthens the multi-scale spatial features of P2-P6 and uses multi-scale information to improve the detection performance of targets of different scales.

[0068] Similarly, the method of implementing multi-scale information extraction based on the large-core multi-scale attention mechanism LKG in YOLOX can refer to the above-mentioned embodiment of implementing multi-scale information extraction based on LKG in ResNet50, and the embodiments of the present invention will not be repeated here.

[0069] It should be noted that the application scenarios of the embodiments of the present invention are relatively broad and are not limited to feature inputs of one scale. They also have good performance even when faced with feature inputs of other scales.

[0070] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0071] In another embodiment of the present invention, a ship target detection system based on a large core multi-scale attention mechanism is provided. The system can be used to implement the above-mentioned ship target detection method based on the large core multi-scale attention mechanism. Specifically, the ship target detection system based on the large core multi-scale attention mechanism includes a data module and a detection module.

[0072] Among them, the data module obtains the image of the ship target to be detected; The detection module uses a target detection neural network based on a multi-scale fusion strategy to perform target detection on the image of the ship target to be detected, thereby realizing ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

[0073] In another embodiment of the present invention, a terminal device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the ship target detection method based on the large-core multi-scale attention mechanism, including: Acquire the image of the ship target to be detected; The target detection neural network based on multi-scale fusion strategy is used to detect the target image of the ship to be detected, so as to realize the ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

[0074] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device, and can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0075] Computer readable storage media also include data signals propagated in baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0076] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0077] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the ship target detection method based on a large-core multi-scale attention mechanism in the above embodiment; one or more instructions in the computer-readable storage medium are loaded and executed by the processor as follows: Acquire the image of the ship target to be detected; The target detection neural network based on multi-scale fusion strategy is used to detect the target image of the ship to be detected, so as to realize the ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

[0078] See also Figure 7 , the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the ship target detection method based on the large core multi-scale attention mechanism in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the ship target detection system based on the large core multi-scale attention mechanism in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0079] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 7 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0080] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, central processing units, graphics processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, data processing logic based on quantum computing, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0081] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0082] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0083] Any reference to a memory, database or other medium used in the embodiments provided in the present application may include at least one of a non-volatile and a volatile memory. Non-volatile memory may include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. Volatile memory may include a random access memory (RAM) or an external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0084] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0085] See also Figure 8 The terminal device is an electronic device 600, and the electronic device is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0086] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 2 Follow the steps shown in .

[0087] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0088] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0089] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0090] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0091] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0092] The beneficial effects of the embodiments of the present invention are further illustrated by simulation experiments below.

[0093] Experimental environment The experimental platform is PyTorchversion 1.13.0, CUDA11.6, GPU (graphics processor) model is RTX4090, hardware memory is 24G, and the compilation language is Python 3.8.

[0094] Experimental Dataset The dataset uses the Seaships7000 dataset, which contains 7,000 images of 6 types of targets from coastal surveillance videos, with image resolutions of 1920 × 1080. In this dataset, there are six types of ships, including ore carriers, container ships, bulk cargo carriers, general cargo ships, fishing boats, and passenger ships.

[0095] Since the images in the Seaships7000 dataset are large in size and not suitable for direct detection, they are cropped to 640×640.

[0096] Then the data subsets are split according to the ratio of 1:1:2 for training set: validation set: test set. There are 1750 images for training and validation, and 3500 images for testing. The distribution of training set labels is as follows: Figure 5 As shown, the horizontal axis corresponds to oil carrier, container ship, bulk cargo carrier, general cargo ship, fishing boat and passenger ship respectively.

[0097] Evaluation indicators In order to evaluate the effectiveness of the object detection method, accuracy, recall, average precision (AP), mean average precision (mAP) and model inference time are used to evaluate the algorithm.

[0098] The accuracy is the proportion of correct samples in the total test, and the calculation formula is:

[0099] Among them, true positives (TP) are the number of positive samples that are accurately predicted, and false positives (FP) are the number of negative samples that are mistakenly detected as positive samples.

[0100] Recall represents the proportion of positive samples that are accurately predicted and is defined as:

[0101] Here, false negatives (FN) is the number of positive samples that are predicted as negative samples.

[0102] AP is the area enclosed by the accuracy curve and the recall rate on the x-axis, which is expressed as:

[0103] in, y is the recall curve under different intersection ratio thresholds, Recall y The corresponding accuracy.

[0104] mAP refers to the average value of all AP categories, and the calculation formula is:

[0105] Where N is the number of categories, and N=6 in the experiment.

[0106] Experimental procedures Based on YOLOX, the ship target detection performance with and without LKG is compared. The experimental results on the test dataset are shown in the following table:

[0107] Among them, YoloX represents the existing YoloX without the large-core multi-scale attention module; YoloX-LKG represents the YoloX with the large-core multi-scale attention module.

[0108] As can be seen from the table above, after using the large-core multi-scale attention module, the mAP@.5:.95 on YOLOX increased from 70.1% to 73.2%, mAP@.5 and mAP@.75 increased by 0.2% and 3.8% respectively, the number of parameters decreased from 54.15M to 51.38M, and the inference time increased from 20.4ms to 24.1ms, and the increase in time consumption is almost negligible.

[0109] In summary, the present invention is a ship target detection method and system based on a large-core multi-scale attention mechanism. By extracting cross-channel local features and multi-scale spatial information from input features, in order to reduce the amount of parameters, the multi-scale spatial information extraction part uses a deep separable convolution. In addition, a large convolution kernel is decomposed into a vertical convolution kernel and a horizontal convolution kernel. The cross-channel local feature extraction part uses a Ghost Module. The convolution first performs a 1×1 cross-channel convolution, and then uses the similarity between feature maps to convolve a part of the feature map, and the other part directly forms a residual connection. The cross-channel local features and multi-scale spatial features will be bit-by-bit multiplied, so that more and larger-sized information can be felt while maintaining rich local features, and then the Ghost Module will be performed again for further fusion. Therefore, the target detection neural network used in the embodiment of the present invention takes into account the extraction of multi-scale information, so ship target detection based on the target detection neural network can further improve the detection performance of ship targets. In particular, the method of the present invention can better solve the problem of ship target detection with large target size changes.

[0110] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0111] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0112] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0113] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0116] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A ship target detection method based on a large-core multi-scale attention mechanism is characterized by: The following steps are involved: Acquire the image of the ship target to be detected; The target detection neural network based on multi-scale fusion strategy is used to detect the target image of the ship to be detected, so as to realize the ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

2. The ship target detection method based on large-core multi-scale attention mechanism according to claim 1 is characterized in that: In the target detection neural network, YOLOX or ResNet50 is used to implement multi-scale information fusion based on the large-core multi-scale attention module.

3. The ship target detection method based on large-core multi-scale attention mechanism according to claim 1 is characterized in that: In the target detection neural network, LKGDH is used as the detection head.

4. The ship target detection method based on large-core multi-scale attention mechanism according to claim 1 is characterized in that: Input feature X to extract local information for: in, represents the input features, Represents a 5×5 depthwise separable convolution.

5. The ship target detection method based on large-core multi-scale attention mechanism according to claim 1 is characterized in that: Deep integration through Ghost Module Specifically: in, represents the input features, represents a 3×3 depthwise separable convolution, represents a 1×1 convolution, Represents the result of bitwise multiplication of local channel features and multi-scale features.

6. The ship target detection method based on large-core multi-scale attention mechanism according to claim 5 is characterized in that: The result of bitwise multiplication of local channel features and multi-scale features for: in, Represents the local and channel features extracted by Ghost Module. represents the extracted multi-scale information, and × represents the bitwise multiplication operation.

7. The ship target detection method based on large-core multi-scale attention mechanism according to claim 6 is characterized in that: Local and channel features extracted by Ghost Module for: 。 8. The ship target detection method based on large-core multi-scale attention mechanism according to claim 6 is characterized in that: In the target detection neural network, multi-scale feature extraction is performed on the extracted spatial information to obtain multi-scale information as follows: in, Represents the local information extracted from the input feature X, Respectively represent There are three branches, each consisting of a horizontal convolution kernel and a vertical convolution kernel, and the sizes of the convolution kernels are and , .

9. The ship target detection method based on large-core multi-scale attention mechanism according to claim 1 is characterized in that: The target ship image to be detected is an offshore ship image.

10. A ship target detection system based on a large-core multi-scale attention mechanism, characterized in that: include: Data module, obtaining the image of the ship target to be detected; The detection module uses a target detection neural network based on a multi-scale fusion strategy to perform target detection on the image of the ship target to be detected, thereby realizing ship target detection; In the target detection neural network, the detection task is divided into bounding box prediction, category prediction and object prediction, and integrated into the large-core multi-scale attention module. The large-core multi-scale attention module first uses a 5×5 depth-separable convolution to aggregate local features, and then decomposes the two-dimensional large convolution kernel (k, k) into cascaded horizontal depth-separable convolution (1, k) and vertical depth-separable convolution (k, 1). The extracted multi-scale information is used to weight the channel local features extracted by the Ghost Module; the weighted features are deeply fused again through the Ghost Module.

Images