Target detection method for sea scene, terminal equipment and storage medium
Through the target detection of regional segmentation of the sea field image, the problem of low computing efficiency in the traditional method is solved, the efficient target detection is achieved, the deficiency alarm rate is reduced and the detection accuracy is improved.
Patent Information
- Application Number
- CN202411423897.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-01-21
AI Technical Summary
In sea area scenarios, traditional object detection methods assume that the target is distributed throughout the image space, resulting in a large number of repeated invalid calculations, which reduces the calculation efficiency.
A target detection method for sea scenes is adopted, and target testing is performed by regional segmentation of the sea field image. This method includes feature extraction modules, feature fusion modules, and semantic segmentation heads, and uses the regional segmentation network model to segmentation and target detection of seas scenario images.
It significantly reduces the calculation amount of manned and ship target detection and improves detection efficiency; it effectively filters out irrelevant background areas, reduces the false alarm rate and improves the recall rate, ultimately improving detection accuracy.
Smart Images
Figure CN119339058A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a target detection method, terminal equipment and storage medium for sea scene. Background Art
[0002] With the implementation and deepening of public safety policies, the application of target detection algorithms based on intelligent analysis in monitoring facilities has become an important technical means to maintain public safety in the sea area, especially emphasizing real-time and accuracy. Traditional target detection methods often assume that the target is distributed in the entire image space and perform a comprehensive scan of the entire image to achieve target positioning and identification, which will lead to a large number of repeated and invalid calculations and reduce computational efficiency. Summary of the invention
[0003] In order to solve the above problems, the present invention proposes a target detection method, a terminal device and a storage medium for sea area scenes.
[0004] The specific plan is as follows:
[0005] A target detection method for a sea scene comprises the following steps:
[0006] S1: Collect sea scene images and mark the areas where targets may exist, and build a training set based on the marked images;
[0007] S2: Build a region segmentation network model and train the model using the training set;
[0008] The model includes a feature extraction module, a feature fusion module, and a semantic segmentation head;
[0009] The feature extraction module consists of a Stem block and four MobileNet modules that are adjacent to each other. A Transformer layer is inserted after the third MobileNet module and the fourth MobileNet module, which are the first Transformer layer and the second Transformer layer respectively. The stride of the first, third, and fourth MobileNet modules is not 1. The Transformer layer consists of multiple Transformer modules.
[0010] The feature fusion module first performs 1×1 convolution and batch normalization on the output of the Stem block, and simultaneously performs 1×1 convolution, batch normalization, sigmoid activation and upsampling operations on the outputs of the second MobileNet module, the first Transformer layer and the second Transformer layer; then multiplies the processing results of the Stem block, the second MobileNet module and the output of the first Transformer layer to obtain the first fused feature; then multiplies the processing result of 1×1 convolution and batch normalization on the first fused feature with the processing result of the output of the second Transformer layer to obtain the second fused feature; finally, the processing result of 1×1 convolution, batch normalization and upsampling on the output of the second Transformer layer is added to the second fused feature to obtain the third fused feature and output it;
[0011] The semantic segmentation head obtains the classification prediction result of each pixel in the image according to the third fusion feature output by the feature fusion module;
[0012] S3: After the trained model is used to segment the sea scene image to be detected, the region of interest is screened and target detection is performed based on the region of interest.
[0013] Furthermore, the targets include human targets and ship targets.
[0014] Furthermore, the sea scene image is collected by frame extraction through videos shot by surveillance cameras deployed in the sea scene.
[0015] Furthermore, the collected sea scene images include images corresponding to different weather conditions and different time periods of the day.
[0016] Furthermore, the X-AnyLabeling labeling tool is used for semi-automatic labeling to mark the areas where the human targets and ship targets are located.
[0017] Furthermore, the Stem block consists of a regular convolution with a stride of 2 and four MobileNet modules connected in sequence, and the stride of the first and third MobileNet modules is 2.
[0018] Furthermore, the MobileNet module adopts MobileNetV2.
[0019] Furthermore, the number of channels of K and Q in the Transformer module is set to 16, and the number of channels of V is set to 32.
[0020] A target detection terminal device for sea area scenes includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described above in the embodiment of the present invention are implemented.
[0021] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above in an embodiment of the present invention are implemented.
[0022] The present invention adopts the above technical solution and has the following beneficial effects:
[0023] (1) By performing regional segmentation on the sea scene image and then performing target detection, the computational complexity of human and ship target detection can be significantly reduced, thus improving detection efficiency;
[0024] (2) The regional segmentation results can effectively filter out irrelevant background areas, effectively reduce the false alarm rate of target detection and improve the recall rate, ultimately improving the detection accuracy;
[0025] (3) Through the training and optimization of deep learning models, accurate segmentation and target detection of complex sea scenes can be achieved, enhancing the robustness and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 Shown is a flow chart of a method according to a first embodiment of the present invention.
[0027] Figure 2 Shown is a schematic diagram of the network structure of the model in this embodiment. DETAILED DESCRIPTION
[0028] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.
[0029] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.
[0030] Embodiment 1:
[0031] Image semantic segmentation technology can realize scene understanding of smart devices by assigning a class to each pixel of a given image. In the marine monitoring scene, human and ship targets usually appear on a specific plane (such as a beach or sea level). By segmenting the marine scene, the area where the human and ship targets are located in the image can be obtained. Target detection in the region of interest can not only achieve accurate sampling of the positions of human and ship targets in the marine monitoring scene, but also significantly improve computing efficiency. Based on this, an embodiment of the present invention provides a target detection method for marine scenes, such as Figure 1 As shown, the method comprises the following steps:
[0032] S1: Collect sea scene images and mark the areas where targets may exist, and build a training set based on the marked images.
[0033] In this embodiment, the targets include human targets and ship targets. The collected sea scene images are collected by using a frame extraction method through a video captured by a surveillance camera deployed in the sea scene. In order to improve the accuracy of subsequent detection, in this embodiment, it is preferably set that the collected sea scene images include images corresponding to different weather conditions and different time periods of the day.
[0034] In the labeling process, this embodiment uses the X-AnyLabeling labeling tool for semi-automatic labeling to mark areas where human targets and ship targets may exist (such as beaches, sea surfaces, rocks, etc.), and ensure that the category names of different areas are different.
[0035] S2: Build a region segmentation network model and train the model using the training set.
[0036] The model constructed in this embodiment is a lightweight region segmentation network model suitable for sea area scenarios, such as Figure 2 As shown, the model includes a feature extraction module, a feature fusion module and a semantic segmentation head.
[0037] (1) Feature extraction module
[0038] The feature extraction module consists of a Stem block and four MobileNet modules that are adjacent to each other, and the first Transformer layer is inserted after the third MobileNet module, and the second Transformer layer is inserted after the fourth MobileNet module, wherein the step sizes of the first, third, and fourth MobileNet modules are not 1, and are used for downsampling operations. In this embodiment, the step sizes of the first, third, and fourth MobileNet modules are set to 2 as three downsampling branches; the step size of the second MobileNet module is 1.
[0039] A traditional Stem block is usually composed of conventional convolutions. The Stem block used in this embodiment is connected to four MobileNet modules on the basis of the existing conventional convolutions. That is, it is composed of a conventional convolution with a stride of 2 and four MobileNet modules connected in sequence, and the stride of the first and third MobileNet modules is 2, so the Stem block is downsampled 8 times in total.
[0040] In this embodiment, the MobileNet module adopts MobileNetV2.
[0041] By using the MobileNet module in the feature extraction module, the expression capability can be increased while maintaining the amount of computation. In order not to further increase the amount of computation, this embodiment connects the second MobileNet module with a step size of 1 after the first downsampling branch (the first MobileNet module), and only connects the Transformer layer after the second and third downsampling branches (the third and fourth MobileNet modules).
[0042] The Transformer layer is composed of multiple Transformer modules. Each Transformer module adopts a conventional structure, that is, it is composed of a multi-head attention module, a forward propagation module, and a residual connection. In this embodiment, the number of channels of K and Q in the multi-head attention module is set to 16, and the number of channels of V is set to 32 to reduce the computational cost when calculating attention mapping and output.
[0043] After the input image received by the feature extraction module is input from the Stem block, the output of the Stem block (1 / 8) and the outputs of the three downsampling branches (1 / 16, 1 / 32, 1 / 64) are used as the output of the feature extraction module. Three outputs are obtained from the three downsampling branches, where the output of the first downsampling branch is the output of the second MobileNet module (1 / 16), and the outputs of the second and third downsampling branches are the outputs of the first and second Transformer layers (1 / 32, 1 / 64). After the feature map of the input image with a channel number and a size of C×H×W passes through the feature extraction module, the feature {F ×8 , F ×16 , F ×32 , F ×64}, indicating that the features are sampled at 8, 16, 32, and 64 times the original resolution.
[0044] (2) Feature Fusion Module
[0045] The feature fusion module first performs 1×1 convolution and batch normalization (Conv+BN) on the output of the Stem block, and simultaneously performs 1×1 convolution, batch normalization, sigmoid activation and upsampling (Conv+BN+Sigmoid+Up) on the output of the second MobileNet module, the first Transformer module and the second Transformer layer (i.e., the output of the three downsampling branches); then multiplies the processing results of the Stem block, the second MobileNet module and the output of the first Transformer layer to obtain the first fused feature (1 / 8); then multiplies the processing result of 1×1 convolution and batch normalization (Conv+BN) on the first fused feature with the processing result of the output of the second Transformer layer to obtain the second fused feature (1 / 8); finally, the processing result of 1×1 convolution, batch normalization and upsampling (Conv+BN+Up) on the output of the second Transformer layer is added to the second fused feature to obtain the third fused feature (1 / 8) and output it.
[0046] In the feature fusion module of this embodiment, the fusion of the four sub-features output by the feature extraction module is divided into two stages, that is, the first three sub-features are fused first, and then the first fused feature obtained by fusion is fused with the fourth sub-feature. The fusion of the two stages can reduce information loss and interference. The first stage focuses on the fusion of detail features, and the second stage further processes and filters these features, thereby improving the robustness of the model to noise and complex backgrounds.
[0047] When the feature fusion module performs upsampling, it is upsampled to the same resolution as the output of the Stem block (i.e., F ×8 ), the weight coefficient of feature fusion can be obtained through the sigmoid activation function. Finally, the F ×64 Adding to the filtered detail features can further enhance the model's understanding of the global context and help improve the segmentation ability of large areas.
[0048] (3) Semantic Segmentation Head
[0049] The semantic segmentation head obtains the classification prediction result of each pixel in the image according to the third fusion feature output by the feature fusion module. In this embodiment, the semantic segmentation head only uses two convolutional layers, and each convolutional layer includes a batch normalization operation.
[0050] In this embodiment, the loss function during model training uses cross entropy loss to calculate pixel classification loss and update model parameters through the back propagation algorithm; repeat the process of forward propagation, loss calculation and back propagation, iterate the training multiple times until the model converges, and finally obtain an optimal solution as the weight of the region segmentation network module.
[0051] S3: After the trained model is used to segment the sea scene image to be detected, the region of interest is screened and target detection is performed based on the region of interest.
[0052] After inputting the sea scene image to be detected (i.e., for human and ship target detection) into the trained model, a segmentation mask can be obtained; the segmentation mask is converted into a binary image, in which the region of interest (such as sea surface, beach, rock, etc., which can be set by technicians in this field) is white (255) and the background (sky, woods) is black (0), and the contour in the mask is extracted through the binary image to obtain the bounding box of the region of interest; according to the extracted bounding box, the region of interest is cropped from the original image, and the image after cropping to obtain the bounding box of the region of interest is adjusted to the input size required by the target detection model, and converted into the format required by the target detection model (such as YOLO); the processed cropped image is input into the target detection model for inference using the pre-trained target detection model to obtain the human and ship target detection results, including the bounding box, category and confidence.
[0053] The embodiments of the present invention have the following beneficial effects:
[0054] (1) By performing regional segmentation on the sea scene image and then performing target detection, the computational complexity of human and ship target detection can be significantly reduced, thus improving detection efficiency;
[0055] (2) The regional segmentation results can effectively filter out irrelevant background areas, effectively reduce the false alarm rate of target detection and improve the recall rate, ultimately improving the detection accuracy;
[0056] (3) Through the training and optimization of deep learning models, accurate segmentation and target detection of complex sea scenes can be achieved, enhancing the robustness and adaptability of the system.
[0057] Embodiment 2:
[0058] The present invention also provides a target detection terminal device for sea area scenes, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the above-mentioned method embodiment of embodiment 1 of the present invention when executing the computer program.
[0059] Further, as an executable solution, the target detection terminal device for the sea scene can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The target detection terminal device for the sea scene may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the composition structure of the target detection terminal device for the sea scene is only an example of the target detection terminal device for the sea scene, and does not constitute a limitation on the target detection terminal device for the sea scene. It may include more or less components than the above, or a combination of certain components, or different components. For example, the target detection terminal device for the sea scene may also include input and output devices, network access devices, buses, etc., which are not limited in the embodiments of the present invention.
[0060] Further, as an executable solution, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the target detection terminal device for the sea scene, and uses various interfaces and lines to connect the various parts of the target detection terminal device for the sea scene.
[0061] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the target detection terminal device for sea scene by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and an application required for at least one function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0062] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.
[0063] If the module / unit integrated in the target detection terminal device for sea scene is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc.
[0064] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.
Claims
1. A target detection method for sea scenes, characterized in that: The following steps are involved: S1: Collect sea scene images and mark the areas where targets may exist, and build a training set based on the marked images; S2: Build a region segmentation network model and train the model using the training set; The model includes a feature extraction module, a feature fusion module, and a semantic segmentation head; The feature extraction module consists of a Stem block and four MobileNet modules that are adjacent to each other. A Transformer layer is inserted after the third MobileNet module and the fourth MobileNet module, which are the first Transformer layer and the second Transformer layer respectively. The stride of the first, third, and fourth MobileNet modules is not 1. The Transformer layer consists of multiple Transformer modules. The feature fusion module first performs 1×1 convolution and batch normalization on the output of the Stem block, and simultaneously performs 1×1 convolution, batch normalization, sigmoid activation and upsampling operations on the outputs of the second MobileNet module, the first Transformer layer and the second Transformer layer; then multiplies the processing results of the Stem block, the second MobileNet module and the output of the first Transformer layer to obtain the first fused feature; then multiplies the processing result of 1×1 convolution and batch normalization on the first fused feature with the processing result of the output of the second Transformer layer to obtain the second fused feature; finally, the processing result of 1×1 convolution, batch normalization and upsampling on the output of the second Transformer layer is added to the second fused feature to obtain the third fused feature and output it; The semantic segmentation head obtains the classification prediction result of each pixel in the image according to the third fusion feature output by the feature fusion module; S3: After the trained model is used to segment the sea scene image to be detected, the region of interest is screened and target detection is performed based on the region of interest.
2. The target detection method for sea scene according to claim 1, characterized in that: Targets include human targets and ship targets.
3. The target detection method for sea scene according to claim 1, characterized in that: The sea scene images are collected by frame extraction through videos shot by surveillance cameras deployed in the sea scene.
4. The target detection method for sea scene according to claim 1, characterized in that: The collected sea scene images include images corresponding to different weather conditions and different time periods of the day.
5. The target detection method for sea scene according to claim 1, characterized in that: During labeling, the X-AnyLabeling labeling tool is used for semi-automatic labeling to mark the areas where the human targets and ship targets are located.
6. The target detection method for sea scene according to claim 1, characterized in that: The Stem block consists of a regular convolution with a stride of 2 and four MobileNet modules connected in sequence, and the stride of the first and third MobileNet modules is 2.
7. The target detection method for sea scene according to claim 1, characterized in that: The MobileNet module uses MobileNetV2.
8. The target detection method for sea scene according to claim 1, characterized in that: The number of channels of K and Q in the Transformer module is set to 16, and the number of channels of V is set to 32.
9. A target detection terminal device for marine scenes, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.