Underwater target detection methods, devices and training methods, devices and electronic equipment
By combining Generative Adversarial Network (GAN) and Laplacian Pyramid fusion algorithm, underwater target images are enhanced and a YOLOv4 model is trained, solving the problems of slow underwater target detection speed and low accuracy, and achieving efficient and accurate underwater target recognition.
Patent Information
- Application Number
- CN202210003943.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-05
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-01-05
AI Technical Summary
Existing underwater target detection methods suffer from slow detection speed, low recognition accuracy, and poor robustness in complex and variable underwater environments, making it difficult to meet the precision operation requirements of underwater robotic arms.
Generative Adversarial Network (GAN) is used to enhance underwater degraded images, and the fourth-generation target detection model YOLOv4 is used for detection. At the same time, the Laplacian pyramid fusion algorithm is used to generate underwater target images to enrich the training data and avoid overfitting.
It improves the accuracy and robustness of underwater target detection, enhances detection speed, solves the image degradation problem in underwater environments, and improves the operational accuracy of underwater robotic arms.
Smart Images

Figure CN116452962B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more particularly to an underwater target detection method, apparatus, training method, apparatus, and electronic equipment. Background Technology
[0002] Real-time and accurate detection and identification of underwater targets is a prerequisite for the precise operation of underwater robotic arms.
[0003] In existing technologies, most underwater robotic arms employ computer vision methods for underwater target detection and identification. With the development of deep learning theory, target detection models based on convolutional neural networks have become an effective approach for accurate underwater target detection and identification.
[0004] However, due to the complex and variable underwater environment, the attenuation and scattering of light cause underwater images to suffer from color distortion, underexposure and blurring, resulting in image degradation problems. Currently, traditional underwater target detection methods have disadvantages such as slow detection speed, low recognition accuracy and poor robustness, which can hardly meet the precision operation requirements of underwater robotic arms. Summary of the Invention
[0005] This application provides an underwater target detection method, apparatus, and computer-readable storage medium, which helps to improve the detection accuracy of underwater targets.
[0006] To achieve the above objectives, this application adopts the following technical solution:
[0007] Firstly, an underwater target detection method is provided. This underwater target detection method includes: acquiring a target image containing underwater targets; performing enhancement processing on the target image to obtain an enhanced image; and detecting the targets in the target image using the fourth-generation target detection model YOLOv4.
[0008] As can be seen from the method described in the first aspect, since the image to be detected, which includes underwater targets, is enhanced before target detection is performed, the image to be detected, which includes degraded underwater targets, can be obtained as an enhanced image with clear underwater targets. This is beneficial for YOLOv4 to extract more accurate target features when performing target detection on the enhanced image. In addition, YOLOv4 has the advantages of fast target detection speed, high detection accuracy, and good robustness, so it can improve the overall accuracy of target detection.
[0009] In conjunction with the first aspect, in one possible design scheme, the enhancement processing of the image to be detected to obtain an enhanced image includes: inputting the image to be detected into a generative adversarial network (GAN), performing enhancement processing on the image to be detected through the GAN, and outputting the enhanced image.
[0010] Based on this possible design scheme, it can be seen that this embodiment uses a generative adversarial network (GAN) to enhance the underwater degraded image to be detected, which is beneficial to improving the clarity of the image to be detected.
[0011] In conjunction with the first aspect, in one possible design, the GAN includes a generator network G and a discriminator network D, and the loss function of the GAN is: in, For non-saturating GAN loss, Loss L1 (G) represents the L1 loss, Loss g (G) represents the gradient loss; λ1 is the first parameter, λ g The second parameter; Loss L1 (G)=||xG(y)||1; The training set P, which represents the distribution of x-sampled free-water target images, is... train , The dataset P represents the distribution of underwater degraded images sampled from the generator network G. gen D(x) represents the probability that the discriminator network D classifies a real underwater target image as an underwater image, and G(y) represents the enhanced image generated by the generator network G based on the degraded underwater target image.
[0012] In conjunction with the first aspect, in one possible design scheme, before detecting the enhanced image using the fourth-generation target detection model YOLOv4, the method further includes: acquiring an underwater background image and an above-water target image; fusing the underwater background image and the above-water target image using a Laplacian pyramid fusion algorithm to generate an underwater target image; and training YOLOv4 to converge using the underwater target image.
[0013] In existing technologies, underwater image acquisition is time-consuming and costly, resulting in a scarcity of underwater target image data. This data is insufficient to support the training of deep learning-based target detection models, leading to overfitting issues. Based on this potential design, this embodiment uses a method of fusing underwater background images with surface target images to generate underwater target images. These underwater target images are then used as training data for YOLOv4, thus avoiding overfitting problems caused by insufficient training data.
[0014] In conjunction with the first aspect, in one possible design scheme, the step of fusing the underwater background image and the surface target image using a Laplacian pyramid fusion algorithm to generate an underwater target image includes: generating a first Laplacian pyramid corresponding to the background image; generating a second Laplacian pyramid corresponding to the surface target image; and fusing the first Laplacian pyramid and the second Laplacian pyramid to generate the underwater target image.
[0015] Secondly, a training method is provided for training the fourth-generation object detection model YOLOv4. This training method includes: acquiring an underwater background image and an above-water target image; fusing the underwater background image and the above-water target image using a Laplacian pyramid fusion algorithm to generate an underwater target image; and training YOLOv4 using the underwater target image until convergence. In existing technologies, due to the long time and high cost of acquiring underwater images, current underwater target image data is scarce and insufficient to support the training of deep learning-based object detection models, leading to overfitting issues. Based on this possible design, this embodiment uses the method of fusing underwater background images and above-water target images to generate underwater target images, and uses these underwater target images as training data for YOLOv4, thus avoiding overfitting problems caused by insufficient training data.
[0016] Thirdly, an underwater target detection device is provided. This underwater target detection device includes: a first acquisition module, an enhancement module, and a detection module.
[0017] The first acquisition module is used to acquire images of the target to be detected, including underwater targets;
[0018] An enhancement module is used to enhance the image to be detected to obtain an enhanced image;
[0019] The detection module is used to detect the enhanced image using the fourth-generation target detection model YOLOv4 to obtain the target in the image to be detected.
[0020] Optionally, the enhancement module of the underwater target detection device described in the third aspect is specifically used to: input the image to be detected into a generative adversarial network (GAN), perform enhancement processing on the image to be detected through the GAN, and output the enhanced image.
[0021] Optionally, in the underwater target detection device described in the third aspect, the GAN includes a generator network G and a discriminator network D, and the loss function of the GAN is: in, For non-saturating GAN loss, Loss L1(G) represents the L1 loss, Loss g (G) represents the gradient loss; λ1 is the first parameter, λ g The second parameter; Loss L1 (G)=||xG(y)||1; The training set P, which represents the distribution of x-sampled free-water target images, is... train , The dataset P represents the distribution of underwater degraded images sampled from the generator network G. gen D(x) represents the probability that the discriminator network D classifies a real underwater target image as an underwater image, and G(y) represents the enhanced image generated by the generator network G based on the degraded underwater image.
[0022] Optionally, the underwater target detection device described in the third aspect may further include a first fusion module and a first training module.
[0023] The first acquisition module can also be used to acquire underwater background images and images of targets on the water.
[0024] The first fusion module is used to fuse the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate an underwater target image;
[0025] The first training module is used to train the YOLOv4 algorithm to convergence using the underwater target image.
[0026] Optionally, the first fusion module of the underwater target detection device described in the third aspect is specifically used to generate a first Laplacian pyramid corresponding to the background image; generate a second Laplacian pyramid corresponding to the surface target image; and fuse the first Laplacian pyramid and the second Laplacian pyramid to generate the underwater target image.
[0027] Furthermore, the technical effects of the underwater target detection device described in the third aspect can be referenced by the technical effects of the underwater target detection method described in the first aspect, and will not be repeated here.
[0028] Fourthly, a training device is provided. The training device includes: a second acquisition module, a second fusion module, and a second training module.
[0029] The second acquisition module is used to acquire underwater background images and images of targets on the water.
[0030] The second fusion module is used to fuse the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate an underwater target image.
[0031] The second training module is used to train the YOLOv4 algorithm to convergence using the underwater target image.
[0032] Fifthly, embodiments of this application provide an electronic device. The electronic device includes a processor and a memory; the memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the method described in any implementation of the first aspect, or to perform the method described in the second aspect.
[0033] In a sixth aspect, embodiments of this application provide a computer-readable storage medium, including: a computer program or instructions; when the computer program or instructions are run on a computer, the computer causes the computer to perform the method described in any possible implementation of the first aspect, or to perform the method described in the second aspect. Attached Figure Description
[0034] Figure 1 This is a flowchart illustrating an underwater target detection method.
[0035] Figure 2 This is one of the partial schematic diagrams of the GAN network mentioned in the underwater target detection method provided in the embodiments of this application;
[0036] Figure 3 A second partial schematic diagram of the GAN network mentioned in the underwater target detection method provided in the embodiments of this application;
[0037] Figure 4 A schematic diagram of the YOLOv4 network mentioned in the underwater target detection method provided in the embodiments of this application;
[0038] Figure 5 A schematic diagram illustrating the training method provided in an embodiment of this application;
[0039] Figure 6 This is a schematic diagram of the underwater target detection device provided in the embodiments of this application;
[0040] Figure 7 This is a schematic diagram of the structure of the training device provided in the embodiments of this application;
[0041] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0042] Reference numerals: 400-Underwater target detection device; 410-First acquisition module; 420-Enhancement module; 430-Detection module; 500-Training device; 510-Second acquisition module; 520-Second fusion module; 530-Second training module; 2000-Electronic device; 2001-Processor; 2002-Memory; 2003-Transceiver; 2004-Processor. Detailed Implementation
[0043] The technical solution in this application will now be described with reference to the accompanying drawings.
[0044] In the embodiments of this application, words such as "exemplarily" and "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as an "example" in this application should not be construed as being better or more advantageous than other embodiments or design options. Specifically, the use of the word "example" is intended to present the concept in a concrete manner. Furthermore, in the embodiments of this application, the meaning expressed by "and / or" can be both, or it can be either one or the other.
[0045] In the embodiments of this application, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.
[0046] In the embodiments of this application, sometimes the subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.
[0047] The following will combine Figure 1 This application provides a specific description of the underwater target detection method provided in its embodiments. For example, Figure 1 This is a flowchart illustrating the underwater target detection method provided in this application embodiment, specifically a process for detecting underwater targets.
[0048] like Figure 1 As shown, the underwater target detection method includes the following steps:
[0049] S110: Acquire the image to be detected, including underwater targets.
[0050] S120: Enhance the image to be detected to obtain an enhanced image.
[0051] S130: The enhanced image is detected using the fourth-generation target detection model YOLOv4 to obtain the target in the image to be detected.
[0052] Among them, the underwater targets included in the image to be detected are degraded underwater targets.
[0053] In the above process, since the image to be detected, including the degraded underwater target, is enhanced before target detection, an enhanced image including the clear underwater target can be obtained. This is beneficial for YOLOv4 to extract more accurate target features when performing target detection on the enhanced image. In addition, YOLOv4 has the advantages of fast target detection speed, high detection accuracy, and good robustness, so it can improve the overall accuracy of target detection.
[0054] The following will address... Figure 1 The implementation details of steps S120-130 are explained in detail.
[0055] Optionally, in some embodiments, when performing step S120: enhancing the image to be detected to obtain an enhanced image, it may include: inputting the image to be detected into a generative adversarial network (GAN), enhancing the image to be detected through the GAN, and outputting an enhanced image.
[0056] Specifically, Generative Adversarial Networks (GANs) are used to transform the image to be detected into an image of an underwater target, thereby enhancing the underwater target. For example, they can eliminate degradation problems such as color distortion, underexposure, and blurring of the underwater target to obtain an enhanced image.
[0057] Generative Adversarial Networks (GANs) can include a generator network G and a discriminator network D.
[0058] In some implementations, the generator network G may include convolutional layers and deconvolutional layers, such as Figure 2 As shown, it includes three convolutional layers and two deconvolutional layers.
[0059] In some implementations, the discriminator network D may include convolutional layers and activation function layers. For example... Figure 3 As shown, it includes five convolutional layers and one activation function layer.
[0060] The generator network G takes as input an image to be detected, including underwater targets. Generator network G first extracts image features of underwater targets from the image through its convolutional layers, then generates an image of surface targets (enhanced image) through its deconvolutional layers. This enhanced image is then fed into the discriminator network D as one of its inputs. It's worth noting that during the training of generator network G, all image pixel values are normalized to the range [-1, 1].
[0061] After acquiring the input image, the discriminator network D first extracts image features through convolutional layers, then performs binary classification using the sigmoid activation function, outputting the probability that the input image is a real water image. In other words, the discriminator network D is used to determine whether its input image is a real water target image or a fake water target image generated by the generator network G.
[0062] To ensure that Generative Adversarial Networks (GANs) can achieve the above functions, the corresponding loss function for GANs is:
[0063] in, For non-saturating GAN loss, Loss L1 (G) represents the L1 loss, Loss g (G) represents the gradient loss; λ1 is the first parameter, λ g This is the second parameter, which can be defined by the user.
[0064]
[0065] Loss L1 (G)=||xG(y)||1;
[0066]
[0067] in, The training set P, which represents the distribution of x-sampled free-water target images, is... train , The dataset P represents the distribution of underwater degraded images sampled from the generator network G. gen D(x) represents the probability that the discriminator network D will classify a real underwater target image as an underwater image, and G(y) represents the enhanced image generated by the generator network G based on the degraded underwater image; x represents the real underwater target image, and y is the degraded image to be detected that includes underwater targets.
[0068] Once the GAN network is trained to convergence, image enhancement can be performed on the image to be detected input to the generator network G, and the enhanced image can be output.
[0069] After obtaining the enhanced image, the enhanced image is used as input to YOLOv4 so that YOLOv4 can detect underwater targets.
[0070] Please refer to Figure 4 , Figure 4 A network structure diagram of YOLOv4 is shown.
[0071] YOLOv4 employs Mosaic data augmentation at the image input end.
[0072] Mosaic data augmentation involves stitching together four images using random scaling, cropping, and arrangement. Randomly scaling and then randomly distributing the images enriches the dataset. Furthermore, random scaling introduces many small objects, improving the overall robustness of the network. Secondly, Mosaic data augmentation directly computes the data from the four images, reducing the need for an excessively large mini-batch size. Good training results can be achieved even with a single GPU, meaning that Mosaic data augmentation reduces the GPU required for training, thus lowering training costs.
[0073] The backbone network of YOLOv4 can use CSPDarknet53.
[0074] CSPDarknet53 is a backbone network developed based on the YOLOv3 backbone network Darknet53 and drawing on the experience of CSPNet. Its core component is the CSP module.
[0075] The CSP module divides the feature maps of the base layer into two parts and then merges them through a cross-stage hierarchical structure, reducing computational cost while maintaining accuracy. Furthermore, the backbone network employs the Mish activation function and Dropback, which further enhances detection precision.
[0076] The neck structure of YOLOv4 uses SPP and PAN, which further enhances the feature extraction capability.
[0077] The loss function of YOLOv4 includes classification loss and bounding box regression loss, where the bounding box regression loss uses CIOU_Loss, and its calculation formula is as follows:
[0078]
[0079]
[0080]
[0081] In this context, IOU loss represents the difference in the intersection-union ratio (IU) between the predicted and target boxes. D2 represents the distance between the center points of the predicted and target boxes, Dc is the distance between the diagonals of the smallest bounding rectangles of the predicted and target boxes, v represents the difference between the aspect ratios of the target and predicted boxes, wgt and hgt represent the width and height of the target box, respectively, and w and t represent the width and height of the predicted box, respectively. Compared to the Smooth L1 loss method used in YOLOv3, CIOU_Loss can improve the accuracy of object detection.
[0082] YOLOv4 uses DIOU NMS instead of traditional NMS for bounding box selection, outperforming traditional NMS in detecting overlapping targets. The formula for calculating DIOU is:
[0083] The following section will introduce the process of training YOLOv4.
[0084] In some implementations, YOLOv4 can be trained by a device that performs underwater target detection.
[0085] In some implementations, real underwater and surface images can be acquired first and used as a training set to train YOLOv4 until convergence. However, in this approach, acquiring underwater images is time-consuming and costly, and real underwater image data is currently scarce and insufficient to support model training, making the model prone to overfitting.
[0086] To address this issue, in other implementations, underwater target images can be generated through image fusion, thereby enriching the training set.
[0087] Optionally, underwater background images and images of the target on the water can be acquired first.
[0088] The underwater background images can be taken using an underwater camera, and the number of images does not need to be large. The surface target images can be taken by the user using a camera, or a large number of images can be extracted from publicly available target detection datasets (such as COCO, VOC, etc.). With the underwater background images and surface target images prepared, the Laplacian pyramid fusion algorithm is used to fuse the underwater background images and surface target images to generate the underwater target images. Finally, the underwater target images are divided into training, validation, and test sets, and the YOLOv4 algorithm is trained on the training set until convergence.
[0089] Optionally, the underwater background image and the surface target image are fused using a Laplacian pyramid fusion algorithm to generate an underwater target image. This includes: generating a first Laplacian pyramid corresponding to the background image; generating a second Laplacian pyramid corresponding to the surface target image; and fusing the first and second Laplacian pyramids. Specifically, this involves reconstructing a Gaussian pyramid based on the first and second Laplacian pyramids, and interpolating the reconstructed Gaussian pyramid layer by layer from its top layer down until the final underwater target image is obtained.
[0090] In summary, combining Figures 1-4As can be seen from the underwater target detection method shown, in this embodiment of the application, before performing target detection on the image to be detected that includes underwater targets, the image to be detected that includes degraded underwater targets is first enhanced. Therefore, an enhanced image that includes clear underwater targets can be obtained. This is beneficial for YOLOv4 to extract more accurate target features when performing target detection on the enhanced image. In addition, YOLOv4 has the advantages of fast target detection speed, high detection accuracy, and good robustness. Therefore, it can improve the overall accuracy of target detection.
[0091] Furthermore, as mentioned above, the training process of the aforementioned fourth-generation target detection model YOLOv4 can be performed by the device executing the underwater target detection method. In other embodiments, the training process of YOLOv4 can also be performed by other third-party devices, so that the device executing the underwater target detection method can directly call the trained YOLOv4 to perform target detection.
[0092] Based on this, embodiments of this application also provide a training method for training the fourth-generation object detection model YOLOv4. For example... Figure 5 As shown, the training method may include the following steps:
[0093] S210: Acquire underwater background images and images of targets on the water.
[0094] S220: The underwater background image and the surface target image are fused using the Laplacian pyramid fusion algorithm to generate an underwater target image.
[0095] S230: Train the YOLOv4 algorithm until it converges using the underwater target image.
[0096] The specific implementation process and effects of this training method have been described in the relevant sections of the above-mentioned underwater target detection method, and will not be repeated here.
[0097] based on Figure 1-4 The underwater target detection method provided in the embodiments of this application is described in detail. Figure 5 The training method provided in the embodiments of this application has been described in detail. The following will describe in detail the virtual device corresponding to the underwater target detection method provided in the embodiments of this application, namely the underwater target detection device, and the virtual device corresponding to the training method provided in the embodiments of this application, namely the training device.
[0098] For example, Figure 6 This is a schematic diagram of the underwater target detection device 400 provided in an embodiment of this application. Figure 6As shown, the underwater target detection device 400 includes: a first acquisition module 410, an enhancement module 420, and a detection module 430.
[0099] For ease of explanation, Figure 6 Only the main components of the underwater target detection device 400 are shown.
[0100] The first acquisition module 410 is used to acquire an image to be detected, including underwater targets;
[0101] Enhancement module 420 is used to enhance the image to be detected to obtain an enhanced image;
[0102] The detection module 430 is used to detect the enhanced image using the fourth-generation target detection model YOLOv4 to obtain the target in the image to be detected.
[0103] Optionally, the enhancement module 420 in the underwater target detection device 400 can be specifically used to: input the image to be detected into a generative adversarial network (GAN), perform enhancement processing on the image to be detected through the GAN, and output the enhanced image.
[0104] Optionally, the GAN includes a generator network G and a discriminator network D, and the loss function of the GAN is: in, For non-saturating GAN loss, Loss L1 (G) represents the L1 loss, Loss g (G) represents the gradient loss; λ1 is the first parameter, λ g The second parameter; Loss L1 (G)=||xG(y)||1; The training set P, which represents the distribution of x-sampled free-water target images, is... train , The dataset P represents the distribution of underwater degraded images sampled from the generator network G. gen D(x) represents the probability that the discriminator network D will classify a real underwater target image as an underwater image, and G(y) represents the enhanced image generated by the generator network G based on the degraded underwater image; x represents the underwater target image, and y is the same degraded underwater image corresponding to x.
[0105] Optionally, the underwater target detection device 400 may further include a first fusion module and a first training module. Figure 6 (Not shown in the text).
[0106] The first acquisition module 410 can also be used to acquire underwater background images and images of targets on the water.
[0107] The first fusion module is used to fuse the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate an underwater target image;
[0108] The first training module is used to train the YOLOv4 algorithm to convergence using the underwater target image.
[0109] Optionally, the first fusion module in the underwater target detection device 400 can be used to generate a first Laplacian pyramid corresponding to the background image; generate a second Laplacian pyramid corresponding to the surface target image; and fuse the first Laplacian pyramid and the second Laplacian pyramid to generate the underwater target image.
[0110] Furthermore, the technical effects of the underwater target detection device 400 can be referenced from the technical effects of any of the aforementioned underwater target detection methods, and will not be elaborated here.
[0111] For example, Figure 7 This is a schematic diagram of the structure of the training device 500 provided in an embodiment of this application. Figure 7 As shown, the training device 500 includes: a second acquisition module 510, a second fusion module 520, and a second training module 530.
[0112] For ease of explanation, Figure 7 Only the main components of the training device 500 are shown.
[0113] The second acquisition module 510 is used to acquire underwater background images and images of targets on the water.
[0114] The second fusion module 520 is used to fuse the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate an underwater target image.
[0115] The second training module 530 is used to train the YOLOv4 algorithm to convergence using the underwater target image.
[0116] Furthermore, the technical effects of the training device 500 can be referred to the description of the aforementioned related embodiments, and will not be repeated here.
[0117] Optionally, embodiments of this application also provide a computer-readable storage medium, which includes a computer program or instructions that, when executed on a computer, cause the underwater target detection method or training method provided in any embodiment of this application to be executed.
[0118] Optionally, embodiments of this application also provide an electronic device for executing the underwater target detection method or apparatus provided in any embodiment of this application, or for executing the training method or apparatus provided in any embodiment of this application.
[0119] like Figure 8 As shown, electronic device 2000 may include processor 2001.
[0120] Optionally, the electronic device 2000 may also include a memory 2002 and / or a transceiver 2003.
[0121] The processor 2001 is coupled to the memory 2002 and the transceiver 2003, which can be connected via a communication bus.
[0122] The following is combined Figure 8 A detailed introduction to each component of the electronic device 2000:
[0123] The processor 2001 is the control center of the electronic device 2000. It can be a single processor or a collective term for multiple processing elements. For example, the processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).
[0124] Optionally, the processor 2001 can perform various functions of the electronic device 2000 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.
[0125] In a specific implementation, as one example, the processor 2001 may include one or more CPUs, for example... Figure 8 CPU0 and CPU1 are shown in the diagram.
[0126] In a specific implementation, as one example, the electronic device 2000 may also include multiple processors, for example... Figure 8The processors 2001 and 2004 are shown. Each of these processors can be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).
[0127] The memory 2002 is used to store the software program that executes the solution of this application, and is controlled by the processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.
[0128] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the processor 2001 or may exist independently and be accessible through the interface circuit of the electronic device 2000. Figure 8 (Not shown in the image) is coupled to the processor 2001, and this embodiment does not specifically limit this.
[0129] Transceiver 2003 is used for communication with other electronic devices. For example, if electronic device 2000 is an underwater robot, transceiver 2003 can be used to communicate with network devices or with another terminal device. As another example, if electronic device 2000 is a network device, transceiver 2003 can be used to communicate with terminal devices or with another network device.
[0130] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 8 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.
[0131] Optionally, the transceiver 2003 can be integrated with the processor 2001, or it can exist independently and be connected via the interface circuit of the electronic device 2000. Figure 8 (Not shown in the image) is coupled to the processor 2001, and this embodiment does not specifically limit this.
[0132] It should be noted that, Figure 8 The structure of the electronic device 2000 shown does not constitute a limitation on the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0133] Furthermore, the technical effects of the electronic device 2000 can be referred to the technical effects of the underwater target detection method described in the above method embodiments, and will not be repeated here.
[0134] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0135] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0136] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0137] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0138] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0139] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0141] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0142] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0144] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0145] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0146] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for detecting underwater targets, characterized in that, The method includes: Acquire images of the target object, including underwater targets; The image to be detected is enhanced to obtain an enhanced image; The enhanced image is detected using the fourth-generation target detection model YOLOv4 to obtain the target in the image to be detected; The step of enhancing the image to be detected to obtain an enhanced image includes: inputting the image to be detected into a generative adversarial network (GAN), enhancing the image to be detected through the GAN, and outputting the enhanced image. The GAN includes a generator network G and a discriminator network D. The generator network G includes three convolutional layers and two deconvolutional layers, and the discriminator network D includes five convolutional layers and one activation function layer. During the training of the generator network G, all image pixel values are normalized to the interval [-1, 1]. The loss function of the GAN is: ; in, For non-saturated GAN loss, For L1 loss, For gradient loss; As the first parameter, The second parameter; ; ; ; Characterization x The training set P consists of the distribution of sampled free-water target images. train , Characterization y The dataset P is composed of the distribution of underwater degraded images sampled from the generator network G. gen D( x G() represents the probability that the discriminator network D classifies a real water target image as a water image. y ) represents the enhanced image generated by the generator network G based on the degraded underwater image; Before detecting the enhanced image using the fourth-generation target detection model YOLOv4, the method further includes: acquiring an underwater background image and an above-water target image; fusing the underwater background image and the above-water target image using a Laplacian pyramid fusion algorithm to generate an underwater target image; and training YOLOv4 using the underwater target image until convergence. The step of fusing the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate the underwater target image includes: generating a first Laplacian pyramid corresponding to the background image; generating a second Laplacian pyramid corresponding to the surface target image; and fusing the first Laplacian pyramid and the second Laplacian pyramid to generate the underwater target image. The underwater target image is generated by fusing the first Laplacian pyramid with the second Laplacian pyramid. Specifically, a Gaussian pyramid is reconstructed based on the first and second Laplacian pyramids, and the reconstructed Gaussian pyramid is interpolated layer by layer from its top layer down until the final underwater target image is obtained.
2. A training method, characterized in that, The method for training the fourth-generation object detection model YOLOv4 as described in claim 1 includes: Acquire underwater background images and images of targets on the water; The underwater background image and the surface target image are fused using the Laplacian pyramid fusion algorithm to generate an underwater target image. The YOLOv4 algorithm was trained until convergence using the underwater target images.
3. An underwater target detection device, characterized in that, The underwater target detection device includes: The first acquisition module is used to acquire images of the target to be detected, including underwater targets; An enhancement module is used to enhance the image to be detected to obtain an enhanced image; The detection module is used to detect the enhanced image using the fourth-generation target detection model YOLOv4 to obtain the target in the image to be detected; The step of enhancing the image to be detected to obtain an enhanced image includes: inputting the image to be detected into a generative adversarial network (GAN), enhancing the image to be detected through the GAN, and outputting the enhanced image. The GAN includes a generator network G and a discriminator network D. The generator network G includes three convolutional layers and two deconvolutional layers, and the discriminator network D includes five convolutional layers and one activation function layer. During the training of the generator network G, all image pixel values are normalized to the interval [-1, 1]. The loss function of the GAN is: ; in, For non-saturated GAN loss, For L1 loss, For gradient loss; As the first parameter, The second parameter; ; ; ; Characterization x The training set P consists of the distribution of sampled free-water target images. train , Characterization y The dataset P is composed of the distribution of underwater degraded images sampled from the generator network G. gen D( x G() represents the probability that the discriminator network D classifies a real water target image as a water image. y ) represents the enhanced image generated by the generator network G based on the degraded underwater image; It also includes a first fusion module and a first training module. The first acquisition module can also be used to acquire an underwater background image and an above-water target image. The first fusion module is used to fuse the underwater background image and the above-water target image using the Laplacian pyramid fusion algorithm to generate an underwater target image. The first training module is used to train the YOLOv4 algorithm to convergence using the underwater target image. The step of fusing the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate the underwater target image includes: generating a first Laplacian pyramid corresponding to the background image; generating a second Laplacian pyramid corresponding to the surface target image; and fusing the first Laplacian pyramid and the second Laplacian pyramid to generate the underwater target image. The underwater target image is generated by fusing the first Laplacian pyramid with the second Laplacian pyramid. Specifically, a Gaussian pyramid is reconstructed based on the first and second Laplacian pyramids, and the reconstructed Gaussian pyramid is interpolated layer by layer from its top layer down until the final underwater target image is obtained.
4. A training device, characterized in that, The apparatus for training the fourth-generation object detection model YOLOv4 as described in claim 1 includes: The second acquisition module is used to acquire underwater background images and images of targets on the water. The second fusion module is used to fuse the underwater background image and the surface target image using the Laplacian pyramid fusion algorithm to generate an underwater target image. The second training module is used to train the YOLOv4 algorithm to convergence using the underwater target image.
5. An electronic device, characterized in that, include: Processor, the processor being coupled to memory; The processor is configured to execute a computer program stored in the memory, so that the electronic device performs the method as described in claim 1 or 2.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program or instructions that, when executed on a computer, cause the method as described in claim 1 or 2 to be performed.
Citation Information
Patent Citations
Underwater target detection method based on improved SSD and transfer learning
CN111209952A
Underwater sea cucumber detection method combined with image enhancement
CN112465803A
Underwater image enhancement method of generative adversarial network
CN113450286A
