Ship target detection method, device, equipment and medium based on improved YOLOX

Abstract

The present invention discloses a ship target detection method, device, equipment and medium based on improved YOLOX, the method obtains a sample data set, the sample data set includes ship annotated images and unannotated images, and the sample data set is divided into a training set and a test set; a YOLOX network is constructed, the backbone network of the YOLOX network is optimized, and the backbone network is replaced with ResNet50 for extracting small target features of images; the sample data set is input into the YOLOX network for training and testing to obtain a ship detection model; the image to be detected is input into the trained ship detection model for detection, so as to output the ship detection result. The present invention improves the traditional YOLO algorithm by methods such as network structure improvement and model simplification, so that the improved algorithm is better adapted to the ship target detection task, and assists ship inspection personnel to detect potential dangers early and make correct judgments.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a ship target detection method, device, equipment and medium based on improved YOLOX. Background Art

[0002] In recent years, ship systems are developing in the direction of intelligence, unmanned operation and systematization. Smart ships have become a new hot spot in the international maritime community. Smart ships combine new technologies such as artificial intelligence, big data, and cloud computing with the characteristics of safety, environmental protection, energy saving and high efficiency. Ships often face fog, high humidity and various sea conditions when sailing. At the same time, they must abide by the rules of navigation at sea, and these difficulties put higher requirements on the perception ability of ships. Therefore, the perception system of smart ships is a key part of connecting with the outside world. my country has a long coastline and many marine supervision tasks. Illegal fishing, violations of sea areas and other issues pose hidden dangers to marine safety. Therefore, it is very necessary to conduct real-time and rapid detection of ship targets.

[0003] For maritime target detection, traditional detection methods are mainly based on images and radar. Mohammed, Zabidi and others built an embedded vision system to use enhancement and smoothing methods for surface targets that may be affected by sunlight reflection, extracted the Hu invariant moment features of four types of ships, and used different neural networks to identify the ships. Predecessors also proposed a moving target detection method that combines background and inter-frame difference methods. However, traditional methods have low detection rates and poor generalization capabilities in extremely complex marine environments; therefore, research on fast real-time algorithms, high precision and high reliability has become a hot topic. Maritime target detection based on deep learning has solved this problem well. Compared with traditional recognition methods, deep learning has a stronger feature expression capability for targets.

[0004] In recent years, convolutional neural networks have made great progress in the field of target detection. Maritime target detection based on deep learning has solved this problem well. Compared with traditional recognition methods, deep learning has a stronger feature expression ability for targets. There are two main types: the region-based Ross R-CNN series algorithm proposed by Girshick et al. The YOLO series algorithm based on regression ideas proposed by Redmon et al. R-CNN first generates candidate regions, then inputs them into the CNN convolutional neural network for feature extraction, compares the features with the sample features, and then determines the location of the target area. This algorithm has high detection accuracy, but slow calculation speed. The V3 algorithm does not generate candidate boxes and directly converts the detection into a regression problem. Only one detection is required to obtain the target position, which improves the detection speed and meets the real-time detection requirements. In the actual detection process, ships are often far away from the coastline, and there are many small-sized ship images on the sea. The current YOLO series of algorithms have poor effects in small-scale applications and there is a phenomenon of missed detection. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a ship target detection method, device, equipment and medium based on improved YOLO to solve the difficulty in identifying densely packed small-sized ships, an anchor-free implementation method of the YOLO algorithm, and the problems of susceptibility to external interference, poor small target detection effect and high small target misrecognition rate in the ship target detection process. The traditional YOLO algorithm is improved by improving the network structure, simplifying the model and other methods, so that the improved algorithm can better adapt to the ship target detection task and assist ship inspection personnel to detect potential dangers as early as possible and make correct judgments.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] The ship target detection method based on improved YOLOX includes the following steps:

[0008] Acquire a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set;

[0009] Build a YOLOX network, optimize the backbone network of the YOLOX network, and replace the backbone network with ResNet50 to extract small object features in the image;

[0010] The sample data set is input into the YOLOX network for training and testing to obtain a ship detection model;

[0011] The image to be detected is input into the trained ship detection model for detection, and the ship detection result is output.

[0012] Preferably, the method further includes the following steps: using CUTMIX to increase the sample data set and then perform network training and testing to improve the image feature extraction effect of the network model.

[0013] Preferably, when the training set is input into the YOLOX network for training, Class=1 is set, and the detected targets are divided into two categories: ships and backgrounds, and the number of targets in the largest number of images is set to 10.

[0014] Preferably, the output of the YOLOX network is connected to the CFE module, and the CFE module includes two branches, wherein the left branch includes: convolution layers with convolution kernel sizes of 1*k and k*1, and convolutions of 1*k and k*1 are used to perform convolution and operations on the input data respectively at the same time; the right branch includes: two convolution layers with convolution kernel sizes of k*1 and 1*k, and convolutions of k*1 and 1*k are used to perform convolution and operations on the input data respectively at the same time.

[0015] Preferably, the sample data set is arranged in the form of a VOC data set.

[0016] Preferably, the sample data set and the image to be identified are preprocessed, and the preprocessing includes random horizontal or vertical flipping, cropping, and scale transformation.

[0017] Preferably, the backbone network of the YOLOX network is replaced by Hourglass.

[0018] The ship target detection device based on improved YOLOX includes: an acquisition module, an image processing module, a model acquisition module and a recognition module, wherein:

[0019] The acquisition module is used to acquire a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set;

[0020] The image processing module is used to pre-process the sample data set;

[0021] The model acquisition module is used to build a YOLOX network, the backbone network of the YOLOX network is replaced with ResNet50 to extract small target features from images, and to input sample data sets into the YOLOX network for training and testing to obtain a ship detection model;

[0022] The recognition module is used to input the image to be detected into the trained ship detection model for detection, so as to output the ship detection result.

[0023] A computer device comprises: a memory for storing a computer program; and a processor for implementing any of the above-mentioned ship target detection methods based on improved YOLOX when executing the computer program.

[0024] A readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the ship target detection method based on the improved YOLOX as described above is implemented.

[0025] Based on the above technical scheme, the beneficial effects of the present invention are as follows: with the continuous development of artificial intelligence technology, deep learning methods have begun to penetrate into various fields, and intelligent supervision of the ocean is urgently needed for us. my country has rich marine resources and vast sea areas. At the same time, the marine economy is also one of the main support points for human development in the 21st century. Due to the huge risks of marine operations, ship target detection has important research significance in both military and civilian applications. Therefore, it is necessary to study the rapid identification of marine targets. In the present invention, we use the latest detection technology YOLOX, which has the highest detection accuracy and speed of natural images. On this basis, we improve the network structure of YOLOX, delete the strategy, and change the data set, so that YOLOX is more suitable for ship detection in the ocean, and it is easier to deploy YOLOX. This method can be used for monitoring port channels, which is convenient for intelligent monitoring of fishing boats and cargo ships. At the same time, it has broad application prospects in port management, cross-border ship detection, autonomous ships, safe navigation, etc. In practical applications, our algorithm is a better trade-off than YOLO v3 and YOLO v4, YOLO v5 networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is an application environment diagram of a ship target detection method based on an improved YOLOX in an embodiment;

[0027] Figure 2 is a flow chart of a ship target detection method based on improved YOLOX in an embodiment;

[0028] Figure 3 is a model framework diagram of a ship target detection method based on improved YOLOX in an embodiment;

[0029] Figure 4 is a structural schematic diagram of a ship target detection device based on improved YOLOX in an embodiment;

[0030] Figure 5 The figure is a diagram of the internal structure of a computer device in an embodiment. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0032] The ship target detection method based on improved YOLOX provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Figure 1As shown, the application environment includes a computer device 110. The computer device 110 can obtain a sample data set; the computer device 110 can build a YOLOX network, optimize the backbone network of the YOLOX network, and replace the backbone network with ResNet50 to extract small target features of the image; the computer device 110 can input the sample data set into the YOLOX network for training and testing to obtain a ship detection model; the computer device 110 can input the image to be detected into the trained ship detection model for detection to output the ship detection result.

[0033] like Figure 2 As shown, a ship target detection method based on improved YOLOX is provided, which specifically includes the following steps:

[0034] Step 210: Obtain a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set.

[0035] Step 220: construct a YOLOX network, optimize the backbone network of the YOLOX network, and replace the backbone network with ResNet50 to extract small object features from the image.

[0036] In this implementation, the YOLOX-Darknet53 network is used as the backbone network of the detector. Due to the poor small target detection effect in the YOLOV3 algorithm and the slightly slower speed than the current first-stage detector, there is still room for improving the model performance by introducing it into YOLOX. To address this problem, Figure 3 As shown in the figure, we use ResNet50 to replace the backbone network to improve the accuracy of model detection. In addition, the ResNet series has great optimization power and can well meet the needs of actual deployment.

[0037] Adjust the data increase and decrease strategy in the YOLOX-Darknet53 network to CUTMIX. Using the CUTMIX data enhancement strategy, the sample data set is increased before network training and testing to improve the network model's extraction of image features. This strategy will prevent non-information pixels from appearing during training, thereby improving training efficiency. By requiring the model to recognize objects from local views and adding information about other samples to the cut area, the model's positioning ability can be further enhanced. The details are as follows:

[0038] x A and x B are two different training samples, y A and B is the corresponding label value. CUTMIX needs to generate new training samples and corresponding labels: and The formula is as follows:

[0039]

[0040]

[0041] M∈{0,1} W×H It is a binary mask for dropping some areas and filling them. is pixel-by-pixel multiplication, l is a binary mask with all elements equal to 1, and λ, like Mixup, belongs to the Beta distribution: λ∈Beta(α,α), and if α=1, λ obeys the uniform distribution of (0, 1).

[0042] In order to sample the binary mask M, we first need to sample the bounding box B of the cropped region = (r x ,r y ,r w ,r h ) is used to sample the sample x A and x B Do the indication calibration of the cropping area. Sample the rectangular mask M (the length and width are proportional to the sample size).

[0043] The bounding box sampling formula for the clipping area is as follows:

[0044]

[0045]

[0046] Where W is the bounding box width, H is the bounding box height, and Unif() is the same function. Ensure that the proportion of the clipping area is After determining the clipping area B, set the clipping area B in the mask M to 0 and the other areas to 1. The sampling of the mask is completed, and then the clipping area B in sample A is removed, and the clipping area B in sample B is clipped and then filled into sample A.

[0047] Step 230: input the sample data set into the YOLOX network for training and testing to obtain a ship detection model.

[0048] In this embodiment, the number of training set classifications is set. Since the classification problem should be simplified for marine ship detection, we will set Class = 1 in the training set, and divide the detected targets into two categories: ships and backgrounds. At the same time, only images with ship annotations are retained in the data set. At the same time, increasing the Batch can improve the efficiency of model training.

[0049] Since ships are large in size, there are generally few ships in an image, so we set the number of detections to 10, and max_labels represents the number of objects in the image. Since our dataset imitates the layout of the VOC dataset file and does not have year information, the year element will be deleted.

[0050] Step 240: input the image to be detected into the trained ship detection model for detection, so as to output the ship detection result.

[0051] In this example, we will set up an end-to-end YOLOX, adding two additional conv layers, one-to-one label assignment, and stop gradient to the model. This enables the detector to execute in an end-to-end manner, but will slightly reduce performance and inference speed, so I will improve these two conv layers. We will replace the two conv layers with CFE modules. Here, we borrow the CFE module in the CFEnet network, replace the original 1x1conv2d structure, enrich the information extraction, and obtain better results than the original algorithm. The CFE module is designed to enhance the shallow characteristics of SSD to detect small targets. It is inspired by modules such as Inception, Xception, Large Separable, and ResNeXt. The CFE module includes two branches, where the left branch includes: convolution layers with convolution kernel sizes of 1*k and k*1, using 1*k and k*1 convolutions to perform convolution and operations on the input data at the same time; the right branch includes: two convolution layers with convolution kernel sizes of k*1 and 1*k, using k*1 and 1*k convolutions to perform convolution and operations on the input data at the same time.

[0052] In one embodiment, for ResNet50, we can also replace the network with Hourglass as the backbone network for feature extraction. Hourglass is currently mainly used in the direction of posture detection. The design of this network is mainly due to the need to capture information at each scale. For example, some local information is important for identifying some features (such as faces, hands, etc.). For ship detection, its local information is essential for identifying the shape features of the bottom of the ship. To identify a target, you need to have a good understanding of the overall target, which requires grasping a lot of local feature information and combining them. For a ship, as long as you grasp its bottom features, you can complete a good recognition. At the same time, hourglass is a simple, minimized design that has the ability to capture all feature information and make the final pixel-level prediction. The network at this step can achieve an anchor-free method. It simultaneously predicts the position of the lower left corner and upper right corner of a target image, gives a bounding box based on the corner points, and then regresses the offset distance of the corner points of the bounding box step by step according to the loss function defined by YOLOX, and uses YOLOX's Decoupled Head to complete the classification task and regression task. For propagation detection, the target type to be detected is single, and the data enhancement module of YOLOX can also be cut off, so as not to cause a decrease in the accuracy of the trained model. At the same time, the training efficiency is improved, and the deployment of YOLOX in ship detection is completed more quickly.

[0053] In one embodiment, Figure 4 As shown, a ship target detection device 300 based on improved YOLOX is provided, comprising: an acquisition module 310, an image processing module 320, a model acquisition module 330 and a recognition module 340, wherein:

[0054] The acquisition module 310 is used to acquire a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set;

[0055] The image processing module 320 is used to pre-process the sample data set;

[0056] The model acquisition module 330 is used to build a YOLOX network, the backbone network of which is replaced by ResNet50 to extract small target features from images, and to input sample data sets into the YOLOX network for training and testing to obtain a ship detection model;

[0057] The recognition module 340 is used to input the image to be detected into the trained ship detection model for detection, so as to output the ship detection result.

[0058] The devices or modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0059] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a ship target detection method based on an improved YOLOX is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0060] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0061] In one embodiment, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the ship target detection method based on the improved YOLOX as described in any one of the above is implemented.

[0062] Those skilled in the art can understand that the implementation of all or part of the processes in the ship target detection method based on the improved YOLOX in the above-mentioned embodiment can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0063] The above are only preferred embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.

Claims

1. The ship target detection method based on improved YOLOX is characterized by: The steps include: Acquire a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set; Build a YOLOX network, optimize the backbone network of the YOLOX network, and replace the backbone network with ResNet50 to extract small object features in the image; The sample data set is increased by using CUTMIX and then input into the YOLOX network for training and testing to obtain a ship detection model; when the training set is input into the YOLOX network for training, Class=1 is set, and the detected targets are divided into two categories: ships and backgrounds. At the same time, the number of targets with the largest number of images is set to 10, and the two conv layers of the YOLOX network are replaced by CFE modules. The output of the YOLOX network is connected to the CFE module, and the CFE module includes two branches, wherein the left branch includes: convolution layers with convolution kernel sizes of 1*k and k*1, and convolutions of 1*k and k*1 are used to perform convolution and operations on the input data respectively at the same time; the right branch includes: two convolution layers with convolution kernel sizes of k*1 and 1*k, and convolutions of k*1 and 1*k are used to perform convolution and operations on the input data respectively at the same time; The image to be detected is input into the trained ship detection model for detection, and the ship detection result is output.

2. The ship target detection method based on improved YOLOX according to claim 1, characterized in that: The sample data set is arranged in the form of a VOC data set.

3. The ship target detection method based on improved YOLOX according to claim 1, characterized in that: The sample data set and the image to be identified are preprocessed, and the preprocessing includes random horizontal or vertical flipping, cropping, and scale transformation.

4. The ship target detection method based on improved YOLOX according to claim 1, characterized in that: The backbone network of the YOLOX network is replaced by Hourglass.

5. The ship target detection device based on improved YOLOX is characterized by: include: acquisition module, image processing module, model acquisition module and recognition module, wherein, The acquisition module is used to acquire a sample data set, wherein the sample data set includes annotated images of ships and unannotated images, and the sample data set is divided into a training set and a test set; The image processing module is used to pre-process the sample data set; The model acquisition module is used to build a YOLOX network, the backbone network of the YOLOX network is replaced with ResNet50 to extract small target features from images, and is used to increase the sample data set by using CUTMIX and input it into the YOLOX network for training and testing to obtain a ship detection model; when the training set is input into the YOLOX network for training, Class=1 is set, and the targets detected are divided into two categories: ships and backgrounds, and the maximum number of targets in the image is set to 10, and the two conv layers of the YOLOX network are replaced with a CFE module, and the output of the YOLOX network is connected to the CFE module, and the CFE module includes two branches, wherein the left branch includes: convolution layers with convolution kernel sizes of 1*k and k*1, and convolutions of 1*k and k*1 are used to perform convolution and operations on the input data at the same time; the right branch includes: two convolution layers with convolution kernel sizes of k*1 and 1*k, and convolutions of k*1 and 1*k are used to perform convolution and operations on the input data at the same time; The recognition module is used to input the image to be detected into the trained ship detection model for detection, so as to output the ship detection result.

6. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the ship target detection method based on improved YOLOX as described in any one of claims 1 to 4 when executing the computer program.

7. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by the processor, the ship target detection method based on the improved YOLOX as described in any one of claims 1 to 4 is implemented.

Images