Navigation radar ship target detection method based on Yolox

Abstract

The invention discloses a navigation radar ship target detection method based on Yolox, and the method comprises the steps: carrying out the data arrangement, and obtaining an image sample of a radar for a sea surface ship in a sea detection data set; preprocessing and marking the ship image; constructing a training data set for training, a verification data set for verification and a test data set for testing; introducing an attention mechanism into the original Yolox model to extract effective features; feature extraction and weight updating of the data set are carried out based on an SWT-Yolox model; and performing ship target detection, obtaining related information and outputting a detection result. According to the method, optimization and improvement are carried out based on an original Yolox model, an attention mechanism structure is introduced, the advantages of the attention mechanism structure are introduced into YoloX target identification, the feature extraction efficiency and effect are improved, and the radar image ship identification and positioning effect in a complex sea area is achieved.

Description

Technical Field

[0001] The invention belongs to the field of navigation radar, and in particular relates to a navigation radar ship target detection method based on Yolox. Background Art

[0002] Detection of sea surface targets usually relies on sea radar, sonar and infrared / optical imaging technology to discover the electromagnetic, scattering, acoustic and optical characteristics of the target. Among them, radar observation of the sea surface to detect targets is widely used in regional marine target supervision scenarios, and the corresponding radar comprehensive detection and information processing technology is the research focus of various countries, marine science and technology institutions and marine enterprises. With the rapid development of modern science and technology, water shipping safety supervision is gradually moving towards informatization, networking and intelligence. The development of the shipping industry has increased the types and number of ships, the traffic density in the sea area and the loading volume of dangerous goods, which is easy to cause marine accidents and seriously threaten the navigation safety of ships and the marine ecological environment. Therefore, the effective and timely tracking and prediction of the navigation trajectory of ships in the monitored sea area is an important technical support for maritime traffic accident warning, and it is also of great significance to the navigation safety of ships and the marine ecological environment.

[0003] How to accurately and effectively locate the position of the ship from the radar image is particularly important. The proposal and application of target detection technology based on deep learning has achieved a better target detection task. This idea extracts and processes radar image features by selecting a deep learning model, and locates and identifies the target ship based on the extracted features.

[0004] However, the existing research methods have low recognition rate and accuracy in complex sea areas and fail to achieve the task of identifying target ships through radar images. Summary of the invention

[0005] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides a Yolox-based navigation radar ship target detection method, which can improve the accuracy of radar image ship recognition and positioning in complex sea areas.

[0006] Technical solution: To achieve the above purpose, the technical solution of the present invention is as follows:

[0007] A Yolox-based navigation radar ship target detection method comprises the following steps:

[0008] S1: Data collation, obtaining image samples of surface ships in the radar sea detection data set;

[0009] S2: preprocessing and labeling of ship images;

[0010] S3: Construct a training dataset for training, a validation dataset for validation, and a test dataset for testing;

[0011] S4: Introducing the attention mechanism into the original Yolox model to extract effective features;

[0012] S5: Feature extraction and weight update of data set based on SWT-Yolox model;

[0013] S6: Perform ship target detection, obtain relevant information and output the detection results.

[0014] Furthermore, in step S4, a Swin-Transformer network structure is embedded in the original Yolox model to form a SWT-Yolox model, wherein the Swin-Transformer network structure includes an attention mechanism.

[0015] Furthermore, deep feature extraction and up- and down-sampling feature fusion are performed through the upper layer features, and the feature matrix is ​​output for processing in the following layers;

[0016] According to the upper feature matrix, the feature points are judged, the regression parameters of each feature point are determined, and whether there is an object corresponding to the feature point is determined to achieve classification and regression.

[0017] Further, a window attention mechanism is used in the weight update of step S5, wherein the window attention mechanism includes a window multi-head self-attention mechanism and a sliding window multi-head self-attention mechanism;

[0018] The window attention mechanism limits the calculation of attention to multiple divided windows, calculates attention and assigns weights only between image blocks in the same window, adds position information to the calculation process of the attention mechanism, and the position information will be updated as the window attention mechanism is calculated.

[0019] Furthermore, the calculation formula of the window attention mechanism is:

[0020]

[0021] Among them, Attention(Q,K,V) represents the window attention mechanism; Q, K, and V are Query, Key, and Value, respectively, representing the query vector, key vector, and value vector. Softmax is a normalization function that normalizes the similarity to obtain a normalized weight matrix. In the matrix, the larger the weight of a value, the higher the similarity. T It represents the inner product, d is the dimension of vector K, and B represents the bias.

[0022] Furthermore, in step S5, the quality of the network is determined by the obtained loss and mAP.

[0023] Furthermore, after the optimal training model is obtained according to step S5, target detection of ships in the radar image to be processed is performed according to the obtained training model, feature points are judged by classifiers and regressors, redundant and overlapping prediction boxes are removed by score screening and non-maximum suppression, and target detection information is output.

[0024] Furthermore, the score screening is used to screen out prediction boxes whose scores satisfy the confidence level; and the non-maximum suppression is used to screen out boxes with the largest scores belonging to the same category within a certain area.

[0025] Furthermore, the process of score screening and non-maximum suppression includes:

[0026] Find the frames in the image whose scores are greater than the threshold function, and filter the scores before filtering the overlapping frames to reduce the number of frames;

[0027] Loop through the categories and perform non-maximum suppression on each category;

[0028] Sort the categories from largest to smallest according to the scores;

[0029] Each time, the frame with the largest score is taken out, and the degree of overlap with all other predicted frames is calculated. Those with too large overlap are removed.

[0030] Furthermore, in the step S2, the collected image is labeled with the sea surface ship target using Label Image.

[0031] Beneficial effects: The present invention optimizes and improves the original Yolox model, introduces the attention mechanism structure, introduces its advantages into YoloX target recognition, improves the feature extraction efficiency and effect, and achieves the recognition and positioning effect of radar image ships in complex sea areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Attached Figure 1 is a flow chart of the present invention;

[0033] Attached Figure 2 This is a diagram of data set analysis results according to an embodiment of the present invention;

[0034] Attached Figure 3 This is a training effect diagram of an embodiment of the present invention;

[0035] Attached Figure 4 This is a detection effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The terms used in the present disclosure are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. The singular forms "a", "said" and "the" used in the present disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0038] A Yolox-based navigation radar ship target detection method comprises the following steps:

[0039] S1: Data collation, obtaining image samples of surface ships in the radar sea detection data set;

[0040] S2: preprocessing and labeling of ship images;

[0041] S3: Construct a training dataset for training, a validation dataset for validation, and a test dataset for testing;

[0042] S4: Optimize the original Yolox model for training: Introduce the attention mechanism into the original Yolox model to extract effective features;

[0043] S5: Feature extraction and weight update of data set based on SWT-Yolox model;

[0044] S6: Perform ship target detection, obtain relevant information and output the detection results.

[0045] Among them, data sorting, obtaining image samples of surface ships in the radar sea detection data set; preprocessing and labeling the ship images are all conventional means in the prior art and will not be described in detail in this solution.

[0046] In some embodiments, in step S2, Label Image is used to label at least the following sea surface ship targets on the collected image.

[0047] In step S3, a Yolo image dataset in VOC format is constructed by using the radar images to be processed and the corresponding ship coordinate information. The dataset includes several radar images and the coordinate information of the ship contained in each image. By processing the above data, a corresponding number of image information datasets are obtained. The above image and information datasets are divided into training, verification and test datasets, and corresponding documents are generated for easy calling.

[0048] In step S4, SWT (Swin-Transformer) is embedded in the basic Yolox model to obtain a training model, and the CSPDarknet backbone network in the original Yolox model is replaced with the Swin-Transformer structure as the backbone network to extract effective features, and obtain three feature layers for the next step of network construction. The FPN in the backbone network has the same structure as the original Yolox network. Deep feature extraction and up and down sampling feature fusion are performed through the upper-layer features, and the three-part feature matrix is ​​output for the following layer processing. The Yolo Head in the backbone network has the same structure as the original Yolox network. The feature points are judged according to the upper-layer feature matrix to determine whether there is an object corresponding to the feature point, so as to achieve classification and regression.

[0049] The attention mechanisms used by SWT are the window multi-head self-attention mechanism and the sliding window multi-head self-attention mechanism. These two attention mechanisms limit the calculation of attention to multiple divided windows, and only the image blocks in the same window will calculate attention and assign weights. The position information is added to the calculation process of the attention mechanism, and the position information will be updated as the attention mechanism is calculated.

[0050] The calculation formula of the window attention mechanism is:

[0051]

[0052] Among them, Attention(Q,K,V) represents the window attention mechanism; Q, K, and V are Query, Key, and Value, respectively, representing the query vector, key vector, and value vector. Softmax is a normalization function that normalizes the similarity to obtain a normalized weight matrix. In the matrix, the larger the weight of a value, the higher the similarity. T It represents the inner product, d is the dimension of vector K, and B represents the bias.

[0053] The YoloX Head in the backbone network is used to classify and regress the extracted feature matrix: Reg, Obj, and Cls are used to obtain the predicted box and score. The functions of each part are as follows:

[0054] Reg is used to determine the regression parameters of each feature point. After the regression parameters are adjusted, the prediction box can be obtained.

[0055] Obj is used to determine whether each feature point contains an object.

[0056] Cls is used to determine the type of object contained in each feature point.

[0057] In step S5, the training process extracts features and updates weights of the data set obtained in step S3 through the network obtained in step S4, and determines the quality of the network through the obtained loss and mAP. Preferably, the optimizer selects SGD, and the learning rate is 0.01 to 0.0001.

[0058] In step S6, after obtaining the optimal training model according to step S5, the target detection of ships in the radar image to be processed is performed according to the obtained model, and the feature points are judged by the classifier and the regressor. The redundant and overlapping prediction boxes are removed by score screening and non-maximum suppression, and the image of the target detection, the frame coordinates, the processing time, the processing FPS, the number of identified ships and other information are output.

[0059] After obtaining the final prediction results, score sorting and non-maximum suppression screening are performed. Score screening is to screen out the prediction boxes whose scores meet the confidence level. Non-maximum suppression is to screen out the boxes with the largest scores of the same type in a certain area. The process of score screening and non-maximum suppression can be summarized as follows:

[0060] Find the boxes in the image whose scores are greater than the threshold function. Sorting the scores before filtering the overlapping boxes can significantly reduce the number of boxes.

[0061] Cycling over categories, the role of non-maximum suppression is to filter out the boxes with the largest scores belonging to the same category in a certain area. Cycling over categories can help us perform non-maximum suppression on each category separately.

[0062] Sort the categories from largest to smallest based on their scores.

[0063] Each time, the frame with the largest score is taken out and the degree of overlap with all other predicted frames is calculated. Those with too large overlap are removed.

[0064] Yolox was improved to obtain the deep learning network model SWT-Yolox. Swin-Transformer includes window attention mechanism, sliding window attention mechanism, relative position encoding, downsampling and channel number adjustment processing modules, which can reduce the computational complexity and comprehensiveness of position information, and adjust resolution and channel number problems in traditional image transformer processing.

[0065] One specific embodiment includes:

[0066] like Figure 1 As shown, it can be divided into six steps: data organization, image annotation, data set construction, network construction, network training, and network detection.

[0067] Use Label Image to annotate at least the sea surface ship targets on the collected images. Specifically, add a numerical label to each image sample of the obtained ship target image sample, and the annotation format is XML format. For example, the image without ship targets is added with a value of "0" and the image with ships is added with a value of "1". Correct annotation content can ensure that the trained model has a good effect in actual operation. The data analysis results are as follows Figure 2 shown.

[0068] The Yolo image dataset in xml format is constructed by using the radar images to be processed and the corresponding ship coordinate information. The dataset includes 996 radar images in .mat format and the coordinate information of the ship in each image. By processing the above data, 996 image information datasets in .xml format are obtained, corresponding to 996 radar images respectively. The above image and information datasets are divided into training, verification and test datasets, including: 617 training datasets, 155 verification datasets, and 194 test datasets, and corresponding documents are generated for easy calling.

[0069] SWT (Swin-Transformer) is embedded in the basic Yolox model to obtain a training model, and the CSPDarknet backbone network in the original Yolox model is replaced with the Swin-Transformer structure as the backbone network to extract effective features, and obtain three feature layers for the next step of network construction. FPN has the same network structure as the original Yolox network. It extracts deep features and fuses up and down sampling features through upper-layer features, and outputs three-part feature matrices for processing in the following layers. Yolo Head has the same network structure as the original Yolox network. It judges feature points according to the upper-layer feature matrix to determine whether there is an object corresponding to the feature point, and realizes classification and regression. In this embodiment, the backbone network structure is improved with reference to the Yolox network structure, and the deep learning network model SWT-Yolox used for radar target fusion is constructed.

[0070] The training process uses the acquired network to extract features and update weights of the acquired data set, and uses the acquired loss and mAP to determine the quality of the network. The optimizer selects SGD, and the learning rate is 0.01 to 0.0001. Figure 3 As shown in the figure, the map value becomes larger and larger during the training process, and the loss value becomes smaller and smaller.

[0071] After obtaining the optimal training model, the target detection of ships in the radar image to be processed is carried out according to the obtained model, and the feature points are judged by the classifier and regressor. The redundant and overlapping prediction boxes are removed through score screening and non-maximum suppression, and the image of target detection, frame coordinates, processing time, processing FPS, number of identified ships and other information are output.

[0072] like Figure 4 As shown, the method of the embodiment of the present invention can identify the target information of sea surface ships, and the performance and recognition accuracy are significantly improved compared with the original Yolox.

[0073] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0074] In the description of the present invention, it is necessary to understand that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0075] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A Yolox-based navigation radar ship target detection method, characterized in that: The following steps are involved: S1: Data collation, obtaining image samples of surface ships in the radar sea detection data set; S2: preprocessing and labeling of ship images; S3: Construct a training dataset for training, a validation dataset for validation, and a test dataset for testing; S4: Introducing the attention mechanism into the original Yolox model to extract effective features; S5: Feature extraction and weight update of data set based on SWT-Yolox model; S6: Perform ship target detection, obtain relevant information and output the detection results.

2. The Yolox-based navigation radar ship target detection method according to claim 1, characterized in that: In step S4, a Swin-Transformer network structure is embedded in the original Yolox model to form a SWT-Yolox model, wherein the Swin-Transformer network structure includes an attention mechanism.

3. The Yolox-based navigation radar ship target detection method according to claim 2, characterized in that: The upper layer features are used to extract deep features and fuse up and down sampling features, and the feature matrix is ​​output for processing in the following layers; According to the upper feature matrix, the feature points are judged, the regression parameters of each feature point are determined, and whether there is an object corresponding to the feature point is determined to achieve classification and regression.

4. The Yolox-based navigation radar ship target detection method according to claim 1, characterized in that: In the weight update of step S5, a window attention mechanism is used, wherein the window attention mechanism includes a window multi-head self-attention mechanism and a sliding window multi-head self-attention mechanism; The window attention mechanism limits the calculation of attention to multiple divided windows, calculates attention and assigns weights only between image blocks in the same window, adds position information to the calculation process of the attention mechanism, and the position information will be updated as the window attention mechanism is calculated.

5. The Yolox-based navigation radar ship target detection method according to claim 4, characterized in that: The calculation formula of the window attention mechanism is: Among them, Attention(Q,K,V) represents the window attention mechanism; Q, K, and V are Query, Key, and Value, respectively, representing the query vector, key vector, and value vector. Softmax is a normalization function that normalizes the similarity to obtain a normalized weight matrix. In the matrix, the larger the weight of a value, the higher the similarity. T It represents the inner product, d is the dimension of vector K, and B represents the bias.

6. The Yolox-based navigation radar ship target detection method according to claim 1, characterized in that: In step S5, the quality of the network is determined based on the obtained loss and mAP.

7. The Yolox-based navigation radar ship target detection method according to claim 1, characterized in that: After the optimal training model is obtained according to step S5, the target detection of the ship in the radar image to be processed is performed according to the obtained training model, the feature points are judged by the classifier and the regressor, the redundant and overlapping prediction boxes are removed by score screening and non-maximum suppression, and the target detection information is output.

8. The Yolox-based navigation radar ship target detection method according to claim 7, characterized in that: The score screening is used to screen out the prediction boxes whose scores meet the confidence level; the non-maximum suppression is used to screen out the boxes with the largest scores belonging to the same category in a certain area.

9. The Yolox-based navigation radar ship target detection method according to claim 8, characterized in that: The process of score screening and non-maximum suppression includes: Find the frames in the image whose scores are greater than the threshold function, and filter the scores before filtering the overlapping frames to reduce the number of frames; Loop through the categories and perform non-maximum suppression on each category; Sort the categories from largest to smallest according to the scores; Each time, the frame with the largest score is taken out, and the degree of overlap with all other predicted frames is calculated. Those with too large overlap are removed.

10. The Yolox-based navigation radar ship target detection method according to claim 1, characterized in that: In the step S2, the collected image is labeled with the sea surface ship target using Label Image.

Images