A method for detecting key parts of rigid objects based on local correlation voting mechanism

Abstract

The present invention discloses a method for detecting key parts of rigid targets based on a local association voting mechanism. The method first collects image domain data and spatial domain data with key parts annotated, and then adds a high-resolution channel module as an image domain detection model based on the YOLOv5 network structure. A local association voting network is constructed as a spatial domain detection model. Then, the image to be detected is input into the image domain detection model to determine whether there is an occluded target that is missed in the detection result of the image domain. When there is a missed detection, the key part position information of the image domain detection result is input into the spatial domain detection model, and the occluded target is predicted using a voting mechanism based on local association, and finally the information of all key parts of the rigid target is solved to obtain the detection result. On the basis of ensuring the timeliness of detection, this method solves the problem of occlusion of key parts of rigid targets that may occur in complex actual environments.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, and relates to key part detection of rigid targets, and in particular to a key part detection method of rigid targets based on a local association voting mechanism. Background Art

[0002] As one of the most common forms of matter that people come into contact with, rigid bodies are present in all aspects of life. In the field of target detection, in detection tasks such as vehicle detection and ship detection, rigid bodies appear as the main targets to be detected in these tasks. Therefore, rigid target detection is of great significance in the fields of security, modern warfare, etc. However, in many refined application scenarios, traditional target detection mostly uses rigid targets as a whole as the target to be detected, which often cannot provide accurate location information. For example, in modern naval warfare, the anti-ship missile seeker's ability to accurately detect the key parts of the ship and the accurate strike capability of the fuze and warhead play a vital role. This puts forward more refined requirements for the detection of rigid targets, so the research on the key parts detection of rigid targets is of great research significance.

[0003] The emergence of convolutional neural networks has made feature extraction simple, optimized the detection method of complex features, and greatly promoted the development of various detection tasks such as ship target detection. However, in the field of rigid target detection, there are still the following deficiencies: 1) Most existing detection methods take the rigid target as a whole as the detection target, which is not enough to provide accurate guidance information for the seeker in strike missions. 2) Some of the key parts detection methods for rigid targets based on deep learning currently proposed have poor processing capabilities for problems such as key parts occlusion and drastic changes in key parts posture in complex scenes.

[0004] Comprehensive analysis of the above problems shows that the current processing methods mainly focus on the appearance features of the target to perform pixel-level sampling on the input image. This method makes it difficult to extract effective semantic information from the occluded target when the target is occluded, making it difficult for the detector based on this mechanism to accurately detect the target. Therefore, a more comprehensive semantic acquisition method is urgently needed to achieve better detection performance.

[0005] Due to the characteristics of rigid objects, the key parts of rigid objects have strong spatial correlation. Even if there is an occlusion problem, the location information of the occluded part can often be inferred from the detected parts. Therefore, based on the appearance information detection, using the spatial correlation information of rigid objects, using image-based detection combined with local correlation based on rigid objects is a promising improvement method. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a method for detecting key parts of rigid targets based on a local correlation voting mechanism. The conventional detection results of YOLOv5 based on image sampling are predicted by a voting mechanism based on local correlation. On the basis of maintaining timeliness, key targets in complex scenes can be stably detected.

[0007] A method for detecting key parts of rigid objects based on a local correlation voting mechanism specifically comprises the following steps:

[0008] Step 1: Collect image data containing rigid targets, and mark the position information and category information of key parts of the rigid targets in the image. The key parts of visible rigid targets are marked in the form of images, and the key parts of invisible rigid targets are marked in the form of space vectors, and are used as image domain data and space domain data respectively.

[0009] Step 2: Add a high-resolution channel to the YOLOv5 network as an image domain detection model. The high-resolution channel is used to perform shallow sampling on the input data, and fuse the extracted features with the features extracted by other channels to output the image domain detection results.

[0010] Step 3: Establish a local association voting network as a spatial domain detection model. The spatial domain detection model generates unit vectors pointing to other key parts based on the pixel information of known key parts, and then uses the RANSAC (RANdomSample Consensus) algorithm to vote for the key point positions and output the spatial domain detection results.

[0011] Step 4: Input the image domain data and spatial domain data obtained in step 1 into steps 2 and 3 respectively to obtain the image domain detection model and the spatial domain detection model, and then adjust the model parameters through back propagation and train the model until convergence.

[0012] Step 5: Input the rigid body target image with key parts occluded into the image domain detection model trained in step 4, compare the number of key parts targets detected by the image domain detection model with the prior information of the number of known targets of each type, and obtain the number of missed detections n:

[0013] ① When the number of missed detections n=0, the output of the image domain detection model is used as the final detection result.

[0014] ② When the number of missed detections n≠0, read the key part position information output by the image domain detection model and generate the target bounding box. Input the position information of all pixels in the target bounding box into the spatial domain detection model trained in step 4, obtain 2n unit vectors on each pixel, output the center point and upper left corner vertex of the missed key part bounding box, and use the output of the image domain detection model and the spatial domain detection model as the final detection result.

[0015] Step 6: Based on the final detection result obtained in step 5, draw the prediction box, confidence level, and corresponding classification information on the input image to complete the detection of key parts of the rigid target.

[0016] The present invention has the following beneficial effects:

[0017] Aiming at the problem of detecting key parts of ships, this method proposes a local correlation voting algorithm to solve the problem of occlusion of key parts of rigid targets that may occur in complex actual environments. By utilizing the characteristics of rigid targets, the correlation between key parts of rigid targets is used as an input feature to train the neural network, so that occluded targets can be detected through unoccluded targets in occluded scenes. In order to make the image-based detection method perform well in the detection of small key parts, the original detection network YOLOv5 is improved and a high-resolution sampling channel is added to make the applicability of the entire algorithm better. On the basis of maintaining timeliness, it can stably detect key targets in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the key parts detection method for rigid targets;

[0019] Figure 2 This is the network structure diagram of the image domain detection model;

[0020] Figure 3 It is a detection result diagram based on the image domain in the embodiment;

[0021] Figure 4 Graph showing the detection results of the method in the embodiment. DETAILED DESCRIPTION

[0022] A method for detecting key parts of rigid objects based on local correlation voting mechanism mainly includes the following three parts:

[0023] 1) Adding a high-resolution channel module: Since some key parts occupy fewer pixels in the entire image, conventional target detection networks may have poor detection capabilities for these targets. Therefore, a high-resolution channel module is added to the YOLOv5 network structure. A separate sampling channel is used to shallowly sample the original image, reducing the loss of pixels of small targets during image downsampling, thereby retaining the features of small targets.

[0024] 2) Local correlation voting network based on detection results: YOLOv5 based on image sampling is difficult to solve the problem of target occlusion, but in actual scenes, it is inevitable that key targets are partially or completely occluded. In order to ensure the timeliness of detection, YOLOv5, a network with strong real-time performance, is used to detect unoccluded key targets. Then, based on the results of image detection, a voting mechanism based on local correlation is used to predict occluded targets to obtain better detection performance.

[0025] 3) Detection result generation module: First, based on the conventional detection information of YOLOv5, some key parts target information is generated. Then this information is sent to the local association voting network to obtain the occluded target prediction information. According to the predicted occluded target information and the detection information of the previous stage, the target information is solved and the bounding box is drawn to generate the final ship key parts detection result map.

[0026] This embodiment takes the key part detection of a ship as an example, and further explains a key part detection method of a rigid target based on a local correlation voting mechanism in conjunction with the accompanying drawings. Figure 1 As shown, the specific steps include:

[0027] Step 1: Collect a series of image data containing ships, and annotate the images with ship guns, phased array radars, fire control radars, and antennas as key parts. The visible key parts are annotated in the form of images, and the invisible key parts are annotated in the form of space vectors, which are used as image domain data and space domain data respectively.

[0028] Step 2: The YOLOv5 network performs well in detecting large and medium-sized objects, but is still insufficient in detecting small objects of about 10×10 pixels. Figure 2 As shown in the figure, a high-resolution channel is added to the YOLOv5 network. Through non-distorted convolution operations, the input data is shallowly sampled, and the features of small targets are retained until the fusion stage. They are deeply fused with features of different levels in other channels to further strengthen the feature weights of small targets. The problem of small target information loss caused by downsampling of the YOLOv5 backbone is improved, and other targets of regular sizes are not affected. Finally, feature maps of different resolutions are input into the detection head of the network for inference. Preset anchor frames of different sizes are matched with feature maps of different resolutions to obtain all target information of different sizes in the entire image. The improved network is used as an image domain detection model to improve the detection capability of small targets, making the model suitable for the detection of key parts of ships at different distances in actual scenes.

[0029] The image domain data obtained in step 1 is input into the image domain detection model in batches, and the predicted information is compared and analyzed with the preset anchor frame to form an offset to construct positive and negative samples. The number of positive samples is increased through the protection mechanism of small targets under high resolution, and then the model weights are fine-tuned through back propagation to make the model converge and achieve the training effect.

[0030] Step 3: Establish a local association voting network as a spatial domain detection model. Input the spatial domain data obtained in step 1 into the spatial domain detection model in batches. First, before training, calculate the unit vector of each pixel in the bounding box of each key part pointing to other key parts. The unit vector carries the semantic information of direction and category. Then, during the training process, the vector information obtained by model prediction is used to obtain the prediction result through the RANSAC algorithm, and then compared with the unit vector calculated before training. The difference obtained by comparison is used to fine-tune the model parameters through back propagation, so that the model converges to achieve the training effect.

[0031] Step 4: Input the rigid body target image with unknown key parts into the image domain detection model trained in step 2, and obtain Figure 3 The detection results based on the image domain are shown. According to the obtained detection results and the known category information, the number of missed detections n is calculated:

[0032] ① When the number of missed detections n=0, the output of the image domain detection model is used as the final detection result.

[0033] ② When the number of missed detections n≠0, read the key part position information output by the image domain detection model and generate the target bounding box. Input the position information of all pixels in the target bounding box into the spatial domain detection model trained in step 3, obtain 2n unit vectors on each pixel, output the center point and upper left corner vertex of the missed key part bounding box, and use the output of the image domain detection model and the spatial domain detection model as the final detection result.

[0034] Step 6: Based on the final detection result obtained in step 5, draw the prediction box, confidence level and corresponding classification information on the input image to complete the detection of key parts of the rigid target, such as Figure 4 shown.

[0035] This method was used to process 100 batches of images on a 1080Ti graphics card. During the entire process of the ship target moving from near to far, even in the presence of various occlusions, this method can accurately and stably detect the key parts of the target ship. And the farthest key parts can be detected within 10×10 pixels. During the entire detection process, good real-time performance is still maintained. After testing, the processing rate of YOLOv5 is 36ms / frame, and the processing time of this method is 62ms / frame, and the inference time of each frame of data increases by less than 30ms.

[0036] The above embodiments are not limitations of the present invention, and the present invention is not limited to the above embodiments. As long as the requirements of the present invention are met, they belong to the protection scope of the present invention.

Claims

1. A method for detecting key parts of rigid objects based on a local correlation voting mechanism, characterized in that: The specific steps include: Step 1, collect image data containing rigid targets, and mark the position information and category information of key parts of the rigid targets in the image, wherein the key parts of visible rigid targets are marked in the form of images, and the key parts of invisible rigid targets are marked in the form of space vectors, and are respectively used as image domain data and space domain data; Step 2: Add a high-resolution channel on the basis of the YOLOv5 network as an image domain detection model; the high-resolution channel is used to perform shallow sampling on the input data, and fuse the extracted features with the features extracted by other channels to output the image domain detection results; Step 3: Establish a local association voting network as a spatial domain detection model; the spatial domain detection model generates unit vectors pointing to other key parts based on the pixel information of the known key parts, and then uses the RANSAC algorithm to vote for the key point positions and output the spatial domain detection results; Step 4: input the image domain data and the spatial domain data obtained in step 1 into steps 2 and 3 respectively to obtain an image domain detection model and a spatial domain detection model, and then adjust the model parameters through back propagation, and train the model until convergence; Step 5: Input the rigid body target image with key parts occluded into the image domain detection model trained in step 4, compare the number of key parts targets detected by the image domain detection model with the prior information of the number of known targets of each type, and obtain the number of missed detections n: ① When the number of missed detections n = 0, the output of the image domain detection model is used as the final detection result; ② When the number of missed detections n≠0, read the key part position information output by the image domain detection model to generate a target bounding box; input the position information of all pixels in the target bounding box into the spatial domain detection model trained in step 4, obtain 2n unit vectors on each pixel, output the center point and upper left corner vertex of the missed key part bounding box, and use the output of the image domain detection model and the spatial domain detection model as the final detection result; Step 6: Based on the final detection result obtained in step 5, draw the prediction box, confidence level, and corresponding classification information on the input image to complete the detection of key parts of the rigid target.

2. A method for detecting key parts of rigid objects based on a local correlation voting mechanism as claimed in claim 1, characterized in that: The rigid body target is a ship, and the corresponding key part category information is ship artillery, phased array radar, fire control radar and antenna.

Images