Target identification method based on 180-degree direction rotation frame detection

Abstract

The invention relates to a target identification method based on 180-degree direction rotating frame detection, which comprises the following steps of: firstly, carrying out any quadrilateral four-point coordinate labeling on a target in a collected photoelectric data set by adopting a labeling tool to obtain a labeling frame, and secondly, carrying out four-point coordinate labeling on a photoelectric image data set subjected to preliminary labeling, performing annotation frame conversion by adopting a long edge representation method in a rotating rectangular frame representation method so as to obtain training data with rotating rectangular frame annotation, and performing model training on a rotating frame detection algorithm improved based on Yolov5 by adopting a converted photoelectric image data set, and performing target prediction on a to-be-predicted target image by using the trained Yov5-based improved rotating frame detection network model to obtain a target feature map, and finally deploying the Yov5-based improved rotating frame detection network model on a mercuric chloride 310 chip to realize target prediction in the to-be-predicted target image. According to the invention, target detection in any direction in different scenes can be rapidly and efficiently realized, and the detection efficiency is greatly improved.

Description

Technical Field

[0001] The invention belongs to the technical field of airborne optoelectronic equipment, and in particular relates to a target recognition method based on 180-degree rotation frame detection. Background Art

[0002] In the field of airborne optoelectronic equipment, optoelectronic equipment is generally mounted on a flying aircraft to perform image acquisition, recording and other tasks. However, due to the large uncertainty of the aircraft's flight angle and posture, as well as the complex and changeable ground scenes collected during the flight, the collected target images are often distorted, rotated, and distorted. Directly using a recognition algorithm based on a conventional horizontal frame to perform target recognition on the above images will cause the recognition algorithm to enlarge the detection frame in order to detect the target, thereby including more background areas, and even overlaps and missed detections of different target detection frames, which will have a negative impact on subsequent recognition classification and target threat assessment tasks; Figure 1 As shown in the figure, in a scene with dense speedboats moored in the port, the results based on conventional horizontal frame detection may lead to missed detections due to the overlap of large and small targets during the detection process. In addition, due to the existence of too much irrelevant background in the detection frame in the dense scene, it is difficult to effectively distinguish the positions of different targets.

[0003] Therefore, it is necessary to design a target recognition method based on 180-degree rotation frame detection to solve the above technical problems. Summary of the invention

[0004] In view of this, the present invention proposes a target recognition method based on 180-degree rotating frame detection, which can quickly realize the 180-degree detection task of the target in the airborne optoelectronic scene, and effectively solve the technical problems of missed detection and excessive irrelevant background in the existing conventional horizontal frame detection size.

[0005] In order to achieve the above technical objectives, the specific technical solutions adopted by the present invention are:

[0006] A method for object detection based on a 180-degree rotation frame includes the following steps:

[0007] S1, rotating box target dataset production;

[0008] The roLabelImg annotation tool is used to annotate the four-point coordinates of any quadrilateral on the target in the collected optoelectronic data set to obtain the annotation box.

[0009] S2. For the optoelectronic image dataset that has been initially annotated, the long side representation method in the rotated rectangular box representation method is used to convert the annotation box to obtain training data with rotated rectangular box annotations.

[0010] S3. Use the converted photoelectric image dataset to train the model of the rotation box detection algorithm improved based on Yolov5.

[0011] S4. Use the trained Yolov5 improved rotation box detection network model to perform target prediction on the target image to be predicted and obtain the target feature map.

[0012] S5. Deployed on the Ascend 310 chip to implement target prediction in the target image to be predicted.

[0013] Furthermore, in step S2, the representation method of the rotated rectangular box is in the form of (x, y, l, s, θ), where (x, y) is the coordinate of the center point of the rotated rectangular box, l is the longest side of the rotated box, h is the other side corresponding to the longest side, θ is the angle passed by the x-axis when rotating clockwise to encounter the longest side, and five parameters (x, y, l, s, θ) are used to represent the rotated rectangular box OBB in the direction of 180 degrees, and θ is set to an integer in the value range of [0 to 179].

[0014] Furthermore, in step S3, the final loss function includes the design of category classification loss function, rotation angle classification loss function, rotation box regression loss function and confidence loss function, and the loss function L of the final network model training is:

[0015] L=L hbb +λ L L cls +λ2L θCE +λ3L conf

[0016] Where, L hbb is the rotation box regression loss function, L θCE is the rotation angle classification loss function, L cls is the category classification loss function, L conf is the confidence loss function, λ1, λ2, and λ3 are the control loss functions L cls , L θCE and L conf The balanced hyperparameters of .

[0017] Furthermore, the category classification loss function in step S3 is:

[0018]

[0019] Among them, p represents the true value of the category, Represents the predicted value, and the number of i represents the total number of categories.

[0020] Furthermore, the rotation angle classification loss function L θCE for:

[0021]

[0022] Wherein, is the true value corresponding to the rotation angle of the rotating box, an integer ranging from [0 to 179]; is the angle predicted by the rotating rectangular box.

[0023] Furthermore, the rotation box regression loss function is L hbb :

[0024]

[0025] in,

[0026] b. are the center points of the predicted rotated rectangular box and the center points of the true rotated rectangular box, s is the Euclidean distance between the predicted rotated rectangular box and the true rotated rectangular box, c is the diagonal distance between the closure area of ​​the predicted rotated rectangular box and the true rotated rectangular box, v is a measure of the relative proportion consistency between the predicted rotated rectangular box and the true rotated rectangular box, α is the regression weight coefficient, and IoU is the intersection over union ratio of the predicted rotated rectangular box and the true rotated rectangular box;

[0027] IoU is one of the commonly used performance evaluation indicators in target detection. The accuracy of the predicted box is judged according to the degree of overlap between the predicted box and the true box. The calculation process is:

[0028]

[0029] Where A represents the true box and B represents the predicted box.

[0030] Furthermore, the confidence loss function L conf for:

[0031]

[0032] Among them, t represents the true value of confidence, represents the confidence prediction value, is in the range [0,1).

[0033] The present invention iteratively trains the rotating box detection network improved by Yolov5 according to the training feature graph, and uses the final loss function to optimize the rotating box detection network improved by Yolov5 according to each iterative training result to indicate the direction of cyclic iterative optimization. After training for a certain number of times, the iteration can be stopped when the desired effect is achieved (for example, when the final loss function L value is less than a certain threshold), and the model saved at this time is the input model required for subsequent model prediction-the rotating box target detection model.

[0034] Compared with the original Yolov5 algorithm, the network model proposed in the present invention adds a one-dimensional output in the prediction network to predict the angle direction. In order to facilitate training and convergence, the angle is regarded as a classification problem, and 180 dimensions are predicted from 0 to 179 degrees. Therefore, the prediction output of each feature layer becomes 5+nc+180 dimensions, where nc is the number of categories.

[0035] An improved rotating box detection network model based on Yolov5 is adopted and deployed on the Ascend 310AI chip to realize target detection in the target image to be predicted. In the model deployment stage, the operators in the network model are optimized and improved to improve the inference speed of the network model in extracting the target feature map in the image to be predicted. At the same time, the post-processing method of the rotating box target recognition algorithm is optimized to further improve the speed of obtaining the detected target box in the image to be predicted. The currently proposed improved rotating box detection method based on Yolov5 can realize the detection of the target in the image to be predicted within 31ms for an image with a resolution of 832*832.

[0036] By adopting the above technical solution, the present invention can also bring the following beneficial effects:

[0037] 1. The present invention mentions a target recognition method based on 180-degree rotating frame detection, which adopts a rotating frame detection network model improved based on Yolov5 and is deployed on the Ascend 310AI chip to realize target detection in the target image to be predicted, and optimizes and improves the operators in the network model in the model deployment stage to improve the reasoning speed of the network model in extracting the target feature map in the image to be predicted. At the same time, the post-processing method of the rotating frame target recognition algorithm is optimized to further improve the speed of obtaining the detected target frame in the image to be predicted, and can realize the detection of the target in the image to be predicted within 31ms for an image with a resolution of 832*832, which has the advantages of fast processing speed and good processing effect.

[0038] 2. The present invention mentions a target recognition method based on 180-degree rotating frame detection. The rotating frame detection can accurately locate the object in the image. The boundary box almost does not include the background area, thereby reducing the influence of the background on the object classification. There is almost no overlap between the rotating detection frames, so that the object contained in the frame can be more clearly identified. The movement direction information of the object can be roughly obtained from the rotating detection frame, thereby judging the movement trajectory of the object. It can meet the needs of existing image target recognition and has good application prospects.

[0039] 3. The present invention mentions a target recognition method based on 180-degree rotating frame detection. For the optoelectronic data set that has been annotated with any quadrilateral, the long edge representation method is used to convert it into a data set that can be used for training a rotating frame target recognition algorithm based on 180-degree direction prediction. On the basis of the original Yolov5 algorithm, a one-dimensional prediction output for the angle direction is added, and the loss function of the network model training is improved. In the model reasoning stage, the operators in the network model are optimized to achieve the requirement of fast reasoning of input images on an embedded platform. In the model deployment stage, the post-processing method of the rotating frame target recognition algorithm is optimized to meet the requirements of implementation on an airborne embedded platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a schematic diagram of target recognition based on a positive frame and based on a rotating frame in the prior art;

[0042] Figure 2 It is a flow chart of a target recognition method based on 180-degree rotation frame detection mentioned in the present invention. DETAILED DESCRIPTION

[0043] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0044] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0045] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present invention, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0046] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0047] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0048] Example 1

[0049] In one embodiment of the present invention, see Figure 2 , a target detection method based on a 180-degree rotation frame, comprising the following steps:

[0050] S1. Use the roLabelImg annotation tool to annotate the coordinates of the four points of any quadrilateral on the target in the collected optoelectronic data set to obtain the annotation box.

[0051] S2. For the photoelectric image dataset that has been initially annotated, the long side representation method in the rotated rectangular frame representation method is used to convert the annotation frame to obtain a training dataset with rotated rectangular frame annotations.

[0052] The representation method of the rotated rectangle is in the form of: (x, y, l, s, θ);

[0053] Where (x, y) is the coordinate of the center point of the rotated rectangular box, l is the longest side of the rotated box, h is the other side corresponding to the longest side, and θ is the angle that the x-axis passes when rotating clockwise to meet the longest side.

[0054] Specifically, in this embodiment, five parameters (x, y, l, s, θ) are used to represent the 180-degree rotated rectangular box OBB. θ, like (x, y, l, s), can be learned through the regression branch. To prevent ambiguity, θ is set to an integer in the range of [0 to 1179].

[0055] S3, using the converted photoelectric image dataset to train the model of the rotation box detection algorithm improved based on Yolov5;

[0056] The Yolov5 improved rotating box detection network is iteratively trained according to the training feature graph, and the final loss function is used to optimize the Yolov5 backbone network according to each iterative training result to obtain a rotating target detection model. During the training optimization, the final loss function is required to indicate the direction of the cyclic iterative optimization for the Yolov5 improved rotating box detection network; after a certain number of training times, when the final loss function L value is less than a certain threshold, the iteration can be stopped, and the model saved at this time is the rotating box target detection model required for subsequent model prediction.

[0057] The final loss function includes the category classification loss function, the rotation angle classification loss function, the rotation box regression loss function and the confidence loss function design. The final loss function L of the network model training is:

[0058] L=L hbb +λ1L cls +λ2L θCE +λ3L conf

[0059] Where, L hbb is the rotation box regression loss function, L θCE is the rotation angle classification loss function, L cls is the category classification loss function, L conf is the confidence loss function, λ1, λ2, and λ3 are the control loss functions L cls , L θCE and L conf The balanced hyperparameters of .

[0060] The category classification loss function is:

[0061]

[0062] Among them, p represents the true value of the category, Represents the predicted value, and the number of i represents the total number of categories.

[0063] The rotation angle classification loss function L θCE for:

[0064]

[0065] Wherein, is the true value corresponding to the rotation angle of the rotating box, an integer ranging from [0 to 179]; is the angle predicted by the rotating rectangular box.

[0066] The rotation box regression loss function is L hbb :

[0067]

[0068] in,

[0069] b. are the center points of the predicted rotated rectangular box and the center points of the true rotated rectangular box, s is the Euclidean distance between the predicted rotated rectangular box and the true rotated rectangular box, c is the diagonal distance between the closure area of ​​the predicted rotated rectangular box and the true rotated rectangular box, v is a measure of the relative proportion consistency between the predicted rotated rectangular box and the true rotated rectangular box, α is the regression weight coefficient, and IoU is the intersection over union ratio of the predicted rotated rectangular box and the true rotated rectangular box;

[0070] IoU is one of the commonly used performance evaluation indicators in target detection. The accuracy of the predicted box is judged according to the degree of overlap between the predicted box and the true box. The calculation process is:

[0071]

[0072] Where A represents the true box and B represents the predicted box.

[0073] The confidence loss function L conf for:

[0074]

[0075] Among them, t represents the true value of confidence, represents the confidence prediction value, is in the range [0,1).

[0076] S4. Based on the rotation box detection network improved by Yolov5, feature extraction is performed on the training data set to obtain a training feature map;

[0077] S5 is deployed on the Ascend 310 AI chip to implement target prediction in predicted target images. Currently, it can process images with an input resolution of 832*832 within 31ms.

[0078] In view of the deployment of the rotating box detection algorithm on embedded AI chips, this chapter adopts a rotating box target recognition network based on 180-degree direction prediction. The network consists of two parts: the feature extraction network and the prediction network of YOLOV5. Compared with the horizontal box detection algorithm, the rotating box detection algorithm adds a one-dimensional output to the prediction network to predict the angle direction. In order to facilitate training and convergence, the angle is regarded as a classification problem, and a total of 180 dimensions are predicted from 0 to 179 degrees. Therefore, the prediction output of each feature layer becomes 5+nc+180 dimensions, where nc is the number of categories.

[0079] During the model training phase, the input training data is supplemented with input angle information based on the original horizontal box prediction input coordinate information. The long edge representation method is used to take the angle passed by the long edge when the x-axis is rotated counterclockwise as the input angle information to train the rotation box detection model.

[0080] In the model inference stage, for the output angle information, the top1 function operator of the original algorithm that implements the maximum probability angle output is changed to the max function operator implemented by Ascend 310 with faster processing, which greatly speeds up the model prediction process.

[0081] During the model deployment phase, since the prediction result of the rotated box has an additional angle information, it cannot be written as an operator and put into the NPU like the straight box. Therefore, an independent post-processing process is required. This part needs to rewrite the post-processing code and use a new four-point box NMS to suppress redundant boxes.

[0082] In summary, based on horizontal frame detection, the present invention designs a rotating frame target recognition network based on 180-degree direction prediction, which can quickly and efficiently realize target detection in any direction in different scenarios, greatly improve the detection efficiency, and solve the problems of missed detection and over-detection in conventional detectors.

[0083] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A target detection method based on a 180-degree rotation frame, characterized in that: The steps include: S1, rotating box target dataset production; Use the annotation tool to annotate the coordinates of four points of any quadrilateral on the target in the collected photoelectric data set to obtain the annotation box; S2, for the photoelectric image data set that has been preliminarily annotated, the long side representation method in the rotated rectangular frame representation method is used to convert the annotation frame to obtain training data with rotated rectangular frame annotations; S3, using the converted photoelectric image dataset to train the model of the rotation box detection algorithm improved based on Yolov5; S4, using the trained Yolov5 improved rotation box detection network model to perform target prediction on the target image to be predicted, and obtain the target feature map; S5. Deployed on the Ascend 310 chip to implement target prediction in the target image to be predicted.

2. The target detection method based on a 180-degree rotation frame according to claim 1, characterized in that: In step S2, the representation method of the rotated rectangular box is in the form of (x, y, l, s, θ), where (x, y) is the coordinate of the center point of the rotated rectangular box, l is the longest side of the rotated box, h is the other side corresponding to the longest side, and θ is the angle passed by the x-axis when rotating clockwise to encounter the longest side. Five parameters (x, y, l, s, θ) are used to represent the rotated rectangular box OBB in the direction of 180 degrees, and θ is set to an integer in the value range of [0 to 179].

3. The target detection method based on a 180-degree rotation frame as claimed in claim 2, characterized in that: In the step S3, the Yolov5 improved rotating box detection network is iteratively trained according to the training feature graph, and the final loss function is used to optimize the Yolov5 improved rotating box detection network according to each iterative training result, indicating the direction of the cyclic iterative optimization, and after a certain number of training times, if the final loss function L value is less than a certain threshold, the iteration can be stopped, and the model saved at this time is the rotating box target detection model required for subsequent model prediction.

4. The target detection method based on a 180-degree rotation frame as claimed in claim 3, characterized in that: In step S3, the final loss function includes the design of category classification loss function, rotation angle classification loss function, rotation box regression loss function and confidence loss function. The loss function L of the final network model training is: L=L hbb +λ1L cls +λ2L θCE +λ3L conf Where, L hbb is the rotation box regression loss function, L θCE is the rotation angle classification loss function, L cls is the category classification loss function, L conf is the confidence loss function, λ1, λ2, and λ3 are the control loss functions L cls , L θCE and L conf The balanced hyperparameters of .

5. The target detection method based on a 180-degree rotation frame as claimed in claim 4, characterized in that: The category classification loss function in step S3 is: Among them, p represents the true value of the category, Represents the predicted value, and the number of i represents the total number of categories.

6. The target detection method based on a 180-degree rotation frame as claimed in claim 5, characterized in that: In step S3, the rotation angle classification loss function L θCE for: Wherein, is the true value corresponding to the rotation angle of the rotating box, an integer ranging from [0 to 179]; is the angle predicted by the rotating rectangular box.

7. A target detection method based on a 180-degree rotation frame as claimed in claim 6, characterized in that: In step S3, the rotation frame regression loss function is L hbb : in, b. are the center points of the predicted rotated rectangular box and the center points of the true rotated rectangular box, s is the Euclidean distance between the predicted rotated rectangular box and the true rotated rectangular box, c is the diagonal distance between the closure area of ​​the predicted rotated rectangular box and the true rotated rectangular box, v is a measure of the relative proportion consistency between the predicted rotated rectangular box and the true rotated rectangular box, α is the regression weight coefficient, and IoU is the intersection over union ratio of the predicted rotated rectangular box and the true rotated rectangular box; IoU is one of the commonly used performance evaluation indicators in target detection. The accuracy of the predicted box is judged according to the degree of overlap between the predicted box and the true box. The calculation process is: Where A represents the true box and B represents the predicted box.

8. The target detection method based on a 180-degree rotation frame as claimed in claim 7, characterized in that: In step S3, the confidence loss function L conf for: Among them, t represents the true value of confidence, represents the confidence prediction value, is in the range [0,1).

Images