Double-stage rotating target detection method and device, storage medium and electronic equipment

Abstract

The invention discloses a two-stage rotating target detection method and device, a storage medium and electronic equipment. The method comprises the steps of obtaining a remote sensing image; inputting the remote sensing image into a ResNet network, calculating a Manhattan distance between pixel points in the remote sensing image, obtaining a global feature based on the Manhattan distance, and obtaining an image feature based on the global feature; inputting the image features into an RPN network for convolution processing to obtain a candidate feature map, and obtaining a candidate frame based on a pre-defined prior frame and the candidate feature map; screening out K prediction horizontal frames based on a non-maximum suppression algorithm; and converting the prediction horizontal frame into a prediction rotation frame, and obtaining a target detection result of the remote sensing image based on the prediction rotation frame. The target detection efficiency and detection precision can be improved, and the method can better adapt to complex scenes.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image detection, and in particular to a dual-stage rotating target detection method, device, storage medium and electronic device. Background Art

[0002] With the continuous development of remote sensing image technology and increasingly diverse application scenarios, target detection models for remote sensing images continue to receive attention.

[0003] The horizontal rectangular target box output by the traditional target detection model has various problems such as being unable to effectively identify dense targets and targets with large aspect ratios in remote sensing images. In this context, the method of rotation target detection has emerged. Rotation target detection represents the target as a rectangular box with a rotation angle. This representation method can better separate different targets and is suitable for target detection in remote sensing images.

[0004] Current target detection algorithms are mainly divided into two-stage target detection algorithms and single-stage target detection algorithms. The single-stage target detection algorithm completes the target detection task in a single forward propagation, without generating candidate regions, and usually has a faster reasoning speed. Unlike the single-stage target detection algorithm, the two-stage target detection algorithm first generates a set of candidate target regions, and then classifies and fine-grainedly locates these candidate regions. In terms of accuracy, the two-stage target detection algorithm is significantly better than the single-stage target detection algorithm, but its detection speed is slower. Summary of the invention

[0005] The embodiments of the present application provide a two-stage rotating target detection method, device, storage medium and electronic device, which can enhance the ability to extract multi-scale features and improve the computing efficiency while maintaining the effect, thereby improving the target detection speed.

[0006] The present application embodiment provides a two-stage rotating target detection method, comprising: Acquisition of remote sensing images; Inputting the remote sensing image into a ResNet network, calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; Input the image features into the RPN network for convolution processing to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; Based on the non-maximum suppression algorithm, K predicted horizontal boxes are selected; The predicted horizontal frame is converted into a predicted rotation frame, and the target detection result of the remote sensing image is obtained based on the predicted rotation frame.

[0007] Furthermore, the above two-stage rotating target detection method, wherein the remote sensing image is input into a ResNet network, the Manhattan distance between pixels in the remote sensing image is calculated, a global feature is obtained based on the Manhattan distance, and an image feature is obtained based on the global feature, includes: Input the remote sensing image into the ResNet network to obtain a first feature map through convolution operation, batch normalization and maximum pooling operation; Calculate the Manhattan distance between pixels based on the two-dimensional coordinates of the pixels marked in the remote sensing image; Calculate the weight of each pixel in the first feature map; Obtaining a global feature based on the weight of each pixel and the Manhattan distance between the pixels; Perform multiple convolution operations on the global features to obtain image features.

[0008] Furthermore, in the above two-stage rotating target detection method, the step of calculating the Manhattan distance between pixels based on the two-dimensional coordinates of the pixels marked in the remote sensing image comprises: The Manhattan distance between pixels is calculated by the first formula, which is:

[0009] Where d represents two pixels ( )and( ), D is the attenuation matrix in two-dimensional space, the elements of which represent the Manhattan distance based on the two-dimensional coordinates of the pixel points. It is an adjustable parameter that controls the influence of the distance between two pixels on the attention weight; The weight of each pixel in the first feature map is calculated, and the global feature is obtained based on the weight of each pixel and the Manhattan distance between the pixels, which is calculated by the following formula:

[0010]

[0011]

[0012] in, is the first feature map, , and is the weight matrix used to transform the input into query (Q), key (K), and value (V) vectors, It is a vector used to represent the weights and biases in the ResNet network. is a complex matrix, The complex conjugate of , the self-attention mechanism used to calculate the attention score; It is a global feature. Softmax() represents the Softmax operation on the inner product of the query and the key, which is used to calculate the attention weight.

[0013] Furthermore, in the above two-stage rotation target detection method, the step of selecting K predicted horizontal boxes based on a non-maximum suppression algorithm includes: Calculating the confidence of the candidate frame according to the coordinates of the candidate frame and the probability that an object exists in the candidate frame; According to the confidence, all candidate boxes are sorted from high to low confidence, and K predicted horizontal boxes with higher confidence are screened out.

[0014] Furthermore, the above two-stage rotation target detection method, wherein the predicted horizontal frame is converted into a predicted rotation frame, and the target detection result of the remote sensing image is obtained based on the predicted rotation frame, comprises: Inputting the K predicted horizontal boxes into the trained Rotated RoI Align network to extract the minimum bounding boxes of the regions of interest in the K predicted horizontal boxes respectively, and obtaining the predicted rotation boxes based on the minimum bounding boxes; The predicted rotation frame is input into a classifier to obtain a target detection result of the remote sensing image.

[0015] Furthermore, in the above-mentioned two-stage rotated object detection method, the Rotated RoI Align network is trained by a SmoothL1 loss function; The calculation process of the SmoothL1 loss function includes:

[0016]

[0017]

[0018]

[0019]

[0020] in, is the coordinate of the center point of the real rotating frame, To predict the coordinates of the center point of the rotating box, is the length and width of the real rotation box, To predict the length and width of the rotation box, is the rotation angle of the real rotating frame, To predict the rotation angle of the rotation box;

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] in, is the SmoothL1 loss function.

[0027] Furthermore, in the above two-stage rotation target detection method, the classifier is trained by a cross entropy loss function, and the calculation process of the cross entropy loss function includes:

[0028] in, is the one-hot encoding of the true label, indicating The true probability of each category, Representative The predicted probability of each class.

[0029] The embodiment of the present application also provides a dual-stage rotating target detection device, comprising: An acquisition module, used for acquiring remote sensing images; A feature extraction module, used for calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; A candidate frame generation module is used to perform convolution processing on the image features to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; The non-maximum suppression module is used to select K predicted horizontal boxes based on the non-maximum suppression algorithm; The conversion and prediction module is used to convert the predicted horizontal frame into a predicted rotation frame, and obtain the target detection result of the remote sensing image based on the predicted rotation frame.

[0030] An embodiment of the present application further provides a computer-readable storage medium, wherein a plurality of instructions are stored in the computer-readable storage medium, and wherein the instructions are suitable for being loaded by a processor to execute any of the above-mentioned two-stage rotating target detection methods.

[0031] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used for the steps in any of the above-mentioned two-stage rotating target detection methods.

[0032] The dual-stage rotation target detection method, device, storage medium and electronic device provided by the present application add the Manhattan self-attention mechanism in the ResNet network, which can enhance the ability to extract multi-scale features and improve the computational efficiency while maintaining the effect. The present application uses a non-maximum suppression algorithm to filter the predicted horizontal box, and by attenuating the score instead of directly removing overlapping targets, these dense targets can be more effectively retained, the integrity of the detection results can be improved, and the detection accuracy can be improved while reducing false detections, so as to better adapt to complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.

[0034] Figure 1 A flowchart of a two-stage rotating target detection method provided in an embodiment of the present application.

[0035] Figure 2 Another flowchart of the two-stage rotating target detection method provided in an embodiment of the present application.

[0036] Figure 3 A flowchart for generating image features provided in an embodiment of the present application.

[0037] Figure 4 A schematic diagram of the structure of a dual-stage rotating target detection device provided in an embodiment of the present application.

[0038] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0039] Figure 6 Another schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0041] The embodiments of the present application provide a dual-stage rotating target detection method, device, storage medium and electronic device. The dual-stage rotating target detection device provided in the embodiments of the present application can be integrated in an electronic device, which can be a terminal, a server and other devices, wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0042] Two-stage object detection algorithms are usually divided into two main stages. The first stage is to generate candidate object regions, usually achieved through the region proposal network RPN or other methods. The second stage is classification and localization, which is to classify and refine the localization of these candidate regions. Two-stage algorithms usually perform well in terms of accuracy in object detection tasks. They can provide high-quality object detection results, especially for complex scenes and applications requiring high precision. Representative algorithms for two-stage object detection include Faster R-CNN, Mask R-CNN, Cascade R-CNN, etc.

[0043] See also Figure 1 Figure 2 , Figure 1 A flowchart of a two-stage rotating target detection method provided in an embodiment of the present application, Figure 2 Another flow chart of the two-stage rotating target detection method provided in an embodiment of the present application, which is applied to an electronic device, the two-stage rotating target detection method comprising the following steps: S1, obtain remote sensing images.

[0044] S2, input the remote sensing image into the network, calculate the Manhattan distance between the pixels in the remote sensing image, obtain the global features based on the Manhattan distance, and obtain the image features based on the global features.

[0045] This application adds a Manhattan self-attention mechanism based on the residual network (ResNet50), extends the time decay mechanism of ResNet to the spatial domain, and uses a two-dimensional bidirectional spatial decay matrix based on Manhattan distance to introduce explicit spatial priors for image data, thereby improving the performance of the self-attention mechanism.

[0046] Figure 3 For a flowchart of generating image features provided in an embodiment of the present application, please refer to Figure 3 , step S2 may include the following steps: S21, input the remote sensing image into the ResNet network to obtain the first feature map through convolution operation, batch normalization and maximum pooling operation.

[0047] Specifically, the ResNet50 network includes five layers from stage0 to stage4. In stage0, the input remote sensing image of size (3, 224, 224) is passed through 64 convolution kernels of size (7, 7) and stride 2, and the number of output channels is 64. Then it passes through the batch normalization layer and the activation function RELU. The batch normalization layer is a regularization technology commonly used in deep neural networks. It can accelerate the training process of neural networks and make the gradient in the network more stable during the back propagation process. Finally, it passes through the maximum pooling layer, in which the convolution kernel size is 3×3 and the stride is 2. The first feature map output at the end has a size of 224 / 2 / 2=56.

[0048] In the RestNet50 network, due to the causal nature of text data, the retention is unidirectional, which makes each token only pay attention to the previous token and not the token after it. This feature is not suitable for tasks that lack causal properties such as image recognition. Therefore, this application expands the retention to a bidirectional form and expands the one-dimensional retention to two dimensions.

[0049] The structures of stage1~stage4 are the same, including convolutional layers Conv Block and ID Block, and the Harmanton self-attention mechanism is added in stage1.

[0050] S22, calculating the Manhattan distance between the pixels based on the two-dimensional coordinates of the pixels marked in the remote sensing image.

[0051] In the image, each pixel has a two-dimensional coordinate in the plane, and the coordinate of the nth pixel is expressed as ( ). To accommodate this, the present invention adjusts each element in the matrix D to represent the Manhattan distance between pixel pairs and uses Softmax to introduce nonlinearity into the model.

[0052] Specifically, the Manhattan distance between pixels is calculated by the first formula, which is:

[0053] Where d represents two pixels ( )and( ), D is the attenuation matrix in two-dimensional space, the elements of which represent the Manhattan distance based on the two-dimensional coordinates of the pixel points. It is an adjustable parameter that controls the influence of the distance between two pixels on the attention weight.

[0054] S23, calculating the weight of each pixel in the first feature map.

[0055] S24, obtain the global feature based on the weight of each pixel and the Manhattan distance between pixels.

[0056] Specifically, steps S23-S24 are calculated by the following formula:

[0057]

[0058]

[0059] in, is the first feature map, , and is the weight matrix used to transform the input into query (Q), key (K), and value (V) vectors, It is a vector used to represent the weights and biases in the ResNet network. is a complex matrix, The complex conjugate of , the self-attention mechanism used to calculate the attention score; It is a global feature. Softmax() represents the Softmax operation on the inner product of the query and the key, which is used to calculate the attention weight.

[0060] S25, performing multiple convolution operations on the global features to obtain image features.

[0061] Specifically, the global features are input into stage1~stage4 in the RestNet50 network, and multiple convolution operations are performed in sequence to obtain image features.

[0062] The Manhattan self-attention mechanism can capture long-distance dependencies in an image. In the embodiment of the present application, the Manhattan distance between each position in the feature map is calculated to enhance the expressive power of important areas in the feature map. In this way, the key features in the image can be more prominent and the recognition ability of the model can be improved. The Manhattan self-attention mechanism can effectively handle noise and deformation in the image. By focusing on global information, the influence of local noise is reduced, and the model is more stable when processing various complex scenes and images of different resolutions. The ResNet50 network has multi-level feature extraction capabilities. Features at different levels contain information of different scales. After adding the Manhattan self-attention mechanism, the expression of these multi-scale features can be further enhanced. The Manhattan self-attention mechanism in this application has a lower computational complexity than the traditional self-attention mechanism, and can improve computational efficiency while maintaining the effect. Therefore, the present application adds the Manhattan self-attention mechanism on the basis of ResNet50, and provides richer and more accurate features for target detection tasks through this combination method, thereby improving the performance of the task.

[0063] S3, input the image features into the RPN network for convolution processing to obtain a candidate feature map, and obtain a candidate box based on the pre-defined prior box and the candidate feature map.

[0064] The image features are input into the RPN (Region Proposal Network), which first undergoes a 3×3 convolution to obtain a 256×16×16 feature map, and then undergoes two 1×1 convolutions to obtain a 18×16×16 feature map and a 36×16×16 feature map, that is, 16×16×9 results, each of which contains 2 scores and 4 coordinates. Combined with the pre-defined prior box Anchors, the candidate box is obtained through post-processing. In addition, when the candidate box is obtained, the candidate box coordinates and the probability of the object existing in the candidate box are automatically generated.

[0065] Among them, the predefined prior frame is a series of anchor frames with different sizes and aspect ratios generated on the image. There is a one-to-one mapping relationship between a random pixel point on the feature map and the original image. Each pixel point on the feature map corresponds to a frame of the original image, but due to the different sizes of the original image and the feature map, this frame is not suitable. Therefore, the center of the frame is used as an anchor point to randomly generate a series of predefined prior frame anchors with different sizes and aspect ratios.

[0066] S4, select K predicted horizontal boxes based on the non-maximum suppression algorithm.

[0067] In one embodiment, a non-maximum suppression algorithm Soft-NMS algorithm is used to delete redundant boxes and obtain K best predicted horizontal boxes of the target. Step S4 may include the following steps: S41, calculating the confidence of the candidate box according to the coordinates of the candidate box and the probability that an object exists in the candidate box.

[0068] Specifically, step S41 can be calculated by the following formula: C = P (Object) × IoU Among them, C is the confidence, P(Object) is the probability of the existence of an object in the candidate box, and IoU is the measurement standard.

[0069] S42, according to the confidence, sort all candidate boxes from high to low confidence, and select K predicted horizontal boxes with higher confidence.

[0070] Specifically, all prediction boxes are sorted according to confidence, and then the prediction box M with the maximum score is selected. The confidence of other boxes whose IoU with the M box exceeds the threshold 0.7 is reduced. The larger the IoU, the greater the degree of suppression. Finally, the top K prediction boxes are selected according to the confidence from high to low.

[0071] The embodiment of the present application uses the Soft-NMS algorithm for non-maximum suppression processing. Since the targets in remote sensing images usually appear at arbitrary angles, traditional NMS is prone to over-suppression due to the fixed IoU threshold when processing rotated targets. Soft-NMS can better handle rotated targets and reduce missed detections and false detections by attenuating the scores. In remote sensing images, targets are often very dense and of different sizes. In this case, traditional NMS may eliminate a large number of overlapping targets. Soft-NMS can more effectively retain these dense targets and improve the integrity of the detection results by attenuating the scores instead of directly eliminating overlapping targets. Since remote sensing images usually contain complex backgrounds and noise, Soft-NMS can improve detection accuracy while reducing false detections and better adapt to complex scenes by making subtle adjustments to the scores of overlapping targets. This is very important for remote sensing image applications that require high-precision detection.

[0072] S5, converting the predicted horizontal frame into a predicted rotation frame, and obtaining the target detection result of the remote sensing image based on the predicted rotation frame.

[0073] In one embodiment, step S5 comprises the following steps: S51, inputting the K predicted horizontal boxes into the trained Rotated RoI Align network to extract the minimum bounding boxes of the regions of interest in the K predicted horizontal boxes respectively, and obtaining the predicted rotation boxes based on the minimum bounding boxes.

[0074] S52, input the predicted rotation box into the classifier to obtain the target detection result of the remote sensing image.

[0075] The embodiment of the present application uses the Rotated RoI Align network to extract the rotated region of interest, and converts the horizontal box obtained by the RPN network into a rotated box. Due to the quantization operation, the traditional extraction method cannot accurately align the area of ​​the rotating target on the feature map. For the rotating target, the boundary cannot be accurately aligned, which will affect the detection accuracy. Rotated RoI Align eliminates the quantization operation, retains the floating-point coordinates, achieves high-precision feature alignment, and ensures more accurate feature extraction of the rotated target area through the bilinear interpolation method. In remote sensing images, targets may appear at various angles. Traditional methods are easily affected by background noise and have low processing efficiency. Rotated RoI Align can accurately align target areas at any angle, extract rotation-invariant features, better capture its boundaries and details, improve detection and segmentation accuracy, and reduce background noise interference.

[0076] Furthermore, the training process of the Rotated RoI Align network is based on the real rotation box and the SmoothL1 loss function. The calculation process of the SmoothL1 loss function includes: The Rotated RoI Align network first extracts the minimum bounding box of the region of interest in the K boxes and obtains the coordinates ( ), the coordinates of the real rotation frame are transformed into the coordinates using the ground truth frame as the coordinate system ( ), the predicted rotation frame coordinates are ( ), the loss value is calculated and used for loss function calculation. The calculation formula is as follows:

[0077]

[0078]

[0079]

[0080]

[0081] in, is the coordinate of the center point of the real rotating frame, To predict the coordinates of the center point of the rotating box, is the length and width of the real rotation box, To predict the length and width of the rotation box, is the rotation angle of the real rotating frame, To predict the rotation angle of the rotation box; First, calculate the difference between the predicted value and the true value ; Then, judge Is the absolute value of less than or equal to 1? if , then the square loss formula is used to calculate the loss value; if , the absolute loss formula is used to calculate the loss value.

[0082] The Smooth L1 loss function is defined as follows:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088] in, is the SmoothL1 loss function.

[0089] Furthermore, the classifier is trained based on the true label and the cross entropy loss function. The calculation process of the cross entropy loss function includes:

[0090] in, is the one-hot encoding of the true label, indicating The true probability of each category, Representative The predicted probability of each class.

[0091] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a dual-stage rotating target detection device. The dual-stage rotating target detection device can be implemented as an independent entity or integrated in an electronic device. The electronic device can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a micro processing box, or other devices.

[0092] See also Figure 4 , Figure 4 The dual-stage rotating target detection device provided by the embodiment of the present application is specifically described, and is applied to an electronic device. The dual-stage rotating target detection device may include: An acquisition module, used for acquiring remote sensing images; A feature extraction module is used to calculate the Manhattan distance between pixels in the remote sensing image, obtain global features based on the Manhattan distance, and obtain image features based on the global features; A candidate box generation module is used to perform convolution processing on image features to obtain a candidate feature map, and obtain a candidate box based on a predefined prior box and the candidate feature map; The non-maximum suppression module is used to select K predicted horizontal boxes based on the non-maximum suppression algorithm; The conversion and prediction module is used to convert the predicted horizontal box into a predicted rotation box, and obtain the target detection result of the remote sensing image based on the predicted rotation box.

[0093] During specific implementation, the above modules and / or units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules and / or units can refer to the previous method embodiments. The specific beneficial effects that can be achieved can also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0094] In addition, the embodiment of the present application also provides an electronic device, which may be a computer, a tablet computer, or other device. Figure 5 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 is electrically connected to the memory 402.

[0095] The processor 401 is the control center of the electronic device 400. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or loading applications stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the electronic device as a whole.

[0096] In this embodiment, the processor 401 in the electronic device 400 loads instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 runs the application programs stored in the memory 402 to implement various functions: Acquisition of remote sensing images; Inputting the remote sensing image into a ResNet network, calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; Input the image features into the RPN network for convolution processing to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; Based on the non-maximum suppression algorithm, K predicted horizontal boxes are selected; The predicted horizontal frame is converted into a predicted rotation frame, and the target detection result of the remote sensing image is obtained based on the predicted rotation frame.

[0097] The electronic device can implement the steps in any embodiment of the two-stage rotating target detection method provided in the embodiments of the present application. Therefore, it can achieve the beneficial effects that can be achieved by any two-stage rotating target detection method provided in the embodiments of the present invention. Please refer to the previous embodiments for details and will not be repeated here.

[0098] Figure 6The specific structural block diagram of the electronic device provided by the embodiment of the present invention is shown, and the electronic device can be used to implement the dual-stage rotating target detection method provided in the above embodiment. The electronic device 500 can be a terminal, a server, etc., wherein the terminal can include a tablet computer, a notebook computer, a personal computer (PC, Personal Computer), a micro processing box, or other devices.

[0099] The RF circuit 510 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks may use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0100] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, realizing the functions of taking pictures with the front camera, processing the captured images, and switching the display color of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 520 may further include a memory remotely arranged relative to the processor 580, and these remote memories may be connected to the electronic device 500 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0101] The input unit 530 may be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function control. The display unit 540 may be used to display information input by the user or information provided to the user and various graphical user interfaces, which may be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541, which may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like.

[0102] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, which is converted into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and converted into audio data, and then the audio data is output to the processor 580 for processing, and then sent to another terminal through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earplug jack to provide communication between an external headset and the electronic device 500.

[0103] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), which provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it is not a necessary component of the electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0104] The processor 580 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 520, and calling data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580.

[0105] The electronic device 500 also includes a power supply 590 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 580 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 590 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0106] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors. One or more programs include instructions for performing the following operations: Acquisition of remote sensing images; Inputting the remote sensing image into a ResNet network, calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; Input the image features into the RPN network for convolution processing to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; Based on the non-maximum suppression algorithm, K predicted horizontal boxes are selected; The predicted horizontal frame is converted into a predicted rotation frame, and the target detection result of the remote sensing image is obtained based on the predicted rotation frame.

[0107] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0108] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, which stores multiple instructions, and the instructions can be loaded by a processor to execute the steps of any embodiment of the dual-stage rotating target detection method provided by the embodiment of the present invention.

[0109] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0110] Since the instructions stored in the storage medium can execute the steps in any embodiment of the two-stage rotating target detection method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any two-stage rotating target detection method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0111] The above is a detailed introduction to a dual-stage rotating target detection method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A two-stage rotating target detection method, characterized in that: The method comprises: Acquisition of remote sensing images; Inputting the remote sensing image into a ResNet network, calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; Input the image features into the RPN network for convolution processing to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; Based on the non-maximum suppression algorithm, K predicted horizontal boxes are selected; The predicted horizontal frame is converted into a predicted rotation frame, and the target detection result of the remote sensing image is obtained based on the predicted rotation frame.

2. The dual-stage rotating target detection method according to claim 1, characterized in that: The step of inputting the remote sensing image into a ResNet network, calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features includes: Input the remote sensing image into the ResNet network to obtain a first feature map through convolution operation, batch normalization and maximum pooling operation; Calculate the Manhattan distance between pixels based on the two-dimensional coordinates of the pixels marked in the remote sensing image; Calculate the weight of each pixel in the first feature map; Obtaining a global feature based on the weight of each pixel and the Manhattan distance between the pixels; Perform multiple convolution operations on the global features to obtain image features.

3. The dual-stage rotating target detection method according to claim 1, characterized in that: The calculating the Manhattan distance between the pixels based on the two-dimensional coordinates of the pixels marked in the remote sensing image comprises: The Manhattan distance between pixels is calculated by the first formula, which is: Where d represents two pixels ( )and( ), D is the attenuation matrix in two-dimensional space, the elements of which represent the Manhattan distance based on the two-dimensional coordinates of the pixel points. It is an adjustable parameter that controls the influence of the distance between two pixels on the attention weight; The weight of each pixel in the first feature map is calculated, and the global feature is obtained based on the weight of each pixel and the Manhattan distance between the pixels, which is calculated by the following formula: in, is the first feature map, , and is the weight matrix used to transform the input into query (Q), key (K), and value (V) vectors, It is a vector used to represent the weights and biases in the ResNet network. is a complex matrix, The complex conjugate of , the self-attention mechanism used to calculate the attention score; It is a global feature. Softmax() represents the Softmax operation on the inner product of the query and the key, which is used to calculate the attention weight.

4. The dual-stage rotating target detection method according to claim 1, characterized in that: The method of selecting K predicted horizontal boxes based on the non-maximum suppression algorithm includes: Calculating the confidence of the candidate frame according to the coordinates of the candidate frame and the probability that an object exists in the candidate frame; According to the confidence, all candidate boxes are sorted from high to low confidence, and K predicted horizontal boxes with higher confidence are screened out.

5. The dual-stage rotating target detection method according to claim 1, characterized in that: Converting the predicted horizontal frame into a predicted rotation frame, and obtaining the target detection result of the remote sensing image based on the predicted rotation frame, comprising: Inputting the K predicted horizontal boxes into the trained Rotated RoI Align network to extract the minimum bounding boxes of the regions of interest in the K predicted horizontal boxes respectively, and obtaining the predicted rotation boxes based on the minimum bounding boxes; The predicted rotation frame is input into a classifier to obtain a target detection result of the remote sensing image.

6. The dual-stage rotating target detection method according to claim 5, characterized in that: The Rotated RoIAlign network is trained using the SmoothL1 loss function; The calculation process of the SmoothL1 loss function includes: in, is the coordinate of the center point of the real rotating frame, To predict the coordinates of the center point of the rotating box, is the length and width of the real rotation box, To predict the length and width of the rotation box, is the rotation angle of the real rotating frame, To predict the rotation angle of the rotation box; in, is the SmoothL1 loss function.

7. The dual-stage rotating target detection method according to claim 5, characterized in that: The classifier is trained by a cross entropy loss function, and the calculation process of the cross entropy loss function includes: in, is the one-hot encoding of the true label, indicating The true probability of each category, Representative The predicted probability of each class.

8. A dual-stage rotating target detection device, characterized in that: include: An acquisition module, used for acquiring remote sensing images; A feature extraction module, used for calculating the Manhattan distance between pixels in the remote sensing image, obtaining global features based on the Manhattan distance, and obtaining image features based on the global features; A candidate frame generation module is used to perform convolution processing on the image features to obtain a candidate feature map, and obtain a candidate frame based on a predefined prior frame and the candidate feature map; The non-maximum suppression module is used to select K predicted horizontal boxes based on the non-maximum suppression algorithm; The conversion and prediction module is used to convert the predicted horizontal frame into a predicted rotation frame, and obtain the target detection result of the remote sensing image based on the predicted rotation frame.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the dual-stage rotating target detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the two-stage rotating target detection method according to any one of claims 1 to 7.

Images