A rapid deployment method, system and device for remote sensing image rotation target detection

Abstract

The present invention provides a method, system, and device for rapid deployment of rotation target detection in remote sensing images. Through Vitis AI's quantization tools and executable files, the remote sensing image target detection model is efficiently mapped to the DPU for execution, thereby maximizing the use of FPGA computing resources and improving computing efficiency. During the mapping period, by optimizing and equivalently replacing operators not supported by the DPU, it is ensured that the preset target detection model can all run on the DPU, realizing the effective allocation of operators. Secondly, the ONNX model format and Vitis AI interface are used to enhance the compatibility and scalability of the preset target detection model, so that the preset target detection model can be seamlessly migrated and deployed on different frameworks and hardware platforms. Finally, the present invention simplifies the deployment process of the preset target detection model on hardware, reduces the technical threshold and deployment cost of developers, and improves the real-time performance of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic design automation (EDA) and programmable logic device (PLD), and in particular to a method, system and device for rapid deployment of rotating target detection in remote sensing images. Background Art

[0002] In modern applications, target detection technology in remote sensing images is becoming increasingly important, especially in areas such as environmental monitoring, resource management, and disaster response. With the widespread use of platforms such as satellites and drones, the ability to acquire high-resolution remote sensing imagery has significantly increased. However, processing this massive amount of data and quickly and accurately identifying valuable information from it has become a challenge. Traditional rule-based methods struggle to cope with the complex and ever-changing natural environment and changes in target characteristics. In recent years, deep learning algorithms, due to their powerful feature extraction capabilities and adaptability, have made significant progress in remote sensing image target detection, greatly improving detection performance.

[0003] In order to meet the stringent requirements for real-time performance and high efficiency, hardware acceleration has become the key to improving the computational efficiency of deep learning models. The combination of multi-processor system-on-chip (MPSoC) and FPGA (Field-Programmable Gate Array) technology provides an effective solution. This heterogeneous architecture integrates programmable logic and programmable processor systems on the same chip, allowing the PL part to perform computationally intensive tasks such as image processing and deep learning reasoning, while the PS part is responsible for running the operating system and applications. For example: Xilinx's Vitis AI development platform provides comprehensive support for accelerating machine learning and deep learning applications on FPGAs, including tools, libraries, and optimized models, allowing developers to more easily deploy AI models to hardware platforms for efficient model reasoning and data processing.

[0004] Existing technical solutions have implemented full-process acceleration methods based on the Vitis stack, deploying deep learning models to hardware platforms through steps such as quantization, compilation, and mapping. However, these solutions are primarily designed with a focus on general application scenarios, with less consideration given to optimizations for specific application scenarios. For example, when processing large-scale remote sensing imagery, data transmission bottlenecks and memory bandwidth limitations may arise, further restricting the system's real-time responsiveness. Furthermore, current technical solutions typically employ relatively fixed model deployment strategies, making it impossible to flexibly adjust the allocation of computing tasks based on the needs of actual application scenarios, thereby affecting the overall performance and flexibility of the system. Summary of the Invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a method, system and device for rapid deployment of remote sensing image rotation target detection.

[0006] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a rapid deployment method for detecting rotating targets in remote sensing images, which is applied to a rapid deployment system. The rapid deployment system includes: a server, a host computer, and a development board, including:

[0008] The server trains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, converts it into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer.

[0009] The host computer obtains the development board model and the Vitis-AI version that matches the development board model and installs it.

[0010] The development board obtains the Vitis-AI version and selects a system image that is consistent with the Vitis-AI version and burns it into its own SD card;

[0011] The host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and then sends the fixed-point .onnx file to the development board.

[0012] The development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and executable file. Among them, the preset target detection model replaces the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model.

[0013] Optionally, before the server sends the floating-point .onnx file to the host computer, it also includes:

[0014] Call the onnx-simplifier tool in the deep learning framework pytorch to remove redundant nodes in the floating-point .onnx file.

[0015] Optionally, the preset target detection model adopts an improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts the ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; the preset convolution processing is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and the batch normalization BN processing is used to equivalently replace the mul operation in the initial yolov8-obb ​​model.

[0016] Optionally, the server obtains a floating-point .pth file based on a preset target detection model training, converts it into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer, including:

[0017] The server obtains initial remote sensing image samples;

[0018] The initial remote sensing image samples are segmented using a sliding window method to obtain a remote sensing image sample set of a preset size;

[0019] Input the remote sensing image sample set into the preset target detection model for training to obtain a floating-point .pth file;

[0020] Use the deep learning framework pytorch to convert the floating-point .pth file into a floating-point .onnx file, and send the floating-point .onnx file to the host computer.

[0021] Optionally, the host computer further adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

[0022] Optionally, after the development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and executable file, it also includes:

[0023] The host computer obtains the remote sensing image to be detected and sends it to the development board;

[0024] The development board loads and parses the fixed-point .onnx file to generate the model graph structure;

[0025] Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure that satisfies the operator type to the remote sensing image to be detected to the DPU for processing;

[0026] The DPU outputs the target detection results for the remote sensing image to be detected.

[0027] Optionally, the preset object detection model adds unsqueeze layers between the preset layers of the initial yolov8-obb ​​model to expand the dimensionality of the 3D tensor of the initial yolov8-obb ​​model.

[0028] In a second aspect, the present invention provides a rapid deployment system for detecting rotating targets in remote sensing images, comprising: a server, a host computer, and a development board;

[0029] The server is used to train a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, convert it into a floating-point .onnx file using the deep learning framework pytorch, and send the floating-point .onnx file to the host computer;

[0030] The host computer obtains the development board model and the compatible Vitis-AI version for installation. It then calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, which is then sent to the development board.

[0031] The development board is used to obtain the Vitis-AI version and executable files compiled by the host computer, and select a system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and executable files, the fixed-point .onnx file is mapped to the development board. Among them, the preset object detection model equivalently replaces the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model.

[0032] In a third aspect, the present invention provides a rapid deployment device for detecting rotating targets in remote sensing images, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the rapid deployment device for detecting rotating targets in remote sensing images is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the rapid deployment method for detecting rotating targets in remote sensing images as described in the first aspect above.

[0033] The present invention provides a rapid deployment method, system, and device for remote sensing image rotation target detection. A rapid deployment method for remote sensing image rotation target detection is applied to a rapid deployment system, which includes a server, a host computer, and a development board. The method includes: the server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set for training, converts the file into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer; the host computer obtains the development board model and obtains the Vitis-AI version compatible with the development board model for installation; the development board obtains the Vitis-AI version and selects the Vitis-AI version compatible with the Vitis-AI version. The system image with the same AI version is burned into its own SD card. The host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and then sends the fixed-point .onnx file to the development board. The development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and executable file. Among them, the preset object detection model performs equivalent replacement for the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model. In the present invention, the remote sensing image target detection model is efficiently mapped to the DPU for execution through Vitis AI's quantization tools and executable files, thereby maximizing the utilization of FPGA computing resources and improving computing efficiency. During the mapping period, by optimizing and equivalently replacing operators not supported by the DPU, it is ensured that the preset target detection model can all run on the DPU, realizing effective allocation of operators. Secondly, the ONNX model format and Vitis AI interface are adopted to enhance the compatibility and scalability of the preset target detection model, enabling the preset target detection model to be seamlessly migrated and deployed on different frameworks and hardware platforms. Finally, the present invention simplifies the deployment process of the preset target detection model on hardware, reduces the technical threshold and deployment cost of developers, and at the same time improves the real-time performance of the system, making it more suitable for remote sensing image target detection scenarios with high efficiency, strong real-time performance, and flexible and changeable task requirements.

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a process flow of a rapid deployment method for detecting rotating objects in remote sensing images provided by an embodiment of the present invention;

[0036] Figure 2 The comparison results of two images tested on the PyTorch side and the development board side are shown as an example;

[0037] Figure 3 A schematic structural diagram of a rapid deployment device for detecting rotating targets in remote sensing images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0039] To improve the computational efficiency of remote sensing image target detection while enhancing system compatibility and scalability, an embodiment of the present invention provides a rapid deployment method for rotating target detection in remote sensing images. This method is applied to a rapid deployment system comprising a server, a host computer, and a development board. Figure 1 A flowchart of a rapid deployment method for detecting rotating targets in remote sensing images provided by an embodiment of the present invention is shown as follows: Figure 1 As shown, including:

[0040] S101. The server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, converts it into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer.

[0041] Optionally, S101 may specifically include:

[0042] The server obtains initial remote sensing image samples;

[0043] The initial remote sensing image samples are segmented using a sliding window method to obtain a remote sensing image sample set of a preset size;

[0044] Input the remote sensing image sample set into the preset target detection model for training to obtain a floating-point .pth file;

[0045] Use the deep learning framework pytorch to convert the floating-point .pth file into a floating-point .onnx file, and send the floating-point .onnx file to the host computer.

[0046] It should be noted that the initial remote sensing image samples in the embodiment of the present invention may specifically be the optical remote sensing image dataset DOTA-1.0. In addition, a sliding window size of 200 may be used to segment the initial remote sensing image samples to obtain a 1024*1024 remote sensing image sample set.

[0047] Optionally, before the server sends the floating-point .onnx file to the host computer, it also includes:

[0048] Call the onnx-simplifier tool in the deep learning framework pytorch to remove redundant nodes in the floating-point .onnx file.

[0049] It should be noted that the embodiment of the present invention converts the preset target detection model into a floating-point .onnx file. This is primarily because the open format of onnx supports conversion between multiple deep learning frameworks, facilitates the interchange of models between different frameworks, and effectively overcomes compatibility barriers between platforms. Furthermore, based on optimization needs, developers can modify operators directly in the onnx model, including but not limited to replacing, adding, or deleting operators. This flexibility allows developers to directly evaluate the effectiveness of new algorithms without having to retrain the entire model, significantly improving the efficiency of the deployment process.

[0050] S102: The host computer obtains the development board model of the development board, obtains the Vitis-AI version that is compatible with the development board model, and performs installation.

[0051] It should be noted that in this embodiment, the development board is a circuit board used for embedded system development, which includes hardware components such as processor, memory, input and output interfaces, and provides an actual operating environment for the development and testing of the rapid deployment system in the present invention.

[0052] S103. The development board obtains the Vitis-AI version and selects a system image that is consistent with the Vitis-AI version and burns it into its own SD card.

[0053] S104. The host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board.

[0054] Specifically, during the quantization process, in order to improve the accuracy of the quantization model, vai_q_onnx will use the calibration dataset to calibrate the activations and adjust the quantization parameters so that the quantized fixed-point .onnx file can better reflect the characteristics of the original data and better map high-precision values ​​to low-precision values, thereby reducing the consumption of hardware resources while maintaining the accuracy of the preset target detection model. No backpropagation is required, and 100 unlabeled images need to be provided as a calibration dataset.

[0055] Key code of quantization step:

[0056]

[0057] pytorch_nndct is a module in the Vitis AI toolchain specifically designed for quantizing, compiling, and deploying PyTorch models. These functions provide basic operators used in quantized inference and are optimized for hardware accelerators such as DPUs. This code demonstrates the configuration process for statically quantizing a .onnx floating-point model into a .onnx file. It specifies several key parameters for fine-grained control of the quantization operation and subsequent optimizations. Setting quant_format to vai_q_onnx.VitisQuantFormat.QDQ specifies the QDQ quantization format, which means that quantization (QuantizeLinear) and dequantization (DeQuantizeLinear) operation nodes are inserted into the model to achieve quantization. Setting calibrate_method to vai_q_onnx.PowerOfTwoMethod.MinMSE determines the calibration method. The MinMSE method aims to minimize the mean squared error (MSE) caused by the data representation before and after quantization, minimizing the loss in accuracy of the quantized model. It also utilizes the power-of-two rule to adapt to the hardware, resulting in higher computational efficiency. The activation_type parameter is set to QuantType.QUInt8. Since the activation function is ReLU, which doesn't require negative numbers, the quantized data range is between 0 and 255. The weight_type parameter is set to QuantType.QInt8, specifying that the quantization type of the model weight data is a signed 8-bit integer (QInt8). This reduces the model's storage and computational complexity while maintaining reasonable accuracy. The optimize_model parameter is set to True, indicating that during or after quantization, the model will be cleaned of redundant nodes and the computation graph will be merged. This allows the quantized model to achieve better inference speed.

[0058] S105. The development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and the executable file.

[0059] In an embodiment of the present invention, the executable file is a coding program input by the programmer based on the VOE platform installed on the host computer, which is used to control and manage the operation of the fixed-point .onnx file on the development board. Specifically, the executable file achieves optimal performance by allocating tasks between the CPU and the DPU. The first step of the executable file is to load and parse the preset target detection model in the .onnx format to generate the graph structure of the model. After that, the specific computing operations in the preset target detection model will be mapped to the DPU for operation according to the operator type supported by the DPU. Finally, the post-processing part is to post-process the data output by the preset target detection model, including decoding the classification results, extracting the bounding box coordinates, converting the rotation angle components, performing non-maximum suppression (NMS) and other operations.

[0060] The VOE platform in this embodiment is based on the Vitis AION NX Runtime Engine. The executable file converts the trained deep learning model (ONNX format) through the VOE platform's quantization, compilation, and other toolchains to microcode that can be executed on a specific hardware accelerator (DPU), thereby enabling rapid target detection for data such as remote sensing images.

[0061] An embodiment of the present invention provides a rapid deployment method for remote sensing image rotation target detection, which is applied to a rapid deployment system. The rapid deployment system includes: a server, a host computer, and a development board. The method includes: using Vitis AI's quantization tools and executable files, efficiently mapping the remote sensing image target detection model to the DPU for execution, thereby maximizing the use of FPGA computing resources and improving computing efficiency. During the mapping period, by optimizing and equivalently replacing operators not supported by the DPU, it is ensured that the preset target detection model can all run on the DPU, thereby achieving effective allocation of operators. Secondly, the ONNX model format and Vitis AI interface are adopted to enhance the compatibility and scalability of the preset target detection model, so that the preset target detection model can be seamlessly migrated and deployed on different frameworks and hardware platforms. Finally, the present invention simplifies the deployment process of the preset target detection model on hardware, reduces the technical threshold and deployment cost of developers, and at the same time improves the real-time performance of the system, making it more suitable for remote sensing image target detection scenarios with high efficiency, strong real-time performance, and flexible and changeable task requirements.

[0062] Optionally, after S105, the following steps are further included:

[0063] The host computer obtains the remote sensing image to be detected and sends it to the development board;

[0064] The development board loads and parses the fixed-point .onnx file to generate the model graph structure;

[0065] Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure that satisfies the operator type to the remote sensing image to be detected to the DPU for processing;

[0066] The DPU outputs the target detection results for the remote sensing image to be detected.

[0067] In addition, it should be noted that the action of loading and parsing the fixed-point .onnx file and generating the model graph structure can be specifically controlled and executed through an executable file.

[0068] Among them, the preset target detection model performs equivalent replacements for the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model.

[0069] Optionally, the preset target detection model adopts an improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts the ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; the preset convolution processing is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and the batch normalization BN processing is used to equivalently replace the mul operation in the initial yolov8-obb ​​model.

[0070] Optionally, the host computer further adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

[0071] It should be noted that, in the embodiment of the present invention, the quantization node QuantizeLinear and the dequantization node DeQuantizeLinear can be exemplarily added between the convolution layer, the pool layer, the add layer and the concat layer.

[0072] In an embodiment of the present invention, QuantizeLinear and DeQuantizeLinear nodes are inserted into the original floating-point model. These nodes simulate the quantization behavior on actual hardware, but maintain the differentiability of the calculation, allowing gradients to propagate through them. The QuantizeLinear node converts floating-point values ​​to lower-precision integer representations (e.g., int8). The DeQuantizeLinear node is responsible for converting these quantized integers back into floating-point numbers. This mode is typically used for models in the onnx format, which can provide more refined control over the quantization process and better maintain the accuracy of the model.

[0073] Optionally, the preset object detection model adds unsqueeze layers between the preset layers of the initial yolov8-obb ​​model to expand the dimensionality of the 3D tensor of the initial yolov8-obb ​​model.

[0074] It should be noted that the unsqueeze layer can be added after the convolution layer in the initial yolov8-obb ​​model that does not meet the data arrangement rules of the development board.

[0075] In addition, in order to reduce the amount of calculation, the confidence threshold can be set to 0.3 and the non-maximum suppression nms threshold can be set to 0.5. The detection frame is first screened with a threshold of 0.3, and then the non-maximum suppression method is used to filter out redundant frames.

[0076] In order to verify the effectiveness of the rapid deployment method for detecting rotating targets in remote sensing images provided by the embodiment of the present invention, a simulation experiment was also conducted in the embodiment of the present invention.

[0077] 1. Simulation experiment conditions:

[0078] The hardware of the simulation experiment of the embodiment of the present invention includes: zcu102 board, serial cable, network cable, and SD card.

[0079] The software platforms for the simulation experiments in the embodiments of the present invention are: Ubuntu 20.04, Vitis-AI 3.5, and finalshell 4.3.

[0080] 2. Simulation content:

[0081] The test data for this embodiment of the present invention consists of 200 randomly selected test images from the DOTA-1.0 dataset, with a size of 1024×1024. The GPU used is an NVIDIA GeForce GTX TITAN X, and the CPU is an Intel(R) Xeon(R) E5-2630 v4 @ 2.20GHz. A comparative experiment was conducted using the original yolov8-obb ​​model and the preset object detection model of this embodiment of the present invention. The results are shown in Table 1.

[0082] Table 1 Comparison of the detection results of the method of the present invention and the original yolov8-obb ​​model

[0083]

[0084] Based on Table 1, it can be seen that by adopting the method of the present invention, after quantization, the model size is reduced to 35% of the previous size, which greatly saves storage space. In terms of time consumption, compared with GPU devices, due to the limited computing power and hardware resources of edge devices, the FPS index during inference is lower than that of GPU devices. Compared with CPU devices, the DPU in the hardware has a computing engine specifically responsible for executing common mathematical operations in deep learning such as matrix multiplication and convolution, and an efficient data flow controller that can effectively utilize various resources on the FPGA. The CPU is limited by the number of cores and main frequency, and the FPS is significantly lower. In addition, due to the existence of quantization errors and the differences in numerical accuracy between different platforms, the board-side results are 1.7% lower than the PC-side performance.

[0085] Figure 2 The comparison results of testing two images on the pytorch side and the development board side are shown as an example. Specifically, Figure 2 The two figures on the left are the target detection results on the pytorch side based on the existing yolov8-obb ​​model. Figure 2 The two figures on the right are the target detection results on the development board based on the preset target detection model of the present invention. The processing conditions and experimental objects of the two methods are the same. Figure 2 It can be seen that the bounding box positions and sizes given by the two are basically the same, which shows that when the model is optimized and ported to a dedicated hardware accelerator, it can still maintain its original performance, verifying the effectiveness of the method of the present invention.

[0086] The rapid deployment method for detecting rotating targets in remote sensing images provided by the embodiments of the present invention has the following technical advantages:

[0087] 1. Maximizing resource utilization: The deployment methods mentioned in the prior art do not place as many low-power and high-efficiency computing tasks as possible on the FPGA based on the resource characteristics of the current heterogeneous platform. However, the present invention quantizes the model through Vitis AI's quantization tool vai_q_onnx, and encodes the host file based on VOE (Vitis AIONNX Runtime Engine) to map the model's control flow and data flow to the hardware platform. This ensures that the model can run efficiently on the DPU (a dedicated processing unit on the FPGA), thereby maximizing the utilization of the FPGA's computing resources.

[0088] 2. Efficient Operator Allocation: Existing technologies fail to consider the different optimal data layouts for different computing units (DPUs, CPUs) and effectively allocate operators within the model based on hardware characteristics. This invention optimizes the floating-point .onnx file before model deployment, modifying or replacing some layers unsupported by the DPU to ensure that the entire model can be mapped to the DPU. This step ensures that operators in the model are effectively allocated to the hardware units most suitable for their execution, thereby leveraging the advantages of the hardware.

[0089] 3. Model deployment flexibility: While the deployment methods mentioned in the prior art propose acceleration methods based on the Vitis stack, they do not fully consider the flexibility of model deployment on different hardware platforms. This invention uses the ONNX model format to enable model sharing between different deep learning frameworks (such as TensorFlow and PyTorch), and imports the model into Xilinx series hardware for optimization and deployment through the vai_q_onnx interface of Vitis AI. This improves the flexibility and compatibility of model deployment.

[0090] In summary, the present invention significantly improves the computational efficiency of remote sensing image target detection by deploying the model on the DPU and utilizing the low power consumption and high energy efficiency characteristics of FPGA. This solution effectively allocates the operators in the model according to the hardware characteristics, ensures the maximum utilization of computing resources, and avoids unnecessary waste of resources. Through the ONNX model format and the vai_q_onnx interface of Vitis AI, seamless migration and deployment of the model between different frameworks and hardware platforms are achieved, enhancing the compatibility and scalability of the model. The complete set of tools, libraries and optimization models provided by Vitis AI simplifies the model deployment process on hardware, reducing the technical threshold and deployment cost of developers. Efficient model deployment and resource utilization enable the remote sensing image target detection network to adapt to military application scenarios with high efficiency, strong real-time performance, and flexible and changeable task requirements, thereby improving the overall performance and real-time response capabilities of the system.

[0091] Based on the same inventive concept, an embodiment of the present invention further provides a rapid deployment system for detecting rotating targets in remote sensing images, the system comprising: a server, a host computer, and a development board;

[0092] The server is used to train a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, convert it into a floating-point .onnx file using the deep learning framework pytorch, and send the floating-point .onnx file to the host computer;

[0093] The host computer obtains the development board model and the compatible Vitis-AI version for installation. It then calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, which is then sent to the development board.

[0094] The development board is used to obtain the Vitis-AI version and executable files compiled by the host computer, and select a system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and executable files, the fixed-point .onnx file is mapped to the development board. Among them, the preset object detection model equivalently replaces the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model.

[0095] Figure 3 A schematic diagram of the structure of a rapid deployment device for rotating target detection in remote sensing images provided by an embodiment of the present invention includes: a processor 310, a storage medium 320, and a bus 330. The storage medium 320 stores machine-readable instructions executable by the processor 310. When the rapid deployment device for rotating target detection in remote sensing images is in operation, the processor 310 and the storage medium 320 communicate via the bus 330, and the processor 310 executes the machine-readable instructions to perform the steps of the above-described method embodiment. The specific implementation methods and technical effects are similar and will not be repeated here.

[0096] The storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the storage medium may be at least one storage device located away from the processor.

[0097] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0098] In the description of this specification, the reference terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0099] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the above-mentioned disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the word "comprising" does not exclude other components or steps, "one" or "an" does not exclude multiple situations, and the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0100] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention cannot be considered to be limited to these descriptions. For those skilled in the art of the present invention, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the scope of protection of the present invention.

Claims

1. A rapid deployment method for detecting rotating targets in remote sensing images, applied to a rapid deployment system, the rapid deployment system comprising: The server, host computer and development board are characterized by including: The server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, converts the file into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to the host computer; The host computer obtains the development board model of the development board, and obtains the Vitis-AI version adapted to the development board model for installation; The development board obtains the Vitis-AI version, and selects a system image consistent with the Vitis-AI version and burns it into its own SD card; The host computer calls the vai_q_onnx tool in the Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board; The development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and the executable file; wherein the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model; Among them, the preset target detection model adopts the improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts the ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; the preset convolution processing is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and the batch normalization BN processing is used to equivalently replace the mul operation in the initial yolov8-obb ​​model; The preset target detection model adds an unsqueeze layer between the preset layers of the initial yolov8-obb ​​model to expand the dimension of the 3D tensor of the initial yolov8-obb ​​model.

2. The rapid deployment method for detecting rotating targets in remote sensing images according to claim 1, characterized in that: Before the server sends the floating-point .onnx file to the host computer, the server further includes: The onnx-simplifier tool in the deep learning framework pytorch is called to remove redundant nodes in the floating-point .onnx file.

3. The rapid deployment method for detecting rotating targets in remote sensing images according to claim 1, characterized in that: The server obtains a floating-point .pth file based on a preset target detection model training, converts it into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer, including: The server obtains an initial remote sensing image sample; Segmenting the initial remote sensing image samples using a sliding window method to obtain a remote sensing image sample set of a preset size; Inputting the remote sensing image sample set into the preset target detection model for training to obtain the floating-point .pth file; The floating-point .pth file is converted into a floating-point .onnx file using the deep learning framework pytorch, and the floating-point .onnx file is sent to the host computer.

4. The rapid deployment method for detecting rotating targets in remote sensing images according to claim 1, characterized in that: The host computer further adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

5. The rapid deployment method for detecting rotating targets in remote sensing images according to claim 1, characterized in that: The development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and the executable file, further comprising: The host computer obtains the remote sensing image to be detected and sends the remote sensing image to be detected to the development board; The development board loads and parses the fixed-point .onnx file to generate a model graph structure; Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure on the remote sensing image to be detected that satisfies the operator type to the DPU for processing; The DPU outputs a target detection result for the remote sensing image to be detected.

6. A rapid deployment system for detecting rotating targets in remote sensing images, characterized in that: include: Server, host computer and development board; The server is used to train a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, convert the file into a floating-point .onnx file using a deep learning framework pytorch, and send the floating-point .onnx file to the host computer; The host computer is used to obtain the development board model of the development board, obtain the Vitis-AI version adapted to the development board model and perform installation, call the vai_q_onnx tool in the Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and send the fixed-point .onnx file to the development board; The development board is used to obtain the Vitis-AI version and the executable file compiled by the host computer, and select a system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and the executable file, the fixed-point .onnx file is mapped to the development board; wherein the preset target detection model performs an equivalent replacement for the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model; Among them, the preset target detection model adopts the improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts the ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; the preset convolution processing is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and the batch normalization BN processing is used to equivalently replace the mul operation in the initial yolov8-obb ​​model; The preset target detection model adds an unsqueeze layer between the preset layers of the initial yolov8-obb ​​model to expand the dimension of the 3D tensor of the initial yolov8-obb ​​model.

7. A rapid deployment device for detecting rotating targets in remote sensing images, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the rapid deployment device for rotating target detection in remote sensing images is in operation, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the rapid deployment method for rotating target detection in remote sensing images as described in any one of claims 1 to 5.

Images