Remote sensing image rotating target detection rapid deployment method, system and equipment

Abstract

The invention provides a remote sensing image rotating target detection rapid deployment method, system and device. According to the method, a remote sensing image target detection model is efficiently mapped to a DPU (Data Processing Unit) for execution through a Vitis AI quantification tool and an executable file, so that the computing resources of an FPGA (Field Programmable Gate Array) are utilized to the maximum extent, and the computing efficiency is improved; in the mapping period, operators which are not supported by the DPU are optimized and equivalently replaced, it is ensured that the preset target detection model can all run on the DPU, and effective distribution of the operators is achieved; secondly, an ONNX model format and a Viis AI interface are adopted, so that the compatibility and expandability of the preset target detection model are enhanced, and the preset target detection model can be seamlessly migrated and deployed on different frames and hardware platforms; and finally, the deployment process of the preset target detection model on hardware is simplified, the technical threshold and deployment cost of developers are reduced, and meanwhile, the real-time performance of the system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic design automation (EDA) and programmable logic device (PLD), and in particular to a method, system and device for quickly deploying rotating target detection in remote sensing images. Background Art

[0002] In modern applications, the importance of remote sensing image target detection technology has become increasingly prominent, especially in the fields of environmental monitoring, resource management, and disaster response. With the widespread use of platforms such as satellites and drones, the ability to obtain high-resolution remote sensing images has been significantly enhanced. However, processing this massive amount of data and quickly and accurately identifying valuable information from it has become a challenge. Traditional rule-based methods have difficulty coping with complex and changing natural environments and changes in target features. In recent years, deep learning algorithms have made significant progress in remote sensing image target detection due to their powerful feature extraction capabilities and adaptability, greatly improving detection performance.

[0003] In order to meet the strict requirements for real-time and high efficiency, hardware acceleration has become the key to improving the computing efficiency of deep learning models. The combination of multi-processor system-on-chip (MPSoC) and FPGA (Field-Programmable Gate Array) technology provides an effective solution. This heterogeneous architecture integrates programmable logic and programmable processor systems on the same chip, allowing the PL part to perform computationally intensive tasks such as image processing and deep learning reasoning, while the PS part is responsible for the operation of the operating system and applications. For example: The Vitis AI development platform launched by Xilinx provides comprehensive support for accelerating machine learning and deep learning applications on FPGAs, including tools, libraries, and optimized models, allowing developers to more easily deploy AI models to hardware platforms to achieve efficient model reasoning and data processing.

[0004] Existing technical solutions have implemented a full-process acceleration method based on the Vitis stack, deploying deep learning models to hardware platforms through steps such as quantization, compilation, and mapping. However, these solutions are mainly designed to focus on general application scenarios, and less consideration is given to optimization in specific application scenarios. For example, when processing large-scale remote sensing images, data transmission bottlenecks and memory bandwidth limitations may be encountered, which further restricts the system's real-time response capabilities. In addition, current technical solutions usually adopt a relatively fixed model deployment strategy, which cannot flexibly adjust the allocation of computing tasks according to the needs of actual application scenarios, thereby affecting the overall performance and flexibility of the system. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a method, system and device for rapid deployment of remote sensing image rotating target detection.

[0006] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] In a first aspect, the present invention provides a rapid deployment method for remote sensing image rotation target detection, which is applied to a rapid deployment system. The rapid deployment system includes: a server, a host computer, and a development board, including:

[0008] The server obtains a floating-point .pth file based on the preset target detection model and remote sensing image sample set, converts it into a floating-point .onnx file using the deep learning framework pytorch, and sends the floating-point .onnx file to the host computer;

[0009] The host computer obtains the development board model of the development board and obtains the Vitis-AI version that is compatible with the development board model for installation;

[0010] The development board obtains the Vitis-AI version and selects a system image that is consistent with the Vitis-AI version and burns it into its own SD card;

[0011] The host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board;

[0012] The development board obtains the executable file compiled by the host computer, and maps the fixed-point .onnx file to the development board under the control of the system image and executable file; among them, the preset target detection model equivalently replaces the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model.

[0013] Optionally, before the server sends the floating-point .onnx file to the host computer, the server also includes:

[0014] Call the onnx-simplifier tool in the deep learning framework pytorch to remove redundant nodes in the floating-point .onnx file.

[0015] Optionally, the preset target detection model adopts an improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts a ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; a preset convolution process is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and a batch normalization BN process is used to equivalently replace the mul operation in the initial yolov8-obb ​​model.

[0016] Optionally, the server obtains a floating-point .pth file based on a preset target detection model training, converts it into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to a host computer, including:

[0017] The server obtains initial remote sensing image samples;

[0018] The initial remote sensing image samples are segmented by a sliding window method to obtain a remote sensing image sample set of a preset size;

[0019] Input the remote sensing image sample set into the preset target detection model for training to obtain a floating point .pth file;

[0020] The deep learning framework pytorch is used to convert the floating-point .pth file into a floating-point .onnx file, and the floating-point .onnx file is sent to the host computer.

[0021] Optionally, the host computer further adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

[0022] Optionally, after the development board obtains the executable file compiled by the host computer and maps the fixed-point .onnx file to the development board under the control of the system image and the executable file, it also includes:

[0023] The host computer obtains the remote sensing image to be detected and sends the remote sensing image to be detected to the development board;

[0024] The development board loads and parses the fixed-point .onnx file to generate the model graph structure;

[0025] Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure that satisfies the operator type to the remote sensing image to be detected to the DPU for processing;

[0026] The DPU outputs the target detection results for the remote sensing image to be detected.

[0027] Optionally, the preset object detection model adds unsqueeze layers between the preset layers of the initial yolov8-obb ​​model to expand the dimension of the 3D tensor of the initial yolov8-obb ​​model.

[0028] In a second aspect, the present invention provides a rapid deployment system for remote sensing image rotation target detection, including: a server, a host computer, and a development board;

[0029] The server is used to train a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, and convert it into a floating-point .onnx file using the deep learning framework pytorch, and send the floating-point .onnx file to the host computer;

[0030] The host computer is used to obtain the development board model of the development board, obtain the Vitis-AI version that is compatible with the development board model, and perform installation. It calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board.

[0031] The development board is used to obtain the Vitis-AI version and the executable file compiled by the host computer, and select the system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and executable file, the fixed-point .onnx file is mapped to the development board; among them, the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model.

[0032] In a third aspect, the present invention provides a rapid deployment device for remote sensing image rotation target detection, comprising: a processor, a storage medium and a bus, the storage medium storing machine-readable instructions executable by the processor, when the rapid deployment device for remote sensing image rotation target detection is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the rapid deployment method for remote sensing image rotation target detection as described in the first aspect above.

[0033] The present invention provides a method, system and device for rapid deployment of remote sensing image rotation target detection. A rapid deployment method for remote sensing image rotation target detection is applied to a rapid deployment system, which includes: a server, a host computer and a development board, including: the server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, and converts it into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to the host computer; the host computer obtains the development board model of the development board, and obtains the Vitis-AI version adapted to the development board model for installation; the development board obtains the Vitis-AI version, and selects the Vitis-AI version compatible with the Vitis- The system image with the same AI version is burned into its own SD card; the host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain the fixed-point .onnx file, and sends the fixed-point .onnx file to the development board; the development board obtains the executable file compiled by the host computer, and maps the fixed-point .onnx file to the development board under the control of the system image and executable file; among them, the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model. In the present invention, the remote sensing image target detection model is efficiently mapped to the DPU for execution through the quantization tool and executable file of Vitis AI, thereby maximizing the use of FPGA computing resources and improving computing efficiency; during the mapping period, by optimizing and equivalently replacing operators not supported by the DPU, it is ensured that the preset target detection model can all run on the DPU, and the effective allocation of operators is achieved; secondly, the ONNX model format and the interface of Vitis AI are adopted to enhance the compatibility and scalability of the preset target detection model, so that the preset target detection model can be seamlessly migrated and deployed on different frameworks and hardware platforms; finally, the present invention simplifies the deployment process of the preset target detection model on hardware, reduces the technical threshold and deployment cost of developers, and improves the real-time performance of the system, making it more suitable for remote sensing image target detection scenarios with high efficiency, strong real-time performance, and flexible and changeable task requirements.

[0034] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a process flow of a rapid deployment method for detecting rotating targets in remote sensing images provided by an embodiment of the present invention;

[0036] Figure 2 The comparison results of testing two images on the pytorch side and the development board side are shown as an example;

[0037] Figure 3 A schematic structural diagram of a rapid deployment device for detecting rotating targets in remote sensing images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0039] In order to improve the computational efficiency of the system for remote sensing image target detection and improve the system compatibility and scalability, an embodiment of the present invention provides a rapid deployment method for remote sensing image rotation target detection. The method is applied to a rapid deployment system, which includes: a server, a host computer and a development board. Figure 1 A schematic diagram of a process flow of a rapid deployment method for detecting rotating targets in remote sensing images provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, including:

[0040] S101. The server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, converts it into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to a host computer.

[0041] Optionally, S101 may specifically include:

[0042] The server obtains initial remote sensing image samples;

[0043] The initial remote sensing image samples are segmented by a sliding window method to obtain a remote sensing image sample set of a preset size;

[0044] Input the remote sensing image sample set into the preset target detection model for training to obtain a floating point .pth file;

[0045] The deep learning framework pytorch is used to convert the floating-point .pth file into a floating-point .onnx file, and the floating-point .onnx file is sent to the host computer.

[0046] It should be noted that the initial remote sensing image samples in the embodiment of the present invention may specifically be the optical remote sensing image dataset DOTA-1.0. In addition, a sliding window size of 200 may be used to segment the initial remote sensing image samples to obtain a 1024*1024 remote sensing image sample set.

[0047] Optionally, before the server sends the floating-point .onnx file to the host computer, the server also includes:

[0048] Call the onnx-simplifier tool in the deep learning framework pytorch to remove redundant nodes in the floating-point .onnx file.

[0049] It should be noted that the embodiment of the present invention converts the preset target detection model into a floating-point .onnx file. The main purpose is that the open format of onnx supports conversion between multiple deep learning frameworks, can promote the interchange of models between different frameworks, and can effectively overcome the compatibility barriers between platforms. In addition, based on optimization requirements, developers can directly modify operators in the onnx model, including but not limited to replacing, adding or deleting operators. This flexibility enables developers to directly evaluate the effectiveness of new algorithms without retraining the entire model, thereby significantly improving the efficiency of the deployment process.

[0050] S102: The host computer obtains the development board model of the development board, and obtains the Vitis-AI version that is compatible with the development board model for installation.

[0051] It should be noted that in this embodiment, the development board is a circuit board used for embedded system development, including hardware components such as processor, memory, input and output interfaces, etc., which provides an actual operating environment for the development and testing of the rapid deployment system in the present invention.

[0052] S103. The development board obtains the Vitis-AI version and selects a system image that is consistent with the Vitis-AI version and burns it into its own SD card.

[0053] S104. The host computer calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board.

[0054] Specifically, during the quantization process, in order to improve the accuracy of the quantization model, vai_q_onnx will use the calibration data set to calibrate the activation and adjust the quantization parameters so that the quantized fixed-point .onnx file can better reflect the characteristics of the original data and better map high-precision values ​​to low-precision values, thereby reducing the consumption of hardware resources while maintaining the accuracy of the preset target detection model. No back propagation is required, and 100 unlabeled images need to be provided as a calibration data set.

[0055] Key code of quantization step:

[0056]

[0057] Among them, pytorch_nndct is a module in the Vitis AI toolchain, which is specifically used for the quantization, compilation and deployment of PyTorch models. These functions provide basic operators used in quantized inference, which are specially optimized for hardware accelerators such as DPU. This code shows the configuration process of statically quantizing the .onnx floating-point model into a .onnx file, specifying multiple key parameters, and finely controlling the quantization operation and subsequent related optimization processing. Among them, quant_format is set to vai_q_onnx.VitisQuantFormat.QDQ, which specifies that the quantization format is QDQ, which means that quantization (QuantizeLinear) and dequantization (DeQuantizeLinear) operation nodes are inserted into the model to achieve the quantization effect. calibrate_method is set to vai_q_onnx.PowerOfTwoMethod.MinMSE, which determines the calibration method. The MinMSE method aims to minimize the mean square error caused by the data representation before and after quantization, so that the quantized model has as little accuracy loss as possible, and at the same time uses the power of two rule to adapt the hardware, which will have higher computational efficiency. The activation_type is set to QuantType.QUInt8. Since the activation function is ReLU, negative numbers do not need to be represented, so the quantized data range is between 0 and 255. The weight_type is set to QuantType.QInt8, which specifies that the quantization type of the weight data in the model is a signed 8-bit integer (QInt8) to reduce the storage and computational complexity of the model, while maintaining reasonable accuracy. The optimize_model is set to True, which means that during or after the quantization process, the model will be subjected to operations such as removing redundant nodes and merging computational graphs, so that the quantized model can have better performance in terms of inference speed.

[0058] S105, the development board obtains the executable file compiled by the host computer, and maps the fixed-point .onnx file to the development board under the control of the system image and the executable file.

[0059] In an embodiment of the present invention, the executable file is a coding program input by the programmer based on the VOE platform installed on the host computer, which is used to control and manage the operation of the fixed-point .onnx file on the development board. Specifically, the executable file achieves optimal performance by allocating tasks between the CPU and the DPU. The first step of the executable file is to load and parse the preset target detection model in the .onnx format to generate the graph structure of the model. After that, according to the operator type supported by the DPU, the specific computing operations in the preset target detection model will be mapped to the DPU for operation. Finally, the post-processing part is to post-process the data output by the preset target detection model, including decoding the classification results, extracting the bounding box coordinates, converting the rotation angle components, and performing non-maximum suppression (NMS) and other operations.

[0060] The VOE platform in this embodiment is a platform based on the Vitis AIONNX Runtime Engine. The executable file converts the trained deep learning model (ONNX format) through the quantization, compilation and other tool chains of the VOE platform into microcode that can be executed on a specific hardware accelerator (DPU), thereby achieving rapid target detection of remote sensing image data.

[0061] The embodiment of the present invention provides a rapid deployment method for remote sensing image rotation target detection, which is applied to a rapid deployment system. The rapid deployment system includes: a server, a host computer and a development board, including: through the quantization tool and executable file of Vitis AI, the remote sensing image target detection model is efficiently mapped to the DPU for execution, thereby maximizing the use of FPGA computing resources and improving computing efficiency; during the mapping period, by optimizing and equivalently replacing operators not supported by the DPU, it is ensured that the preset target detection model can all run on the DPU, and the effective allocation of operators is achieved; secondly, the ONNX model format and the interface of Vitis AI are adopted to enhance the compatibility and scalability of the preset target detection model, so that the preset target detection model can be seamlessly migrated and deployed on different frameworks and hardware platforms; finally, the present invention simplifies the deployment process of the preset target detection model on hardware, reduces the technical threshold and deployment cost of developers, and improves the real-time performance of the system, making it more suitable for remote sensing image target detection scenarios with high efficiency, strong real-time performance and flexible and changeable task requirements.

[0062] Optionally, after S105, the following steps are further included:

[0063] The host computer obtains the remote sensing image to be detected and sends the remote sensing image to be detected to the development board;

[0064] The development board loads and parses the fixed-point .onnx file to generate the model graph structure;

[0065] Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure that satisfies the operator type to the remote sensing image to be detected to the DPU for processing;

[0066] The DPU outputs the target detection results for the remote sensing image to be detected.

[0067] In addition, it should be noted that the action of loading and parsing the fixed-point .onnx file and generating the model graph structure can be specifically controlled and executed by an executable file.

[0068] Among them, the preset target detection model makes equivalent replacements for the SiLU activation function, slice operator, and mul operation in the initial yolov8-obb ​​model.

[0069] Optionally, the preset target detection model adopts an improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts a ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; a preset convolution process is used to equivalently replace the slice operator in the initial yolov8-obb ​​model; and a batch normalization BN process is used to equivalently replace the mul operation in the initial yolov8-obb ​​model.

[0070] Optionally, the host computer further adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

[0071] It should be noted that, in the embodiment of the present invention, the quantization node QuantizeLinear and the dequantization node DeQuantizeLinear can be exemplarily added between the convolution layer, the pool layer, the add layer and the concat.

[0072] In an embodiment of the present invention, QuantizeLinear and DeQuantizeLinear nodes are inserted into the original floating-point model. These nodes simulate the quantization behavior on actual hardware, but maintain the differentiability of the calculation, allowing gradients to propagate through them. The QuantizeLinear node converts floating-point values ​​to lower-precision integer representations (e.g., int8). The DeQuantizeLinear node is responsible for converting these quantized integers back to floating-point numbers. This mode is typically used for models in onnx format, which can provide more refined control over the quantization process and better maintain the accuracy of the model.

[0073] Optionally, the preset object detection model adds unsqueeze layers between the preset layers of the initial yolov8-obb ​​model to expand the dimension of the 3D tensor of the initial yolov8-obb ​​model.

[0074] It should be noted that, specifically, an unsqueeze layer can be added after the convolution layer in the initial yolov8-obb ​​model that does not meet the data arrangement rules of the development board.

[0075] In addition, in order to reduce the amount of calculation, the confidence threshold can be set to 0.3, and the non-maximum suppression nms threshold can be set to 0.5. The detection frame is first screened with a threshold of 0.3, and then the non-maximum suppression method is used to filter out redundant frames.

[0076] In order to verify the effectiveness of a rapid deployment method for detecting rotating targets in remote sensing images provided by an embodiment of the present invention, a simulation experiment was also conducted in the embodiment of the present invention.

[0077] 1. Simulation experiment conditions:

[0078] The hardware of the simulation experiment of the embodiment of the present invention includes: zcu102 board, serial port cable, network cable, and SD card.

[0079] The software platform for the simulation experiment of the embodiment of the present invention is: Ubuntu20.04, Vitis-AI3.5.finalshell4.3.

[0080] 2. Simulation content:

[0081] The test data of the embodiment of the present invention is 200 test images randomly selected from the DOTA-1.0 data set, with a size of 1024×1024. The GPU model used is NVIDIAGeForce GTX TITAN X, and the CPU model is Intel(R)Xeon(R)E5-2630 v4@2.20GHz. The original yolov8-obb ​​model and the preset target detection model of the embodiment of the present invention are used for comparative experiments, and the comparison results are shown in Table 1.

[0082] Table 1 Comparison of the detection results of the method of the present invention and the original yolov8-obb ​​model

[0083]

[0084] Based on Table 1, it can be seen that by adopting the method of the present invention, after quantization, the model size is reduced to 35% of the previous size, which greatly saves storage space. In terms of time consumption, compared with GPU devices, due to the limited computing power and hardware resources of edge devices, the FPS index is lower than that of GPU devices during inference. Compared with CPU devices, the DPU in the hardware has a computing engine specifically responsible for executing common mathematical operations in deep learning such as matrix multiplication and convolution, and an efficient data flow controller that can effectively utilize various resources on the FPGA. The CPU is limited by the number of cores and the main frequency, and the FPS is significantly lower. In addition, due to the existence of quantization errors and the differences in numerical accuracy between different platforms, the board-side results are 1.7% lower than the PC-side performance.

[0085] Figure 2 The comparison results of testing two images on the pytorch side and the development board side are shown as an example. Specifically, Figure 2 The two pictures on the left are the target detection results on the pytorch side based on the existing yolov8-obb ​​model. Figure 2 The two figures on the right are the target detection results on the development board based on the preset target detection model of the present invention. The processing conditions and experimental objects of the two methods are the same. Figure 2 It can be seen that the bounding box positions and sizes given by the two are basically the same, which shows that after the model is optimized and ported to a dedicated hardware accelerator, it can still maintain the original performance, verifying the effectiveness of the method of the present invention.

[0086] The rapid deployment method for remote sensing image rotation target detection provided by the embodiment of the present invention has the following technical advantages:

[0087] 1. Maximizing resource utilization: The deployment methods mentioned in the prior art do not place as many low-power and high-efficiency computing tasks as possible on the FPGA for execution based on the resource characteristics of the current heterogeneous platform. The present invention quantizes the model through Vitis AI's quantization tool vai_q_onnx, and encodes the host file based on VOE (Vitis AIONNXRuntime Engine) to map the model's control flow and data flow to the hardware platform. This ensures that the model can run efficiently on the DPU (a dedicated processing unit on the FPGA), thereby maximizing the use of FPGA computing resources.

[0088] 2. Effective allocation of operators: The prior art does not take into account that different computing units (DPU, CPU) have different optimal data layouts, and effectively allocates operators in the model according to hardware characteristics. Before the model is deployed, the present invention optimizes the floating-point .onnx file, modifies or replaces some layers that are not supported by the DPU, and ensures that the model can be fully mapped to the DPU. This step ensures that the operators in the model are effectively allocated to the hardware units that are most suitable for their execution, thereby giving full play to the advantages of the hardware.

[0089] 3. Model deployment flexibility: Although the deployment methods mentioned in the prior art propose acceleration methods based on the Vitis stack, they do not fully consider the deployment flexibility of the model on different hardware platforms. The present invention uses the ONNX model format to realize the sharing of models between different deep learning frameworks (such as TensorFlow, PyTorch, etc.), and imports the model to the Xilinx series hardware for optimization and deployment through the vai_q_onnx interface of Vitis AI. This improves the flexibility and compatibility of model deployment.

[0090] In summary, the present invention significantly improves the computational efficiency of remote sensing image target detection by deploying the model on the DPU and utilizing the low power consumption and high energy efficiency characteristics of FPGA. This solution effectively allocates the operators in the model according to the hardware characteristics, ensures the maximum utilization of computing resources, and avoids unnecessary waste of resources. Through the ONNX model format and the vai_q_onnx interface of Vitis AI, the seamless migration and deployment of the model between different frameworks and hardware platforms are realized, and the compatibility and scalability of the model are enhanced. Through the complete set of tools, libraries and optimization models provided by Vitis AI, the deployment process of the model on hardware is simplified, and the technical threshold and deployment cost of developers are reduced. Efficient model deployment and resource utilization enable the remote sensing image target detection network to adapt to military application scenarios with high efficiency, strong real-time performance, and flexible and changeable task requirements, thereby improving the overall performance and real-time response capabilities of the system.

[0091] Based on the same inventive concept, an embodiment of the present invention also provides a rapid deployment system for detecting rotating targets in remote sensing images, the system comprising: a server, a host computer and a development board;

[0092] The server is used to train a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, and convert it into a floating-point .onnx file using the deep learning framework pytorch, and send the floating-point .onnx file to the host computer;

[0093] The host computer is used to obtain the development board model of the development board, obtain the Vitis-AI version that is compatible with the development board model, and perform installation. It calls the vai_q_onnx tool in Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board.

[0094] The development board is used to obtain the Vitis-AI version and the executable file compiled by the host computer, and select the system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and executable file, the fixed-point .onnx file is mapped to the development board; among them, the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model.

[0095] Figure 3 A schematic diagram of the structure of a rapid deployment device for rotating target detection of remote sensing images provided by an embodiment of the present invention includes: a processor 310, a storage medium 320 and a bus 330, wherein the storage medium 320 stores machine-readable instructions executable by the processor 310, and when the rapid deployment device for rotating target detection of remote sensing images is running, the processor 310 communicates with the storage medium 320 via the bus 330, and the processor 310 executes the machine-readable instructions to execute the steps of the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0096] The storage medium may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk storage. Optionally, the storage medium may also be at least one storage device located away from the aforementioned processor.

[0097] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0099] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art can understand and implement other changes of the above disclosed embodiments by viewing the drawings and the disclosed content. In the description of the present invention, the term "comprising" does not exclude other components or steps, "one" or "an" does not exclude multiple situations, and the meaning of "multiple" is two or more, unless otherwise clearly and specifically limited. In addition, certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0100] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A rapid deployment method for remote sensing image rotation target detection, applied to a rapid deployment system, the rapid deployment system comprising: The server, host computer and development board are characterized by including: The server obtains a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, converts the file into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to the host computer; The host computer obtains the development board model of the development board, and obtains the Vitis-AI version adapted to the development board model for installation; The development board obtains the Vitis-AI version, and selects a system image consistent with the Vitis-AI version and burns it into its own SD card; The host computer calls the vai_q_onnx tool in the Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and sends the fixed-point .onnx file to the development board; The development board obtains the executable file compiled by the host computer, and under the control of the system image and the executable file, maps the fixed-point .onnx file to the development board; wherein the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model.

2. The rapid deployment method for remote sensing image rotation target detection according to claim 1 is characterized in that: Before the server sends the floating-point .onnx file to the host computer, the server also includes: The onnx-simplifier tool in the deep learning framework pytorch is called to remove redundant nodes in the floating-point .onnx file.

3. The rapid deployment method for remote sensing image rotation target detection according to claim 1 is characterized in that: The preset target detection model adopts an improved yolov8-obb ​​model, and the improved yolov8-obb ​​model adopts the ReLU activation function to equivalently replace the SiLU activation function in the initial yolov8-obb ​​model; adopts the preset convolution processing to equivalently replace the slice operator in the initial yolov8-obb ​​model; and adopts the batch normalization BN processing to equivalently replace the mul operation in the initial yolov8-obb ​​model.

4. The rapid deployment method for remote sensing image rotation target detection according to claim 1 is characterized in that: The server obtains a floating-point .pth file based on a preset target detection model training, converts it into a floating-point .onnx file using a deep learning framework pytorch, and sends the floating-point .onnx file to the host computer, including: The server obtains an initial remote sensing image sample; The initial remote sensing image samples are segmented by a sliding window method to obtain a remote sensing image sample set of a preset size; Inputting the remote sensing image sample set into the preset target detection model for training to obtain the floating point .pth file; The floating-point .pth file is converted into a floating-point .onnx file using the deep learning framework pytorch, and the floating-point .onnx file is sent to the host computer.

5. The rapid deployment method for remote sensing image rotation target detection according to claim 1 is characterized in that: The host computer also adds a quantization node QuantizeLinear and a dequantization node DeQuantizeLinear between each layer of the fixed-point .onnx file.

6. The rapid deployment method for remote sensing image rotation target detection according to claim 1 is characterized in that: The development board obtains the executable file compiled by the host computer, and after mapping the fixed-point .onnx file to the development board under the control of the system image and the executable file, further includes: The host computer acquires the remote sensing image to be detected, and sends the remote sensing image to be detected to the development board; The development board loads and parses the fixed-point .onnx file to generate a model graph structure; Input the remote sensing image to be detected into the model graph structure, and according to the operator type supported by the corresponding DPU on the development board, map the corresponding operation of the model graph structure satisfying the operator type on the remote sensing image to be detected to the DPU for processing; The DPU outputs a target detection result for the remote sensing image to be detected.

7. The rapid deployment method for remote sensing image rotation target detection according to claim 1, characterized in that: The preset target detection model adds an unsqueeze layer between the preset layers of the initial yolov8-obb ​​model to expand the dimension of the 3D tensor of the initial yolov8-obb ​​model.

8. A rapid deployment system for remote sensing image rotation target detection, characterized in that: include: Server, host computer and development board; The server is used to obtain a floating-point .pth file based on a preset target detection model and a remote sensing image sample set, and convert it into a floating-point .onnx file using a deep learning framework pytorch, and send the floating-point .onnx file to the host computer; The host computer is used to obtain the development board model of the development board, obtain the Vitis-AI version adapted to the development board model for installation, call the vai_q_onnx tool in the Vitis-AI to perform PTQ quantization on the floating-point .onnx file to obtain a fixed-point .onnx file, and send the fixed-point .onnx file to the development board; The development board is used to obtain the Vitis-AI version and the executable file compiled by the host computer, and select a system image consistent with the Vitis-AI version to burn into its own SD card. Under the control of the system image and the executable file, the fixed-point .onnx file is mapped to the development board; wherein the preset target detection model performs equivalent replacement of the SiLU activation function, slice operator and mul operation in the initial yolov8-obb ​​model.

9. A rapid deployment device for detecting rotating targets in remote sensing images, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the rapid deployment device for rotating target detection of remote sensing images is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the rapid deployment method for rotating target detection of remote sensing images as described in any one of claims 1 to 7.

Images